Data Engineer - AI & Machine Learning

Description:


Role: Data Engineer - Artificial Intelligence & Machine Learning

Location Options: Bay Area - CA

Responsibilities: -

1. Develop AI/ML Models:

•Design, build, and train machine learning models using appropriate algorithms (e.g., supervised, unsupervised, reinforcement learning, deep learning).

•Use various machine learning and AI frameworks (TensorFlow, PyTorch, Scikit-learn, Keras, etc.) to implement models.

•Experiment with different approaches (e.g., decision trees, neural networks, ensemble methods) and optimize for the best performance.

•Perform model selection, training, tuning, and validation using real-world data to achieve the most accurate results.

2. Data Preparation & Feature Engineering:

•Clean, preprocess, and structure raw data for analysis, ensuring it is suitable for model training.

•Implement data augmentation techniques, handle missing data, and remove outliers.

•Engineer features that will improve model performance, understanding the data's underlying relationships.

3. Algorithm Design and Optimization:

•Develop and optimize algorithms for specific use cases like image recognition, natural language processing (NLP), speech recognition, or recommendation systems.

•Optimize algorithms for high efficiency, scalability, and real-time performance.

•Regularly assess and improve model accuracy by experimenting with various hyperparameters, architectures, and optimization techniques.

4. Deploy Machine Learning Models:

•Collaborate with software engineers to deploy machine learning models into production environments, integrating them with existing systems.

•Ensure the models are scalable, performant, and able to handle real-time or batch data as required.

•Implement model monitoring and performance tracking tools to evaluate accuracy and detect any model drift over time.

5. Model Evaluation & Testing:

•Use cross-validation and other techniques to evaluate the model's generalization capabilities.

•Implement performance metrics to measure model accuracy, precision, recall, F1-score, and other relevant metrics based on project needs.

•Perform A/B testing and compare the performance of multiple models.

6. Continuous Improvement & Research:

•Stay up-to-date with the latest AI/ML research and advancements in the field, such as new algorithms, architectures, and technologies.

•Participate in code reviews and contribute to best practices in AI/ML development.

•Experiment with new AI and machine learning techniques to continually improve performance and solve complex problems.

7. Collaboration & Communication:

•Work closely with Data Scientists, Software Engineers, Product Managers, and other stakeholders to understand business problems and translate them into machine learning tasks.

•Communicate findings, insights, and progress to non-technical stakeholders in a clear, understandable manner.

•Collaborate on projects, providing expertise on AI/ML concepts to help shape product features or solutions.

8. Ethical Considerations and Bias Mitigation:

•Ensure that the models and algorithms are free from biases and ethically sound, particularly when dealing with sensitive data.

•Evaluate fairness, transparency, and interpretability of models, especially in critical applications like healthcare, finance, and legal sectors.

9. Documentation:

•Document the model-building process, algorithm choice, and data used, ensuring reproducibility and transparency.

•Write clear technical documentation and user guides to facilitate collaboration and knowledge transfer.

10. Innovation and Prototyping:

•Prototype AI-driven solutions to demonstrate their potential and feasibility.

•Develop proof of concepts (PoCs) and new algorithms for emerging AI and ML technologies (e.g., federated learning, reinforcement learning, generative models)

Qualifications:


1.Educational Background:

Bachelor's or Master's degree in Computer Science, Data Science, Artificial Intelligence, Mathematics, or a related field.

Ph.D. in a relevant field is a plus but not required

2.Technical Skills:

•Strong programming skills in Python, R, or similar languages for machine learning and data analysis.

•Deep knowledge of machine learning libraries such as TensorFlow, PyTorch, Scikit-learn, Keras, and XGBoost.

•Strong foundation in linear algebra, probability, statistics, and optimization techniques.

•Proficiency in algorithms and data structures.

• Experience with deep learning techniques, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and reinforcement learning.

•Familiarity with NLP techniques, computer vision, time series analysis, and other AI sub-domains.

•Knowledge of data preprocessing, feature extraction, and feature selection techniques.

•Proficiency in cloud platforms (AWS, Azure, Google Cloud) for model training and deployment.

•Experience with containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.

•Familiarity with big data technologies (Hadoop, Spark, etc.) is a plus.

3.Soft Skills:

•Strong problem-solving and analytical skills.

•Ability to work in a collaborative, cross-functional environment.

•Effective communication skills to explain complex AI/ML concepts to non-technical stakeholders.

•Strong attention to detail and ability to troubleshoot and debug code.

•Passion for continuous learning and staying up-to-date with AI and ML advancements.

4.Experience:

•Proven experience (3+ years) in developing and deploying machine learning or AI models in a production environment.

•Familiarity with MLOps (machine learning operations) principles, such as model versioning, CI/CD pipelines for ML, and model monitoring in production

5.Preferred Qualifications:

•Experience with reinforcement learning, unsupervised learning, or generative models (e.g., GANs).

•Knowledge of ethics in AI, such as mitigating bias and ensuring fairness in models.

Familiarity with NLP libraries like SpaCy, NLTK, Hugging Face Transformers, etc.

•Experience in building and deploying AI-powered products in a commercial setting.

•Knowledge of edge computing and deploying AI models on edge devices

6.Work Environment:

•Collaborative and fast-paced work environment.

•Opportunity to work with state-of-the-art technologies.

•Supportive and dynamic team culture

•The position may require collaborating across multiple teams, including product, engineering, and research groups, to develop and implement AI/ML solutions

#LI-AD1