Azure Databricks Machine Learning Engineer

ROLE: Azure Databricks Machine Learning Engineer


Job Description


We are looking for an experienced Azure Databricks ML Engineer with a strong background in machine learning model deployment. The ideal candidate will have at least 7 years of experience and be proficient in PySpark, Python, and SQL. A solid understanding of statistical analysis is a plus. Applies engineering disciplines to cloud computing and brings a systematic approach to concerns of commercialization, standardization, and governance of cloud computing applications. Leverages the methods and tools of engineering in conceiving, developing, operating and maintaining cloud computing systems and solutions. Uses the economics of cloud resources to solve business problems.


Key Responsibilities:

• Design, develop, and deploy machine learning models using Azure Databricks.
• Implement and manage Azure Databricks ML Flow for tracking and managing ML experiments.
• Apply ML Ops best practices to ensure robust and scalable model deployment.
• Deploy and manage Large Language Models (LLMs) and Generative AI (GenAI) models on Databricks.
• Collaborate with data scientists and engineers to integrate ML models into production environments.
• Optimize and maintain existing ML models to ensure high performance and scalability.
• Utilize PySpark, Python, and SQL for data processing and analysis.
• Conduct statistical analysis to support data-driven decision-making.
• Troubleshoot and resolve issues related to ML model deployment and performance.
• Stay up-to-date with the latest advancements in machine learning and cloud technologies.

Qualifications:

• 7+ years of experience in ML model deployment using Azure Databricks.
• Proficiency in PySpark, Python, and SQL.
• Experience with Azure Databricks ML Flow and ML Ops best practices.
• Experience with deploying and managing LLMs and GenAI models on Databricks.
• Experience with statistical analysis is a plus.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork abilities.
• Bachelor's degree in Computer Science, Data Science, or a related field (Master's preferred)

Technical Skills

• 3+ years of experience with creating stored procedures in SQL
• Azure Databricks
• Azure Databricks machine learning certificate
• Bayesian Machine Learning
• Machine Learning Operations
• pandas
• PySpark
• Quantitative Statistical Analysis