Lead Machine Learning Ops Engineer

Lead Machine Learning Ops Engineer

Location: Houston, TX Duration: 6+ months Probably long term

Lead Machine Learning Ops Engineer Must have 10 plus years and solid DevOps and Client experience

As a Lead Machine Learning Ops Engineer, you will play a pivotal role in implementing DevOps and Client Ops practices within the Corporate Data & Analytics Team to support AI/Client application enablement across all operating companies. Your primary responsibility will be to drive the adoption of best practices in DevOps and Client Ops, accelerating the deployment of AI/Client and data-driven solutions that meet the business needs.

We seek a motivated and skilled individual with a strong background in DevOps and Client Ops, a deep understanding of Infra Ops, and solid knowledge of AI/Client data and analytics cloud services and components. You will collaborate closely with data scientists, machine learning engineers, data engineers, software engineers, and platform architects, utilizing the latest tools and technologies to deploy and maintain AI/Client and advanced analytics solutions, as well as integrate analytic models with existing business applications.

Skills and Experience

• Bachelor's Degree Computer Science, Computer Engineering, Information Technology, Software Engineering or equivalent technical discipline and 10+ years of experience in software engineering with a strong background in DevOps and Infrastructure as Code, supporting Machine Learning and Data Science workloads preferred. or
• Master's Degree Computer Science, Computer Engineering, Information Technology, Software Engineering or equivalent technical discipline and 5+ years of experience in software engineering with a strong background in DevOps and Infrastructure as Code, supporting Machine Learning and Data Science workloads preferred.
• Expertise on code versioning tools, such as Gitlab, GitHub, Azure DevOps, Bitbucket etc., GitHub Preferred, familiar with branch level code repository management.
• Experience deploying Machine Learning solutions on cloud platforms (e.g., AWS, Azure, or GCP ). Databricks, and AWS Preferred.
• Proficient with GitHub actions to automate testing and deployment of data and Client workloads from CI/CD provider to Databricks.
• Strong knowledge of infrastructure automation tools such as Terraform, Ansible, CloudFormation etc.
• Experience with data processing frameworks/tools/platform such as Databricks, Apache Spark, Kafka, Flink, AWS cloud services for batch processing, batch streaming and streaming.
• Experience containerizing analytical models using Docker and Kubernetes or other container orchestration platforms.
• Technical expertise across all deployment models on public cloud, private cloud, and on-premises infrastructure.

Responsibilities

• Develop automated build and deployment processes to enable continuous delivery of software releases, enhance the existing CI/CD pipelines for AIML application development and deployment.
• Collaborate with data scientists, data engineers, data analysts, software engineers, IT specialists, and stakeholders to accelerate deployment of AI applications via CI/CD pipelines and maintain the SLAs of those applications at the centralized platform.
• Design, develop and maintain infrastructure using infrastructure as code tools such as Terraform, Ansible, CloudFormation etc.
• Templatize existing Databricks CLI codes to manage Databricks platform as code for AIML data pipelines (batch processing, batch streaming and streaming) and model serving endpoints.
• Enhance the existing DevOps practices to improve the overall AIML application development lifecycle.
• Work closely with cross-functional teams to ensure that applications are highly available and scalable.
• Collaborate with development teams and cloud platform team to ensure that infrastructure meets the requirements of the application.
• Establish and maintain best practices for cloud security, compliance, and cost optimization.