Senior ML Test Engineer

Salary: $70,000 - 110,000 per year Requirements:

• Bachelors degree in Computer Science, Engineering, or equivalent practical experience
• 7+ years of professional experience in software engineering or QA for ML/AI systems
• Strong Python programming ability, including test automation work
• Familiarity with CAD environments and related tooling
• Proficiency with automation and UAT test suites/frameworks
• Experience building QA frameworks or platforms used across multiple teams
• Strong problem-solving skills and keen attention to detail
• Excellent communication and collaboration abilities
• Understanding of software architecture and design patterns
• Ability to thrive in an agile development environment
• Experience with data validation tools or labeling workflows
• Familiarity with ML frameworks such as PyTorch or TensorFlow
• Experience with CI/CD tools and processes
• Experience with data pipelines and orchestration tools such as Airflow or Metaflow
• Familiarity with MLOps practices, including monitoring, drift detection, and deployment checks
• Experience with ML evaluation methods, metrics, and benchmarking
• Interest in learning new technologies and improving existing systems
• Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform
• Experience testing ML services in production environments
• Knowledge of experiment tracking tools such as Comet, MLflow, or Weights & Biases
• Initiative to propose solutions and develop new technologies
• Comfort building QA systems from the ground up and writing maintainable automation
• Ability to collaborate effectively across global locations
• Comfort working in ambiguous, fast-evolving areas
• Ability to build scalable, maintainable systems that others will rely on
• Strong interpersonal communication skills

• Define our ML quality strategy and set acceptance criteria across data, model, and system levels
• Design and maintain model evaluation suites, metrics, and test datasets
• Evaluate CAD RL model outputs for geometric validity and policy stability
• Create structured rubrics that convert qualitative observations into measurable evaluation gates
• Test ML models from the product side
• Perform API testing
• Automate ML QA workflows using Python and CI/CD tools such as GitHub Actions and Jenkins
• Build and maintain test harnesses for ML services and APIs
• Coach teams on ML QA best practices and consistent evaluation standards
• Establish quality gates for training and deployment pipelines, including regression checks and drift detection
• Contribute to multi-team projects and shared codebases while maintaining code quality and consistency
• Participate in code reviews and provide constructive peer feedback
• Document and present findings and ideas across our organization

• AI
• Airflow
• API
• AWS
• Azure
• CI/CD
• Cloud
• GitHub
• Jenkins
• Machine Learning
• MLflow
• MLOps
• PyTorch
• Python
• TensorFlow
• Support

More:

We are Autodesk, a global hybrid-first company with teams in London, San Francisco, Toronto, and remote locations. We are hiring a Senior Machine Learning Test Engineer for our Research Enablement team, reporting to an Engineering Manager. This role is based on the United States East Coast time zone and sits at the intersection of research and product, where we focus on dependable evaluation of ML models and data, strong quality gates, and scalable automation. We are committed to building an inclusive, diverse workplace and welcome qualified applicants in line with our equal opportunity approach.

last updated 33 week of 2026