Senior Machine Learning Test Engineer (Boston)

Job Requisition ID #26WD98377Senior Machine Learning Test EngineerLocation: United States East CoastPosition OverviewAs a Senior Machine Learning Test Engineer in the Research Enablement team, you will work side-by-side with researchers, Machine Learning developers and developers, and software developers and developers to define and uphold quality standards for ML systems. You are a quality-focused developer who is passionate about reliable, repeatable evaluation of ML models and data. Your skills span test strategy, automation, and a little MLOps, with a strong software engineering base. You are excited to collaborate across research and product to ship ML capabilities with clear quality gates. You are comfortable working at the intersection of research and product and are competent in using Autodesk CAD software.Reporting Structure: You will report to an Engineering Manager in Research Enablement.Location: United States, East Coast Time Zone.We are a global team, located in London, San Francisco, Toronto, and remotely. Autodesk is a hybrid-first company, allowing workers to work remotly, in an office, or a mix of both.ResponsibilitiesDefine ML quality strategy and acceptance criteria across data, model, and system levelsDesign and maintain model evaluation suites, metrics, and test datasetsEvaluating CAD RL model outputs for geometric validity or policy stabilityDefining structured rubrics that translate qualitative findings into measurable evaluation gatesTesting ML Models from product sideAPI TestingAutomate ML QA workflows using Python and CI/CD (e.g., GitHub Actions, Jenkins)Create and maintain test harnesses for ML services and APIsMentor teams on ML QA best practices and consistent evaluation standardsBuild quality gates for training and deployment pipelines (e.g., regression checks, drift detection)Contribute to multi-team projects and codebases, ensuring code quality and consistencyParticipate in code reviews and provide constructive feedback to peersDocument and present findings and ideas across the companyMinimum QualificationsBachelor’s degree in Computer Science, Engineering, or equivalent experience7+ years of professional experience in software engineering or QA for ML/AI systemsStrong programming skills in Python, with experience in test automationFamiliarity with popular CAD environments toolingProficient in Automation and UAT test suite/frameworkExperience designing QA frameworks or platforms used by multiple teamsExcellent problem-solving skills and attention to detailStrong communication and collaboration skillsUnderstanding of software architecture and design patternsAbility to work in an agile development environmentPreferred QualificationsExperience with data validation tooling (e.g., Great Expectations) or labeling workflowsFamiliarity with ML frameworks (e.g., PyTorch, TensorFlow)Experience with CI/CD tools and processesExperience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)Familiarity with MLOps practices (model monitoring, drift, deployment checks)Experience with ML evaluation methods, metrics, and benchmarkingPassion for learning new technologies and improving existing systemsExperience with cloud providers (e.g., AWS, Azure, Google Cloud Platform)Experience testing ML services in production environmentsKnowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases)The Ideal CandidateYou demonstrate initiative to provide solutions and to learn and develop new technologiesComfortable building QA systems from scratch and writing maintainable automationYou enjoy learning and collaborating across global locationsYou are comfortable working in newly forming ambiguous areasYou are comfortable building scalable and maintainable systems that will be relied on by othersYou can communicate well with othersAt Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.Are you an existing contractor or consultant with Autodesk? Please search for open jobs and apply internally (not on this external site). If you have any questions or require support, contact Autodesk Careers.SummaryLocation: Boston, MA, USA; New Jersey, USA - Remote; New Hampshire, USA - Remote; Massachusetts, USA - Remote; West Virginia, USA - Remote; Washington, DC, USA; Vermont, USA - Remote; Novi, MI, USA; Atlanta, GA, USA; District of Columbia, USA - Remote; Maine, USA - Remote; Maryland, USA - Remote; Ohio, USA - Remote; North Carolina, USA - Remote; New York, USA - RemoteType: Full time