Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Machine Learning / Data Engineer - Evaluation

VTI Aerospace

Job Description

Job Description

About VTI Aerospace

VTI Aerospace builds AI-powered perception and pilot assist technologies to unlock the future of aviation. With offices in Bozeman, MT and Seattle, WA, we are a team of engineers and technologists from Boeing, Airbus, Aurora, and beyond. We are passionate about pushing the boundaries of autonomy and aviation safety. Our company is dedicated to advancing the field of aviation with cutting-edge solutions

The Role

As a Senior Software Engineer – Evaluation, you will design and implement systems that measure and monitor the performance of our computer vision, automatic speech recognition (ASR), and small language model (SLM) systems. You will develop evaluation methodologies, benchmarking pipelines, and monitoring tools that ensure our AI systems perform reliably in real-world environments. You will help establish the evaluation standards and performance benchmarks that guide the development of all AI systems at the company.

You will work closely with machine learning and data engineering teams to evaluate model performance, identify failure modes, and guide improvements to data collection and model training, helping bring AI-powered aviation tools to market.

This role is ideal for someone who enjoys designing robust evaluation systems and uncovering insights from model performance data, and wants to have a direct impact on the quality and reliability of our AI systems.

What You'll Do

  • Define and implement evaluation methodologies for computer vision, ASR, and language systems
  • Identify and track key performance indicators (KPIs) that measure system and model effectiveness
  • Perform dataset coverage analysis to understand strengths, gaps, and biases in training and evaluation data
  • Identify model deficiencies and collaborate with ML engineers to improve training data and model performance
  • Build scalable model evaluation pipelines in Python for automated benchmarking and regression testing
  • Design and maintain systems for monitoring model performance and drift in production environments
  • Architect data models and storage systems for evaluation results using relational and time-series databases
  • Build dashboards and reporting tools to visualize model performance and evaluation metrics
  • Collaborate with ML and data engineering teams to improve evaluation workflows and data pipelines
  • Contribute to ground-truth data pipelines and processes that support reliable evaluation

What We're Looking For

  • 5+ years of experience developing production software systems in Python
  • Strong experience designing and implementing data processing or analysis pipelines
  • Experience building systems for machine learning evaluation, experimentation, or benchmarking
  • Experience working with large datasets and building scalable data workflows
  • Familiarity with statistical methods, experimental design, and model performance analysis
  • Experience building automated pipelines using tools such as Airflow or similar orchestration frameworks
  • Experience working with evaluation or experiment tracking tools such as MLflow or similar systems
  • Experience designing data models and working with relational databases and time-series data
  • Strong collaboration skills and ability to work closely with machine learning and data engineering teams

Bonus Points

  • Experience working with computer vision, ASR, or language models in production environments
  • Experience using PyTorch or other deep learning frameworks
  • Experience evaluating multimodal or large-scale AI systems
  • Experience building monitoring systems for ML models in production
  • Experience designing dataset analysis or data quality tooling
  • Experience building dashboards or monitoring tools using Grafana or similar platforms

Why You'll Love Working Here

  • Opportunity to work on meaningful problems in aviation and deliver products to industry at a rapid pace
  • Small team with significant impact on company direction
  • Collaborative and innovative environment
  • Competitive compensation and benefits
  • Opportunities for growth and leadership

Location

Work arrangement: On-site

Primary location: Seattle or Bozeman

Compensation & Benefits

Salary range: DOE

Benefits may include:

• Health, dental, and vision insurance
• Retirement plan or 401(k)
• Paid time off and holidays
• Professional development opportunities

Equal Opportunity Statement

VTI is proud to be an equal opportunity employer. We value diverse perspectives and encourage candidates from all backgrounds to apply

Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Senior Machine Learning / Data Engineer - Evaluation in Seattle, WA vacancy
  • $150.75k

     ...impacting the world for the greater good? As an ML Engineer at Axon, you will contribute to developing AI...  ...Axon Air, VR and more) by supporting the training, evaluation, testing and deployment of machine learning models on Axon devices and cloud. As part of a multidisciplinary... 
    Senior
    Full time
    Work experience placement
    Work at office
    Remote work

    Axon

    Seattle, WA
    1 day ago
  • $177.69k - $341.73k

     ...Senior Machine Learning Engineer, Risk Data Mining - USDS The USDS-Platform and Community Integrity (PaCI) team is missioned to: Protect U.S. TikTok users, including and beyond content consumers, creators, advertisers; Secure platform health and community experience authenticity... 
    Senior
    Temporary work

    Tik Tok

    Seattle, WA
    1 day ago
  •  ...seeking an experienced software engineer with a deep focus on AI/ML to design and deliver end-to-end machine learning solutions. You will stay...  ...will lead the ML lifecycle from data collection and preprocessing to model training, evaluation, and deployment, working with... 
    Senior

    Apple

    Seattle, WA
    1 day ago
  • $172.5k - $306.63k

     ...Senior Machine Learning Engineer Adobe Stock's Content Intelligence team is positioned at the center...  ...industry experience in developing, evaluating, and deploying ML models into production...  ...of Python and Python-based ML and Data Science frameworks, with proficiency... 
    Senior
    Temporary work
    Local area
    Worldwide

    Adobe

    Seattle, WA
    1 day ago
  • $180k - $215k

     ...place. We go beyond typical data-driven approaches or pure...  ...architectures, combining rigorous engineering with learning systems proven in globally...  ...What You'll Get To Do Machine Learning Modeling...  ...architectures, optimization, and evaluation, with a strong grasp of end... 
    Senior
    Local area

    Field AI

    Seattle, WA
    3 days ago
  •  ...Machine Learning Engineer Implements machine learning (ML) models for production with minimal guidance...  ...in deployment. Identifies potential data quality, security, and/or privacy...  ...PyTorch, TensorFlow, Keras) to continuously evaluate their performance and scalability, and... 
    Senior
    Shift work

    Hackajob

    Seattle, WA
    16 hours ago
  • $185.6k - $255k

     ...Now Most customer data is a lie by omission....  ...without waiting on an engineering queue. The bet underneath...  ...teams. As a Senior ML Engineer, you'll lead...  ...our customers through machine learning products and features....  ...blameless post-mortems. Evaluate MLOps tooling that raises... 
    Senior
    Work at office
    Local area
    Remote work

    Amperity

    Seattle, WA
    4 days ago
  • $184.5k - $258k

     ...singular technology platform powered by data and machine learning provides secure, differentiated,...  ...traveler satisfaction. This Senior Machine Learning Engineer role is part of the Distribution...  ..., and model development and evaluation through implementation, testing,... 
    Senior
    Local area

    11105 Expedia, Inc.

    Seattle, WA
    19 hours ago
  • $175k - $215k

     ...Who you are We are seeking a Senior Machine Learning Engineer to play a key role to join our growing...  ...unique blend of technical expertise in data and machine learning, innovative thinking...  ...decision tool (AEDT) to assess or evaluate your candidacy for employment or... 
    Senior
    Temporary work
    Work experience placement
    Work at office
    Local area

    Metropolis Corp

    Seattle, WA
    4 days ago
  • $115k - $230k

     ...Senior ML Engineer At GEICO, we offer a rewarding career where your ambitions are met with endless...  ...with cross-functional teams, including data scientists, ML engineers, software...  ...finetuning, real-time/batch inferencing, and evaluation systems for AIML models and LLMs. ~2... 
    Senior
    Hourly pay
    Work experience placement
    Local area

    GEICO

    Seattle, WA
    2 days ago
  • $150.75k - $241.2k

     ...a positive impact at scale? As a Machine Learning Engineer at Axon , you'll help build AI solutions...  ..., and more -supporting the training, evaluation, testing, and deployment of machine...  ...scale evaluation, inference optimization, data pipelines, cloud infrastructure, and... 
    Senior
    Work experience placement
    Work at office
    Remote work

    Axon

    Seattle, WA
    2 days ago
  • $150k - $220k

     ...closely with product, engineering and business leaders to...  ...a difference with data. With access to multiple...  ...company we value curiosity, learning and growth. As an...  ...inference and machine learning practice....  ...techniques, such as Model Evaluation and Validation, Deep Learning... 
    Senior
    Work at office

    Grid

    Seattle, WA
    4 days ago
  • $114.6k - $234.6k

     ...Machine Learning Engineer Implements machine learning (ML) models for production with minimal guidance...  ...in deployment. Identifies potential data quality, security, and/or privacy...  ...PyTorch, TensorFlow, Keras) to continuously evaluate their performance and scalability, and... 
    Senior
    Temporary work
    Flexible hours
    Shift work

    Oracle

    Seattle, WA
    3 days ago
  • $232.3k - $406.6k

     ...payment flows, business and data solutions, cyber security, and...  ..., you will oversee diverse engineering teams, guiding them as they...  ...will be expected to rapidly evaluate and integrate new ideas and...  ...Experience with Generative AI and Machine Learning is essential. Prior... 
    Senior
    Work experience placement
    Work at office
    Local area
    Immediate start

    Visa

    Bellevue, WA
    5 days ago
  • $200k - $250k

     ...are Metropolis is seeking a Senior Manager of Machine Learning Engineering within the Advanced Technologies Group...  ...within the Machine Learning org:data engineering, annotation pipelines,...  ...decision tool (AEDT) to assess or evaluate your candidacy for employment or... 
    Senior
    Temporary work
    Work at office
    Local area

    Metropolis Corp

    Seattle, WA
    5 days ago
  • $160k - $250k

     ...out if you are interested in joining the future of AI! Senior Machine Learning Engineer In order to execute our vision, we need to grow our...  ...team, and is capable of planning out scalable, maintainable data pipelines. \n Responsibilities Everything involved... 
    Senior
    Full time

    Hive

    Seattle, WA
    1 day ago
  •  ...The 3D Simulation group at Zoox is looking for machine learning engineers to bring the latest research in 3D vision to improve diversity and blur...  ...simulation and reality. You will have access to the best sensor data in the world and an incredible infrastructure for testing... 
    Senior
    Full time
    Temporary work
    Relocation package

    Zoox

    Seattle, WA
    1 day ago
  • $148.5k - $223.9k

     ...Job Category Software Engineering Job Details About...  ...future of Salesforce. Senior Member of Technical Staff - Senior Machine Learning Engineering Job...  ...future of business with AI+ Data +CRM. Leading with our...  ...recruiters assess and evaluate candidates' resumes and... 
    Senior

    Salesforce.Com Inc

    Bellevue, WA
    5 days ago
  • $106.9k - $160.4k

     ...ML Engineer At Weyerhaeuser, we sustainably manage forests and...  ..., build, and operationalize machine learning solutions that are reliable,...  ...sits at the intersection of data science, software engineering...  ...development, training, evaluation, and operationalizing models... 
    Senior
    Full time
    Temporary work

    Weyerhaeuser

    Seattle, WA
    3 days ago
  • $170k

     ...Sr. Machine Learning Engineer Seattle, Washington, United States At PitchBook...  ...and grow with us! As a Senior Machine Learning Engineer (...  ...and unstructured data including reports, news, and...  ...transparency, monitoring, evaluation, and compliance. Help maintain... 
    Senior
    Visa sponsorship

    PitchBook

    Seattle, WA
    1 day ago
  •  ...impact via measurement and evaluation, for products and services used...  ..., the mission of Data Science and Insights team is...  ...You will partner with various engineering teams to improve search quality...  ...product engineering teams on core machine learning algorithms and user... 
    Senior
    Work experience placement

    Apple

    Seattle, WA
    2 days ago
  • $64k - $99k

     ...Office Hours or the support work. Design and implement scalable data architectures, ensuring high availability, reliability, and performance...  ...commitment for all systems and software plans, as well as evaluate and guide the selection of technologies required to complete... 
    Senior
    Work at office

    Tata Consultancy Services

    Bellevue, WA
    2 days ago
  • $220k - $280k

     ...Senior Staff Machine Learning Engineer Seattle / San Francisco Scowtt is an early-stage startup transforming the way businesses convert leads into...  ...integrating CRM, web signals, and product interaction data into a real-time conversion model, we help businesses turn... 
    Senior
    Immediate start

    Scowtt

    Seattle, WA
    5 days ago
  • $182k - $242k

     ...: CRWV) in March 2025. Learn more at Our Team...  ...full control over their data, environment, and reward...  ...'re looking for strong engineers with great taste. The most...  ...in Computer Science, Machine Learning, Robotics, or...  ...Experience developing, evaluating, and deploying machine... 
    Senior
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    Bellevue, WA
    2 days ago
  • $118.3k - $188.6k

     ...Senior Information Security EngineerThe Senior Information Security Engineer will be responsible for the ongoing development...  ...intelligence and machine learning security best...  ...help protect systems, data, and workflows from...  ...complex security events.Evaluate security... 
    Senior

    Physicians Insurance A Mutual Company

    Seattle, WA
    4 days ago
  • Job Title: Senior Machine Learning Engineer100% Remote Full Time As a Senior Member of...  ...AI product.Own work end-to-end: data preparation, training, evaluation, inference, and iteration.Turn research...  ...with research, product, and engineering to deliver real user impact.... 
    Senior
    Full time
    Remote work

    Spectraforce Technologies

    Seattle, WA
    6 days ago
  •  ...Role: Senior Data Engineer Identity Resolution & Large-Scale Data Engineering Location: Bellevue, WA Overview: We are seeking...  ...to identify technical and scalability constraints early, evaluate alternatives quickly, and deliver tested, scalable, and production... 
    Senior

    Sumeru Solutions

    Bellevue, WA
    3 days ago
  •  ...and commodity recommendation. We are a group of applied machine learning engineers and data scientists that focus on E-commerce recommendations. We...  ...explore users' latent interests efficiently• Design, develop, evaluate and iterate on predictive models for candidate... 
    Senior
    Temporary work
    Work experience placement

    TikTok

    Seattle, WA
    7 days ago
  •  ...large-scale video and multimodal data ingest, storage, processing,...  ...Build scalable training, evaluation, inference, and serving infrastructure...  ...technical direction, raise engineering standards, and mentor other...  ...mental wellness support. Learning and development programs and... 
    Senior
    Full time
    Work at office
    Remote work

    Axon

    Seattle, WA
    17 days ago
  •  ...into large-scale AI platforms. Create evaluation methods, reward functions, evaluators,...  ...platform, application, and product engineers to deliver reliable AI solutions. Set...  ...deploying scalable, production-ready machine learning systems. Proficiency with LangGraph... 
    Senior
    Full time
    Flexible hours

    Truveta

    Seattle, WA
    11 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Machine Learning / Data Engineer - Evaluation. Be the first to apply!