Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Machine Learning Engineer, Assessments

Full-time

Speak

About us

Our mission is to reinvent the way people learn, starting with language.

Learning a language can change a life by opening doors to new cultures, careers, and communities. Two billion people around the world are actively trying to learn a language, but the best way to learn (one-on-one tutoring) is hard to access at scale and hasn’t been meaningfully improved in decades. Speak is building a human-level, AI-powered tutor in your pocket: a conversation-first experience that lets learners actually speak, get instant feedback, and progress through carefully designed lessons. The result is a complete path from beginner to confident speaker across multiple languages.

Speak first launched in South Korea in 2019, where Speak has now become the number one language learning app, and we now serve learners across many markets and 15+ languages. Speak is one of the world’s leading AI companies, with over $150m raised in venture investment from OpenAI, Accel, Founders Fund, Khosla Ventures, and more, with a distributed team across San Francisco, Seoul, Tokyo, Taipei, and Ljubljana.

About this role

We’re hiring an ML Engineer, Assessments to help build best-in-class assessment systems across multiple products (Speak for Business, B2C, and new surfaces). You will work in a tight loop with our Assessment Design Lead (Content/Learning Design) , Machine Learning, Product, and Engineering to turn assessment constructs and rubrics into reliable, scalable scoring + feedback systems.

This role owns the implementation, deployment, and ongoing quality of our assessment algorithms and ML systems. While there is immediate need to improve and expand production assessments, this work is also building a platform capability that can be reused across the app.

What you’ll be doing

  • Ship and own assessment ML systems end-to-end

    • Build, deploy, and maintain scoring models/pipelines (feature extraction → model training → inference → feedback generation)

    • Own monitoring, regression tests, and ongoing iteration to maintain accuracy targets

  • Define and operationalize evaluation

    • Implement validation/evaluation frameworks for assessments, including metrics, test sets, and offline/online analysis

    • Translate assessment requirements into measurable acceptance criteria and guardrails

  • Partner deeply with the Assessment Design Lead

    • Co-develop the strategy, together with the Content team, to grow assessments into a core platform at Speak

    • Work in a tight weekly loop to deliver incremental improvement

  • Drive near-term delivery across products

    • Stand up or improve summative assessments (spoken language ability) and bring them reliably to production

    • Prototype and validate formative assessment approaches to measure improvement over weeks/months

  • Support data and labeling strategy

    • Help define data needs for training/evaluation (including psychometric measurement needs)

    • Build or improve pipelines that support label collection and analysis (especially for efficacy studies)

What we’re looking for

  • Domain expertise in spoken language proficiency assessment (linguistics, applied linguistics, pedagogy, or equivalent experience)

  • Strong experience designing and running evaluation + validation for assessment/scoring systems, and tailoring approaches to a specific product use case

  • 4+ years building automatic proficiency assessment systems (or equivalent depth in closely related scoring/evaluation domains)

    • PhD is helpful but not required

  • Proven ability to ship ML models to production (not only research), including reliability, monitoring, and iteration

  • Strong generalist ML/analysis skills (statistics, Python, PyTorch/model training)

  • Ability to operate cross-functionally and communicate clearly with non-technical partners (Content/LD, PM, leadership)

Nice to have

  • Experience with speech/audio ML

  • Experience with psychometrics concepts (reliability/validity, calibration)

How we work (collaboration expectations)

This role is designed to be highly collaborative with the Assessment Design Lead. Success depends on a tight loop where constructs/rubrics and model outputs co-evolve — not a sequential handoff.

Why work at Speak

  1. Join a fantastic, tight-knit team at the right time: we're growing very quickly, we've most recently raised our Series C from some of the top investors in the valley, and we've achieved product-market fit in our initial markets. You'd join at a magical time when a single person could significantly change the course of the company.

  2. Do your life's work with people you’ll love working with: we care strongly about our craft and want every person at Speak to feel like they're growing every day. We believe in the idea that working with people you both enjoy and have respect for makes everything better. We hire thoughtfully and only work with people we admire deeply.

  3. Global in nature: We're live in over 40 countries and launching in a number of new markets soon. We have dedicated offices in San Francisco, Ljubljana, Seoul, and Tokyo, and you’ll have the opportunity to talk to users in each of these regions on a regular basis as well as travel.

  4. Impact people's lives in a major way: Learning a language is one of the single most life-changing skills one can learn, and right now 99% of people never achieve their goal because the process is broken. We’re helping millions of people achieve their goals and improve their lives.

Speak does not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Machine Learning Engineer, Assessments in San Francisco, CA vacancy
  •  ...mission is to reinvent the way people learn, starting with language. Learning a...  ...About this role We’re hiring an ML Engineer, Assessments to help build best-in-class assessment...  ...Design Lead (Content/Learning Design) , Machine Learning, Product, and Engineering to turn... 
    Suggested
    Full time
    Live in
    Immediate start

    Speak

    San Francisco, CA
    1 day ago
  • $213k - $263k

     ...that a virtual world is "real"? We are seeking visionary machine learning engineers and researchers to architect the scalable deep learning systems...  ...and deployment of cutting-edge evaluation approaches to assess realism of state-of-the-art multimodel world models and... 
    Suggested
    Full time
    Remote work

    Waymo

    San Francisco, CA
    1 day ago
  • $150k - $190k

     ...-driven simulation software stack for engineering and manufacturing across advanced industries...  .... Who We're Looking For As a Machine Learning Engineer in Delivery, you are a...  ...depending on experience  Seniority will be assessed throughout our interview process... 
    Suggested
    Remote job
    Full time
    Flexible hours

    Physicsx

    San Francisco, CA
    1 day ago
  • $140k - $200k

     ...About the RoleWe’re looking for an ML Engineer to build the production systems that train...  ..., monitor, retrain, and serve our machine-learning models reliably. You sit between software...  ...(30 minutes)Hands-on-Keys Technical Assessment (1 hour)Onsite Interview: Systems... 
    Suggested
    Temporary work
    Work at office
    Monday to Friday
    Monday to Thursday

    Sprinter Health

    San Francisco, CA
    1 day ago
  •  ...coherent system. We are building Learning Intelligence Technology (LIT)...  ...to plan, teach, assess, and respond to student needs...  ...looking for a seasoned Senior ML Engineer with a track record building...  ...involves Architect and scale machine learning systems for search,... 
    Suggested
    Permanent employment
    Full time
    Local area
    Flexible hours

    Kiddom

    San Francisco, CA
    1 day ago
  • $138k - $208k

     ...experience for job seekers. Day to Day As a Senior Machine Learning Engineer, you will work on developing, deploying predictive models...  ...artificial intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by... 
    Work experience placement
    Local area

    Indeed Inc.

    San Francisco, CA
    3 days ago
  • $290k - $359.6k

     ...design, develop and test robotic and machine learning applications for an AV company. Duties...  ...Master's degree in Data Science, Aerospace Engineering, Automotive Engineering, Computer...  ...successfully complete a role-related assessment(s) and/or a pre-employment screening prior... 
    Full time
    Temporary work
    Local area
    Remote work
    Work from home

    General Motors

    San Francisco, CA
    5 days ago
  • $112k - $269k

    SummaryYelp engineering culture is driven by our values: we’re a cooperative team that values...  ...requires the use of cutting-edge Machine Learning (ML) and Artificial Intelligence (AI)...  ...will be subject to an individualized assessment that takes into account the candidate'... 
    Work experience placement
    Local area
    Remote work

    Yelp

    San Francisco, CA
    3 days ago
  • $242.8k - $357k

     ...experiences. As a Senior Staff Engineer, you will lead the technical...  ...as sequence modeling, deep learning, and large language models (LLMs...  ...Apply state-of-the-art machine learning and LLM techniques to...  ...forefront of ML and AI innovation by assessing emerging research and... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash Usa

    San Francisco, CA
    1 day ago
  •  ...and accelerated entirely within that runtime. As a Senior Machine Learning Engineer for On-Device & Mobile AI, you will take state-of-the-art multi...  ...attention, distillation, reduced-step diffusion) and assess them pragmatically: what actually moves latency/memory/power... 
    Full time
    Work at office
    Remote work
    Worldwide

    Unity Technologies

    San Francisco, CA
    1 day ago
  • $185.1k - $335.3k

     ...and autonomy at scale.The RoleWe are looking for a Staff Machine Learning Engineer to serve as a technical leader for automated map reconstruction...  ...where applicable, to successfully complete a role-related assessment(s) and/or a pre-employment screening prior to beginning... 
    Full time
    Local area
    Remote work
    Work from home
    Relocation package
    Flexible hours

    General Motors

    San Francisco, CA
    2 days ago
  • $155k - $180k

     ...half the Fortune 100, use Roboflow’s machine learning open source and hosted tools. That includes...  ...on all roles (not only product and engineering), so Roboflow employs developers...  ...Introduction Phase: ~[15m] Technical Assessment Team Interview Phase: ~ Live coding... 
    Full time
    Second job
    Remote work
    Work from home
    Relocation package
    Flexible hours
    Night shift

    Roboflow

    San Francisco, CA
    1 day ago
  • $200k - $280k

     ...Founding Senior Machine Learning Engineer Join to apply for the Founding Senior Machine Learning Engineer role at Retell AI. Base pay range $...  ...not experience or past salaries. INTERVIEW PROCESS Online Assessment (30 min): Coding questions on practical problem‑solving (7... 
    H1b
    Work at office

    Retell AI

    San Francisco, CA
    2 days ago
  • $163k - $245k

     ...Comscore, Total Visits, March 2025) Day to Day As a Staff Machine Learning Engineer, you will be a team lead on the Employer Agents Team. Your...  ...intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by analyzing... 
    Work experience placement
    Local area

    Indeed

    San Francisco, CA
    5 days ago
  • $225k - $300k

     ...Machine Learning Engineer About Latent Health Healthcare today is only truly personalized for two groups: those with wealth and access, and those with physicians in their immediate family. For everyone else, care is fragmented and impersonal. Medical history... 
    Full time
    Work at office
    Immediate start

    Latent

    San Francisco, CA
    1 day ago
  •  ...published research papers and won research competitions at top conferences like ICLR, ICML, AAAI. Role Description As a Machine Learning Engineer at Advex, you will play a pivotal role in shaping the company's technical direction. As an MLE you are well versed with... 
    Full time

    Openreq

    San Francisco, CA
    1 day ago
  • $250k - $295k

     ...current delivery mechanism. The real product is a scalable risk engine, our Stand World Model . We stay when traditional insurers...  ...operate with far less friction. The Opportunity: As a Machine Learning Engineer on the Applied Science team, you will design, train... 
    Full time
    Temporary work
    H1b
    Work at office
    Remote work
    Visa sponsorship
    Work visa
    Flexible hours

    Stand Insurance

    San Francisco, CA
    1 day ago
  •  ...have to know a.) precisely where they are, and b.) everything about the fruit they are seeing. We are looking for a Machine Learning Engineer to build creative, practical, and robust solutions to ML/CV software and infrastructure problems, relating to training edge... 
    Full time
    Work at office
    Flexible hours
    Weekend work

    Orchard Robotics

    San Francisco, CA
    1 day ago
  • $150k - $200k

     ...practice—real customer use-cases with clear success criteria. Required Qualifications ● PhD or MS degree in Computer Science, Machine Learning, Robotics, or equivalent technical discipline ● Deep expertise in machine learning fundamentals, reinforcement learning, and... 
    Full time

    Deft Ai, Inc.

    San Francisco, CA
    1 day ago
  • $200k - $400k

     ...a team of the world’s top interpretability researchers and engineers from organizations like OpenAI and DeepMind. We’ve raised $5...  ...Clinic, and Rakuten. About the role We’re looking for Machine Learning Engineers to help build our platform for training,... 
    Full time

    Goodfire

    San Francisco, CA
    1 day ago
  •  ...About the company We’re a team of engineers, neuroscientists, and designers solving the most difficult...  ...of autonomy and demand intense, fast-paced learning. You will: Critically evaluate and implement the best machine learning approaches for our unique design... 
    Full time

    Orbit Neuro Co.

    San Francisco, CA
    1 day ago
  •  ...About the Role We're looking for founding Machine Learning Engineers (MLEs) to own and improve our core action models end-to-end - the intelligence that powers Composite's proactive automation platform. You'll work at the intersection of LLM inference, browser understanding... 
    Full time
    Sleeping nights

    Composite

    San Francisco, CA
    1 day ago
  •  ...compounds with every signal. About the Role At Poesis, machine learning and artificial intelligence open the door to improved alpha...  ...management. We're looking for an exceptional Machine Learning Engineer to help build the systems that make this possible. In this... 
    Full time
    Work at office
    Visa sponsorship
    Work visa
    Relocation package
    3 days per week

    Poesis Inc

    San Francisco, CA
    1 day ago
  •  ...bridging the two worlds together. Check out our About page to learn more. The Mission: Fully Autonomous Underwriting We...  ...mile driven. The Role You will be Shepherd’s first Machine Learning Engineer, embedded in the Fully Autonomous Underwriting (FAU) team.... 
    Full time
    Work at office

    Shepherd

    San Francisco, CA
    1 day ago
  • $110k - $170k

     ...Databricks, GM, and Character, our mission is to revolutionize how engineering decisions are made, turning complexity into clarity for the...  ...Engineer at Apiphany, you’ll develop and implement advanced machine learning models to tackle some of the hardest problems in the... 
    Full time
    Work at office
    Visa sponsorship
    Flexible hours

    Apiphany

    San Francisco, CA
    1 day ago
  • $120k - $180k

     ...Please reach out if you are interested in joining the future of AI! Machine Learning Role In order to execute our vision, we need to grow our team of best-in-class machine learning engineers. We are looking for developers who are excited about staying at the... 
    Full time

    Hive

    San Francisco, CA
    1 day ago
  •  ...Our mission is to recover human embodied intelligence as a learned model. To achieve this, we build custom hardware products,...  ...civilizational scale, join us. The Opportunity As a Machine Learning Engineer, you’ll work on multimodal perception, VLA training, robotics... 
    Full time
    Shift work

    Human Archive

    San Francisco, CA
    1 day ago
  •  ...future of healthcare, we’d love to meet you. Apply now to join our growing team. About the Role We're looking for a Machine Learning Engineer to design, build, and deploy production-grade ML systems that power the next generation of Plenful's AI platform. You'll own... 
    Full time
    Work at office
    Remote work
    Flexible hours
    2 days per week

    Plenful

    San Francisco, CA
    1 day ago
  •  ...global presence across the United States, EMEA, and APAC. Role Overview We are seeking a motivated and detail-oriented Machine Learning Engineer to join our team. As an ML Engineer, you will be involved in designing and implementing machine learning models and data... 
    Full time

    Rzr Global Inc.

    San Francisco, CA
    1 day ago
  • $175k - $300k

     ...meaningful piece of the company. High talent density: Be a part of a great founding team in SF. Shivaal Roy (CTO) was a founding engineer at Glean ($100M+ ARR today) and managed the Assistant and Search teams, where he solved similar problems. Danny Goldman (CEO)... 
    Full time

    Trove

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Machine Learning Engineer, Assessments. Be the first to apply!