Machine Learning Engineer, Assessments
Speak
About us
Our mission is to reinvent the way people learn, starting with language.
Learning a language can change a life by opening doors to new cultures, careers, and communities. Two billion people around the world are actively trying to learn a language, but the best way to learn (one-on-one tutoring) is hard to access at scale and hasn’t been meaningfully improved in decades. Speak is building a human-level, AI-powered tutor in your pocket: a conversation-first experience that lets learners actually speak, get instant feedback, and progress through carefully designed lessons. The result is a complete path from beginner to confident speaker across multiple languages.
Speak first launched in South Korea in 2019, where Speak has now become the number one language learning app, and we now serve learners across many markets and 15+ languages. Speak is one of the world’s leading AI companies, with over $150m raised in venture investment from OpenAI, Accel, Founders Fund, Khosla Ventures, and more, with a distributed team across San Francisco, Seoul, Tokyo, Taipei, and Ljubljana.
About this role
We’re hiring an ML Engineer, Assessments to help build best-in-class assessment systems across multiple products (Speak for Business, B2C, and new surfaces). You will work in a tight loop with our Assessment Design Lead (Content/Learning Design) , Machine Learning, Product, and Engineering to turn assessment constructs and rubrics into reliable, scalable scoring + feedback systems.
This role owns the implementation, deployment, and ongoing quality of our assessment algorithms and ML systems. While there is immediate need to improve and expand production assessments, this work is also building a platform capability that can be reused across the app.
What you’ll be doing
Ship and own assessment ML systems end-to-end
Build, deploy, and maintain scoring models/pipelines (feature extraction → model training → inference → feedback generation)
Own monitoring, regression tests, and ongoing iteration to maintain accuracy targets
Define and operationalize evaluation
Implement validation/evaluation frameworks for assessments, including metrics, test sets, and offline/online analysis
Translate assessment requirements into measurable acceptance criteria and guardrails
Partner deeply with the Assessment Design Lead
Co-develop the strategy, together with the Content team, to grow assessments into a core platform at Speak
Work in a tight weekly loop to deliver incremental improvement
Drive near-term delivery across products
Stand up or improve summative assessments (spoken language ability) and bring them reliably to production
Prototype and validate formative assessment approaches to measure improvement over weeks/months
Support data and labeling strategy
Help define data needs for training/evaluation (including psychometric measurement needs)
Build or improve pipelines that support label collection and analysis (especially for efficacy studies)
What we’re looking for
Domain expertise in spoken language proficiency assessment (linguistics, applied linguistics, pedagogy, or equivalent experience)
Strong experience designing and running evaluation + validation for assessment/scoring systems, and tailoring approaches to a specific product use case
4+ years building automatic proficiency assessment systems (or equivalent depth in closely related scoring/evaluation domains)
PhD is helpful but not required
Proven ability to ship ML models to production (not only research), including reliability, monitoring, and iteration
Strong generalist ML/analysis skills (statistics, Python, PyTorch/model training)
Ability to operate cross-functionally and communicate clearly with non-technical partners (Content/LD, PM, leadership)
Nice to have
Experience with speech/audio ML
Experience with psychometrics concepts (reliability/validity, calibration)
How we work (collaboration expectations)
This role is designed to be highly collaborative with the Assessment Design Lead. Success depends on a tight loop where constructs/rubrics and model outputs co-evolve — not a sequential handoff.
Why work at Speak
Join a fantastic, tight-knit team at the right time: we're growing very quickly, we've most recently raised our Series C from some of the top investors in the valley, and we've achieved product-market fit in our initial markets. You'd join at a magical time when a single person could significantly change the course of the company.
Do your life's work with people you’ll love working with: we care strongly about our craft and want every person at Speak to feel like they're growing every day. We believe in the idea that working with people you both enjoy and have respect for makes everything better. We hire thoughtfully and only work with people we admire deeply.
Global in nature: We're live in over 40 countries and launching in a number of new markets soon. We have dedicated offices in San Francisco, Ljubljana, Seoul, and Tokyo, and you’ll have the opportunity to talk to users in each of these regions on a regular basis as well as travel.
Impact people's lives in a major way: Learning a language is one of the single most life-changing skills one can learn, and right now 99% of people never achieve their goal because the process is broken. We’re helping millions of people achieve their goals and improve their lives.
Speak does not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.
$150k - $190k
...-driven simulation software stack for engineering and manufacturing across advanced industries... .... Who We're Looking For As a Machine Learning Engineer in Delivery, you are a... ...depending on experience Seniority will be assessed throughout our interview process...SuggestedRemote jobFull timeFlexible hours$213k - $263k
...that a virtual world is "real"? We are seeking visionary machine learning engineers and researchers to architect the scalable deep learning systems... ...and deployment of cutting-edge evaluation approaches to assess realism of state-of-the-art multimodel world models and...SuggestedFull timeRemote work$140k - $200k
...About the RoleWe’re looking for an ML Engineer to build the production systems that train... ..., monitor, retrain, and serve our machine-learning models reliably. You sit between software... ...(30 minutes)Hands-on-Keys Technical Assessment (1 hour)Onsite Interview: Systems...SuggestedTemporary workWork at officeMonday to FridayMonday to Thursday- ...coherent system. We are building Learning Intelligence Technology (LIT)... ...to plan, teach, assess, and respond to student needs... ...looking for a seasoned Senior ML Engineer with a track record building... ...involves Architect and scale machine learning systems for search,...SuggestedPermanent employmentFull timeLocal areaFlexible hours
$200k - $375k
...About the job Founding Machine Learning Engineer Founding Machine Learning Engineer Posted by Transparent Search Group on behalf... ...the core of our recommendation engine Designing evals to assess recommendation ability and RL systems to learn from results...SuggestedFull time$170k - $230k
...Machine Learning Engineer Help us solve fraud asap with Casap — where we're building the world's first AI-native disputes automation and fraud... ..., including relevant knowledge, skills, and experience assessed during the interview process. Start-up equity Full...Full timeWork at officeImmediate startHome officeMonday to FridayFlexible hours$138k - $208k
...experience for job seekers. Day to Day As a Senior Machine Learning Engineer, you will work on developing, deploying predictive models... ...artificial intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by...Work experience placementLocal area$118k - $176k
...(*Comscore, Total Visits, March 2025) Day to Day The Machine Learning Engineer I role partners closely with business partners across various... ...intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by analyzing...Work experience placementLocal area$112k - $269k
SummaryYelp engineering culture is driven by our values: we’re a cooperative team that values... ...requires the use of cutting-edge Machine Learning (ML) and Artificial Intelligence (AI)... ...will be subject to an individualized assessment that takes into account the candidate'...Work experience placementLocal areaRemote work$290k - $359.6k
...design, develop and test robotic and machine learning applications for an AV company. Duties... ...Master's degree in Data Science, Aerospace Engineering, Automotive Engineering, Computer... ...successfully complete a role-related assessment(s) and/or a pre-employment screening prior...Full timeTemporary workLocal areaRemote workWork from home$242.8k - $357k
...experiences. As a Senior Staff Engineer, you will lead the technical... ...as sequence modeling, deep learning, and large language models (LLMs... ...Apply state-of-the-art machine learning and LLM techniques to... ...forefront of ML and AI innovation by assessing emerging research and...Hourly payWork at officeLocal areaRemote workFlexible hours- ...and accelerated entirely within that runtime. As a Senior Machine Learning Engineer for On-Device & Mobile AI, you will take state-of-the-art multi... ...attention, distillation, reduced-step diffusion) and assess them pragmatically: what actually moves latency/memory/power...Full timeWork at officeRemote workWorldwide
$185.1k - $335.3k
...and autonomy at scale.The RoleWe are looking for a Staff Machine Learning Engineer to serve as a technical leader for automated map reconstruction... ...where applicable, to successfully complete a role-related assessment(s) and/or a pre-employment screening prior to beginning...Full timeLocal areaRemote workWork from homeRelocation packageFlexible hours$155k - $180k
...half the Fortune 100, use Roboflow’s machine learning open source and hosted tools. That includes... ...on all roles (not only product and engineering), so Roboflow employs developers... ...Introduction Phase: ~[15m] Technical Assessment Team Interview Phase: ~ Live coding...Full timeSecond jobRemote workWork from homeRelocation packageFlexible hoursNight shift$138k - $208k
...Comscore, Total Visits, March 2025) Day to Day As a Senior Machine Learning Engineer on our Sourcing team, you will work on developing and... ...intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by analyzing...Work experience placementLocal area$200k - $280k
...Founding Senior Machine Learning Engineer Join to apply for the Founding Senior Machine Learning Engineer role at Retell AI. Base pay range $... ...not experience or past salaries. INTERVIEW PROCESS Online Assessment (30 min): Coding questions on practical problem‑solving (7...H1bWork at office$163k - $245k
...Comscore, Total Visits, March 2025) Day to Day As a Staff Machine Learning Engineer, you will be a team lead on the Employer Agents Team. Your... ...intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by analyzing...Work experience placementLocal area- ...to build the foundation of how code gets written at scale, this is the place for you. The Role We’re looking for a Machine Learning Engineer who loves getting close to the metal. This is a hands-on engineering role focused on making models faster, more efficient...Full timeWork at office
- ...published research papers and won research competitions at top conferences like ICLR, ICML, AAAI. Role Description As a Machine Learning Engineer at Advex, you will play a pivotal role in shaping the company's technical direction. As an MLE you are well versed with...Full time
$225k - $300k
...Machine Learning Engineer About Latent Health Healthcare today is only truly personalized for two groups: those with wealth and access, and those with physicians in their immediate family. For everyone else, care is fragmented and impersonal. Medical history...Full timeWork at officeImmediate start- ...firm). We've raised over 100M from world class investors like A16z, Benchmark, and First Round Capital, and are hiring a Machine Learning Engineer to help us train and deploy the models critical to the performance of our core product. We would love to meet you if...Full timeWork at officeLocal area
- ...worldwide to eliminate busywork and focus on what matters. Learn more at superhuman.com and about our values here. The Opportunity... ...complex tasks, leveraging Superhuman ubiquitous UI. As a Machine Learning Engineer on this team, you will be at the heart of our company's...Full timeWorldwideHome officeFlexible hours
- ...world’s top educational institutions Work together with engineers, scientists, operators, and more from Palantir, Meta, Scale... ...at the largest scale. The ML Opportunity at Handshake: Machine Learning is at the heart of Handshake's mission to democratize access...Full timeWork at officeRemote workFlexible hours
- ...You’ll collaborate with construction veterans and world-class engineers to solve physical-world problems that simulations can’t... ...alongside a talented team—we'd love to have you join us. Machine Learning Engineer: Perception Analytics We are looking for a perception...Full timeWork at officeFlexible hours
- ...tools consistently fail. We are a small, fast-growing team of engineers in San Francisco powering Fortune 100 enterprises, YC startups,... ...~5 days in-office at our San Francisco office ~ Eager to learn and adapt quickly ~ Prior startup or founding experience is a...Full timeWork at officeVisa sponsorshipRelocation package
$55 per hour
...industry. Listen to the Visa episode of the Acquired podcast to learn how credit card networks did this for retail trade. We think... ...— you'll be setting the direction for how we do modeling, data engineering, and production ML at Wholesail for years to come. The Role...Full timeTemporary workWork at officeLocal area$137.1k - $201.6k
...We leverage artificial intelligence and advanced ML, deep learning techniques to power decision-making in real time — from optimizing... ...marketplace. About the Role We’re looking for a Machine Learning Engineer to help design, build, optimize and scale large-scale ML...Hourly payWork at officeLocal areaRemote workFlexible hours$150k - $200k
...uniting high-quality instructional materials with dynamic digital learning. Through unparalleled curriculum management functionality,... ...work closely with other departments, including Product, Engineering, Machine Learning and Analytics, to understand and cater to their data...Permanent employmentFull timeWork at officeLocal areaRemote work- ...have to know a.) precisely where they are, and b.) everything about the fruit they are seeing. We are looking for a Machine Learning Engineer to build creative, practical, and robust solutions to ML/CV software and infrastructure problems, relating to training edge...Full timeWork at officeFlexible hoursWeekend work
$200k - $400k
...a team of the world’s top interpretability researchers and engineers from organizations like OpenAI and DeepMind. We’ve raised $5... ...Clinic, and Rakuten. About the role We’re looking for Machine Learning Engineers to help build our platform for training,...Full time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Machine Learning Engineer, Assessments. Be the first to apply!
- graduate machine learning engineer San Francisco, CA
- senior ml engineer San Francisco, CA
- machine learning engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- machine learning software engineer San Francisco, CA
- ai ml engineer San Francisco, CA
- computer vision machine learning engineer San Francisco, CA
- junior machine learning research engineer San Francisco, CA
- machine learning researcher San Francisco, CA
- machine learning part time San Francisco, CA



