Applied Research — Evaluations & Data
Jobleads-US
Building the autonomous company
Human Intuition is building the autonomous company. Businesses run on accumulated judgment: how to interpret a situation, choose an action, and learn from its consequences. Much of that knowledge lives in people, even when the decisions they make leave traces in software.
We are working to make that judgment learnable. A business has defined systems, tools, permissions, histories, and objectives. Those boundaries create an opportunity to build agents that learn from how work is done, act within clear constraints, and improve through feedback. Our ambition is to turn the knowledge inside institutions into software that compounds.
The role
Build the evidence that tells us whether an agent is ready to do useful work. You will turn business workflows, decision histories, and operational outcomes into datasets and evaluations that distinguish a plausible response from a correct action. Your work will define what progress means for the autonomous company.
What you’ll do
Work with domain experts to identify the decisions, constraints, and outcomes that matter in a workflow.
Design tasks, evaluation environments, rubrics, and verifiers that test complete workflows, including ambiguous inputs and failure recovery.
Build data pipelines that preserve the relationship between context, decisions, actions, and outcomes, with clear provenance and appropriate access controls.
Create evaluation splits and review processes that limit leakage, expose distribution shifts, and catch regressions.
Study agent traces to identify recurring failures and turn the findings into better training data, tools, and reward signals.
Build repeatable evaluation reports that let researchers and engineers compare quality, reliability, latency, and cost.
What you’ll bring
Strong Python and practical experience building data or machine learning systems.
Evidence of designing useful evaluations, experiments, or datasets for models or complex software.
Sound statistical judgment and the ability to explain what a metric establishes and what it misses.
Care with data quality, reproducibility, and the handling of business information.
The ability to learn an unfamiliar domain and communicate clearly with both technical and operational colleagues.
Useful experience
Agent evaluation, reward modeling, human feedback systems, synthetic data, process mining, or research involving sequential decisions. Published work and open-source contributions are welcome; concrete examples of careful measurement matter just as much.
What success looks like
The team can tell whether a change improves real task outcomes, reproduce the result, and trace failures back to actionable causes.
Apply
Show us something you have built, investigated, or improved, and tell us what you learned. That might be a product, a research result, an open-source contribution, or work you can describe without disclosing confidential information. We welcome strong evidence of ability from a range of backgrounds.
#J-18808-Ljbffr Jobleads-US$183.8k - $248.7k
...Sr. Applied Scientist, Agent Evaluation and Experimentation, DTx Science Job ID: 10572608 | Amazon.com Services... ...roadmap, promote a culture of data-driven decision-making, and deliver... ...Master's degree and 6+ years of applied research experience ~ Experience...DataWorldwideFlexible hours- ...workflows, decision histories, and outcomes into datasets and evaluations that distinguish plausible actions from correct ones. You will design tasks, evaluation environments, and verifiers, build data pipelines with provenance, and create repeatable reports to compare...Data
- ...spirited. The Role Mistral AI is seeking Applied Scientists and Research Engineers to drive innovative research and... ...wanting to talk. Generate and curate data for pre-training and post-training, working on evaluations and making sure the model's performance beats...DataRelocation package
$276k - $345k
...treasure trove of observability data. We are transforming... ...canvas.We're hiring a Staff Applied Scientist to define and guarantee... ...of open questions. How do you evaluate an agent end-to-end when the... ...product-driven environment, from research through production....DataWork at office$183.8k - $248.7k
We are seeking a Sr. Applied Scientist to develop and optimize Visual... ..., hardware, and real-world data, while prototyping, testing,... ...Build and maintain calibration, evaluation, and benchmarking... ...Lead technical projects from research prototyping through production...DataFlexible hours- The Applied Research Scientist will drive rigorous research to deepen our understanding of complex... ...expertise to uncover emerging risks, evaluate safety interventions, and translate... ...The researcher will partner closely with Data Science, Policy, Product, Operations, and...Data
$183.8k - $248.7k
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus... ..., or Master's degree and 6+ years of applied research experience- 3+ years of industry or academic research...DataFlexible hours- ...The Research & Intelligence team builds the machine learning and semantic... ...and causal modeling, data valuation systems, and retrieval... .... We are seeking a Senior Applied Scientist to lead and expand... ...rigorous standards for model evaluation, monitoring, and lifecycle management...DataWork at officeLocal areaRemote work
$172.4k - $223.4k
...re looking for an experienced Applied Scientist with exceptional... ...and innovative capabilities to research, design, and create elegant machine... ...build ML models and perform data analysis to deliver scalable... ...of millions of customers, evaluate the impact of your...DataFlexible hours$165.6k - $207k
...leading AI labs to provide high quality data and accelerate progress in GenAI research. We are looking for Research... ...(SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod... ...both text and multimodal modalities.Apply post-training expertise (SFT, RLHF...DataFull time$167.1k - $226.1k
...bring problems that will give Applied Scientists like you endless opportunities to see your research have a positive and immediate... ...have access to virtually endless data and computational resources,... ...multimodal), agentic AI, agentic evaluations, MTRL, ML gyms.Basic...DataImmediate startFlexible hours$216k - $270k
Scale Labs, Research Scientist — Frontier Risk EvaluationsAs the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding... ...to our mission, we encourage you to apply, even if your experience doesn’t perfectly align...DataFull time$192.2k - $260k
...process massive volumes of advertiser data every single day. You'll work with us... ...ensure our solutions are constantly evaluating themselves and improving.The Ads Real... ...Service team is seeking an exceptional Applied Scientist to research and develop novel approaches for...DataLocal areaImmediate startFlexible hoursNight shiftDay shift$172.4k - $223.4k
...team is looking for a driven Applied Scientist to help safeguard AWS... ...models (LLMs) and synthetic data generation, to enhance fraud... ...feature engineering through evaluation, productionalization, and deployment... ...Computer Science, Operations Research or in another highly...DataWorldwideFlexible hoursShift work$192.2k - $260k
...job responsibilitiesAs a Senior Applied Scientist on this team, you will:... ...Design and conduct A/B experiments to evaluate proposed solutions based on in-depth data analyses. * Identify... ...s degree and 6+ years of applied research experience- Experience programming...DataLocal areaFlexible hours$172.4k - $223.4k
We are seeking an Applied Scientist to contribute to research and development of novel security validation and monitoring... ..., including adversarial robustness evaluation, safety guardrail bypass detection,... ...appropriate technologies (e.g., data protection, private inference,...DataFlexible hours$183.8k - $248.7k
We are seeking a Senior Applied Scientist to lead research and development of novel security validation and... ...tool-use authorization boundaries, and data exfiltration detection.3. AI-Enabled... ..., including adversarial robustness evaluation, safety guardrail bypass detection,...DataTemporary workFlexible hours$192.2k - $260k
...motivated, passionate and resourceful Applied Scientist to bring diverse... ...machine learning practitioner and a research leader. You will play a key role... ...and online (A/B) experiments to evaluate proposed solutions based on in-depth data analyses; Effectively communicate...DataLocal areaWorldwideFlexible hours$306.3k - $349.5k
Staff Applied Researcher At Capital One, we are creating trustworthy and reliable AI systems, changing... ...with a cross-functional team of data scientists, software engineers, machine... ...development, from design through training, evaluation, validation, and implementation....DataFull timePart timeLocal areaFlexible hours$172.4k - $223.4k
...are looking for a passionate Applied Scientist with technical expertise... ...with real-time operational data, and thrive in an environment... ..., deployment, and serving.* Research and implement new machine... ...orchestration logic, and write new evaluation cases. You will design agent...DataFlexible hours$70k - $130k
Research --> Basic Science/TechnicalNew York • ID: 1164908... ...for Surgical and Transplant Applied Research (CSTAR). The Research... ...teams to develop, implement, and evaluate research studies aligned with... ...), utilizing a variety of data sources, including:...DataFull timeTraineeship$198.2k - $368k
...Labs is seeking an experienced applied science leader with a strong... ...customers through applied research and innovation, working closely... ...to end-to-end platforms for data management, filing, audit, and... ...usage data, quality metrics, evaluations, and customer feedback.Develop...DataFull timeLocal areaFlexible hours$198.2k - $368k
Job Description: Director of Applied Research, Information RetrievalThomson Reuters Labs About... ...TBs of legal, regulatory, news, and tax data, we are building the next generation of... ...and contextual retrieval techniquesBuild evaluation frameworks and benchmarks for search...DataFull timeLocal areaFlexible hours$295k
...customers. Cohere is a team of researchers, engineers, designers, and... ...Seoul. Join us!Role Overview:Evaluation is critical to making progress... ...LLM judges; refining LLM-based data synthesis pipelines; and... ...experience, we still encourage you to apply. We strive to create an...DataFull timeWork at officeLocal areaRemote workHome office$172.4k - $223.4k
...the Measurement, Ad Tech, and Data Science (MADS) team of Amazon... ..., and Computer Vision.As an Applied Scientist on the team, you will... ...problems to propose clear evaluation frameworks and success criteriaWork... ...scientists across Applied, Research, Data Science and Economist...DataFlexible hours$218.7k - $249.6k
Applied Researcher 4 (AI Foundations, LLM Customization and Finetuning) At Capital One, we are creating... ...with a cross-functional team of data scientists, software engineers, machine... ...development, from design through training, evaluation, validation, and implementation....DataFull timePart timeLocal areaFlexible hours$146.8k - $272.6k
...excited about working at the forefront of applied research in an industry setting? TR Labs is the... ...technical judgment and evaluation expertiseExcellent communication skills... ...insights to empower professionals with the data, intelligence, and solutions needed to...DataFull timeWork at officeLocal areaFlexible hours- ...platform, we are building a Data + AI team to unlock the next... ...Role We're looking for an Applied AI Scientist who will design... .... You'll bridge research and production, working on the... ...Validate Models: Build and evaluate experimental AI models using...DataTemporary workRemote workWork from homeWorldwideHome officeFlexible hours
$135.66k
...and access information. Our Applied Science team is at the forefront... ...yourself in groundbreaking research, exploring novel machine learning... ...repositories of unstructured data, fueling the next generation... ..., and experimentally evaluate new recommendation and search...DataFull timeInternshipWorldwideRelocation$197.3k - $313.7k
...experience, please consider applying for a maximum of 3 roles within... ...directly enable world-class research and products used by millions... ...services, like APIs, UIs, agentic evaluators, and more.Roll out scalable... ...how we use your personal data and your rights, including with...DataFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Applied Research — Evaluations & Data. Be the first to apply!




