Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Applied Research — Evaluations & Data

Jobleads-US

Building the autonomous company

Human Intuition is building the autonomous company. Businesses run on accumulated judgment: how to interpret a situation, choose an action, and learn from its consequences. Much of that knowledge lives in people, even when the decisions they make leave traces in software.

We are working to make that judgment learnable. A business has defined systems, tools, permissions, histories, and objectives. Those boundaries create an opportunity to build agents that learn from how work is done, act within clear constraints, and improve through feedback. Our ambition is to turn the knowledge inside institutions into software that compounds.

The role

Build the evidence that tells us whether an agent is ready to do useful work. You will turn business workflows, decision histories, and operational outcomes into datasets and evaluations that distinguish a plausible response from a correct action. Your work will define what progress means for the autonomous company.

What you’ll do

  • Work with domain experts to identify the decisions, constraints, and outcomes that matter in a workflow.

  • Design tasks, evaluation environments, rubrics, and verifiers that test complete workflows, including ambiguous inputs and failure recovery.

  • Build data pipelines that preserve the relationship between context, decisions, actions, and outcomes, with clear provenance and appropriate access controls.

  • Create evaluation splits and review processes that limit leakage, expose distribution shifts, and catch regressions.

  • Study agent traces to identify recurring failures and turn the findings into better training data, tools, and reward signals.

  • Build repeatable evaluation reports that let researchers and engineers compare quality, reliability, latency, and cost.

What you’ll bring

  • Strong Python and practical experience building data or machine learning systems.

  • Evidence of designing useful evaluations, experiments, or datasets for models or complex software.

  • Sound statistical judgment and the ability to explain what a metric establishes and what it misses.

  • Care with data quality, reproducibility, and the handling of business information.

  • The ability to learn an unfamiliar domain and communicate clearly with both technical and operational colleagues.

Useful experience

Agent evaluation, reward modeling, human feedback systems, synthetic data, process mining, or research involving sequential decisions. Published work and open-source contributions are welcome; concrete examples of careful measurement matter just as much.

What success looks like

The team can tell whether a change improves real task outcomes, reproduce the result, and trace failures back to actionable causes.

Apply

Show us something you have built, investigated, or improved, and tell us what you learned. That might be a product, a research result, an open-source contribution, or work you can describe without disclosing confidential information. We welcome strong evidence of ability from a range of backgrounds.

#J-18808-Ljbffr Jobleads-US
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Applied Research — Evaluations & Data in New York, NY vacancy
  • $183.8k - $248.7k

     ...Sr. Applied Scientist, Agent Evaluation and Experimentation, DTx Science Job ID: 10572608 | Amazon.com Services...  ...roadmap, promote a culture of data-driven decision-making, and deliver...  ...Master's degree and 6+ years of applied research experience ~ Experience... 
    Data
    Worldwide
    Flexible hours

    Jobleads-US

    New York, NY
    10 hours ago
  •  ...workflows, decision histories, and outcomes into datasets and evaluations that distinguish plausible actions from correct ones. You will design tasks, evaluation environments, and verifiers, build data pipelines with provenance, and create repeatable reports to compare... 
    Data

    Jobleads-US

    New York, NY
    3 days ago
  •  ...spirited. The Role Mistral AI is seeking Applied Scientists and Research Engineers to drive innovative research and...  ...wanting to talk. Generate and curate data for pre-training and post-training, working on evaluations and making sure the model's performance beats... 
    Data
    Relocation package

    Mistral AI

    New York, NY
    1 day ago
  • $276k - $345k

     ...treasure trove of observability data. We are transforming...  ...canvas.We're hiring a Staff Applied Scientist to define and guarantee...  ...of open questions. How do you evaluate an agent end-to-end when the...  ...product-driven environment, from research through production.... 
    Data
    Work at office

    Datadog

    New York, NY
    2 days ago
  • $183.8k - $248.7k

    We are seeking a Sr. Applied Scientist to develop and optimize Visual...  ..., hardware, and real-world data, while prototyping, testing,...  ...Build and maintain calibration, evaluation, and benchmarking...  ...Lead technical projects from research prototyping through production... 
    Data
    Flexible hours

    Amazon

    New York, NY
    4 days ago
  • The Applied Research Scientist will drive rigorous research to deepen our understanding of complex...  ...expertise to uncover emerging risks, evaluate safety interventions, and translate...  ...The researcher will partner closely with Data Science, Policy, Product, Operations, and... 
    Data

    TikTok

    New York, NY
    4 days ago
  • $183.8k - $248.7k

    We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus...  ..., or Master's degree and 6+ years of applied research experience- 3+ years of industry or academic research... 
    Data
    Flexible hours

    Amazon

    New York, NY
    4 days ago
  •  ...The Research & Intelligence team builds the machine learning and semantic...  ...and causal modeling, data valuation systems, and retrieval...  .... We are seeking a Senior Applied Scientist to lead and expand...  ...rigorous standards for model evaluation, monitoring, and lifecycle management... 
    Data
    Work at office
    Local area
    Remote work

    Carbon Arc, Inc.

    New York, NY
    3 days ago
  • $172.4k - $223.4k

     ...re looking for an experienced Applied Scientist with exceptional...  ...and innovative capabilities to research, design, and create elegant machine...  ...build ML models and perform data analysis to deliver scalable...  ...of millions of customers, evaluate the impact of your... 
    Data
    Flexible hours

    Amazon

    New York, NY
    1 day ago
  • $165.6k - $207k

     ...leading AI labs to provide high quality data and accelerate progress in GenAI research. We are looking for Research...  ...(SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod...  ...both text and multimodal modalities.Apply post-training expertise (SFT, RLHF... 
    Data
    Full time

    Scale AI

    New York, NY
    3 days ago
  • $167.1k - $226.1k

     ...bring problems that will give Applied Scientists like you endless opportunities to see your research have a positive and immediate...  ...have access to virtually endless data and computational resources,...  ...multimodal), agentic AI, agentic evaluations, MTRL, ML gyms.Basic... 
    Data
    Immediate start
    Flexible hours

    Amazon

    New York, NY
    20 hours ago
  • $216k - $270k

    Scale Labs, Research Scientist — Frontier Risk EvaluationsAs the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding...  ...to our mission, we encourage you to apply, even if your experience doesn’t perfectly align... 
    Data
    Full time

    Scale AI

    New York, NY
    3 days ago
  • $192.2k - $260k

     ...process massive volumes of advertiser data every single day. You'll work with us...  ...ensure our solutions are constantly evaluating themselves and improving.The Ads Real...  ...Service team is seeking an exceptional Applied Scientist to research and develop novel approaches for... 
    Data
    Local area
    Immediate start
    Flexible hours
    Night shift
    Day shift

    Amazon

    New York, NY
    2 days ago
  • $172.4k - $223.4k

     ...team is looking for a driven Applied Scientist to help safeguard AWS...  ...models (LLMs) and synthetic data generation, to enhance fraud...  ...feature engineering through evaluation, productionalization, and deployment...  ...Computer Science, Operations Research or in another highly... 
    Data
    Worldwide
    Flexible hours
    Shift work

    Amazon

    New York, NY
    1 day ago
  • $192.2k - $260k

     ...job responsibilitiesAs a Senior Applied Scientist on this team, you will:...  ...Design and conduct A/B experiments to evaluate proposed solutions based on in-depth data analyses. * Identify...  ...s degree and 6+ years of applied research experience- Experience programming... 
    Data
    Local area
    Flexible hours

    Amazon

    New York, NY
    3 days ago
  • $172.4k - $223.4k

    We are seeking an Applied Scientist to contribute to research and development of novel security validation and monitoring...  ..., including adversarial robustness evaluation, safety guardrail bypass detection,...  ...appropriate technologies (e.g., data protection, private inference,... 
    Data
    Flexible hours

    Amazon

    New York, NY
    1 day ago
  • $183.8k - $248.7k

    We are seeking a Senior Applied Scientist to lead research and development of novel security validation and...  ...tool-use authorization boundaries, and data exfiltration detection.3. AI-Enabled...  ..., including adversarial robustness evaluation, safety guardrail bypass detection,... 
    Data
    Temporary work
    Flexible hours

    Amazon

    New York, NY
    20 hours ago
  • $192.2k - $260k

     ...motivated, passionate and resourceful Applied Scientist to bring diverse...  ...machine learning practitioner and a research leader. You will play a key role...  ...and online (A/B) experiments to evaluate proposed solutions based on in-depth data analyses; Effectively communicate... 
    Data
    Local area
    Worldwide
    Flexible hours

    Amazon

    New York, NY
    3 days ago
  • $306.3k - $349.5k

    Staff Applied Researcher At Capital One, we are creating trustworthy and reliable AI systems, changing...  ...with a cross-functional team of data scientists, software engineers, machine...  ...development, from design through training, evaluation, validation, and implementation.... 
    Data
    Full time
    Part time
    Local area
    Flexible hours

    Capital One

    New York, NY
    4 days ago
  • $172.4k - $223.4k

     ...are looking for a passionate Applied Scientist with technical expertise...  ...with real-time operational data, and thrive in an environment...  ..., deployment, and serving.* Research and implement new machine...  ...orchestration logic, and write new evaluation cases. You will design agent... 
    Data
    Flexible hours

    Amazon

    New York, NY
    2 days ago
  • $70k - $130k

     Research --> Basic Science/TechnicalNew York • ID: 1164908...  ...for Surgical and Transplant Applied Research (CSTAR). The Research...  ...teams to develop, implement, and evaluate research studies aligned with...  ...), utilizing a variety of data sources, including:... 
    Data
    Full time
    Traineeship

    NYU Langone Medical Center

    New York, NY
    1 day ago
  • $198.2k - $368k

     ...Labs is seeking an experienced applied science leader with a strong...  ...customers through applied research and innovation, working closely...  ...to end-to-end platforms for data management, filing, audit, and...  ...usage data, quality metrics, evaluations, and customer feedback.Develop... 
    Data
    Full time
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    4 days ago
  • $198.2k - $368k

    Job Description: Director of Applied Research, Information RetrievalThomson Reuters Labs About...  ...TBs of legal, regulatory, news, and tax data, we are building the next generation of...  ...and contextual retrieval techniquesBuild evaluation frameworks and benchmarks for search... 
    Data
    Full time
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    1 day ago
  • $295k

     ...customers. Cohere is a team of researchers, engineers, designers, and...  ...Seoul. Join us!Role Overview:Evaluation is critical to making progress...  ...LLM judges; refining LLM-based data synthesis pipelines; and...  ...experience, we still encourage you to apply. We strive to create an... 
    Data
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    New York, NY
    2 days ago
  • $172.4k - $223.4k

     ...the Measurement, Ad Tech, and Data Science (MADS) team of Amazon...  ..., and Computer Vision.As an Applied Scientist on the team, you will...  ...problems to propose clear evaluation frameworks and success criteriaWork...  ...scientists across Applied, Research, Data Science and Economist... 
    Data
    Flexible hours

    Amazon

    New York, NY
    3 days ago
  • $218.7k - $249.6k

    Applied Researcher 4 (AI Foundations, LLM Customization and Finetuning) At Capital One, we are creating...  ...with a cross-functional team of data scientists, software engineers, machine...  ...development, from design through training, evaluation, validation, and implementation.... 
    Data
    Full time
    Part time
    Local area
    Flexible hours

    Capital One

    New York, NY
    2 days ago
  • $146.8k - $272.6k

     ...excited about working at the forefront of applied research in an industry setting? TR Labs is the...  ...technical judgment and evaluation expertiseExcellent communication skills...  ...insights to empower professionals with the data, intelligence, and solutions needed to... 
    Data
    Full time
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    2 days ago
  •  ...platform, we are building a Data + AI team to unlock the next...  ...Role We're looking for an Applied AI Scientist who will design...  .... You'll bridge research and production, working on the...  ...Validate Models: Build and evaluate experimental AI models using... 
    Data
    Temporary work
    Remote work
    Work from home
    Worldwide
    Home office
    Flexible hours

    AppGate

    New York, NY
    1 day ago
  • $135.66k

     ...and access information. Our Applied Science team is at the forefront...  ...yourself in groundbreaking research, exploring novel machine learning...  ...repositories of unstructured data, fueling the next generation...  ..., and experimentally evaluate new recommendation and search... 
    Data
    Full time
    Internship
    Worldwide
    Relocation

    Amazon

    New York, NY
    20 hours ago
  • $197.3k - $313.7k

     ...experience, please consider applying for a maximum of 3 roles within...  ...directly enable world-class research and products used by millions...  ...services, like APIs, UIs, agentic evaluators, and more.Roll out scalable...  ...how we use your personal data and your rights, including with... 
    Data
    Full time

    Salesforce

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Applied Research — Evaluations & Data. Be the first to apply!