Signal, Data & Evaluation Engineer: Benchmarks & Datasets
Jobleads-US
Matter Intelligence is hiring a Signal, Data, and Evaluation Engineer to build datasets, benchmarks, environments, and evidence for defensible models and AI systems. You will work across data curation, simulation and learning environments, model and agent evaluation, and release-quality evidence.
This role reports to a designating leader and requires building versioned datasets, testing, and clear communication of evidence supporting releases in an onsite San Francisco environment.
#J-18808-Ljbffr Jobleads-USVacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Signal, Data & Evaluation Engineer: Benchmarks & Datasets in San Francisco, CA vacancy
$163k - $208k
...ML Data Engineer San Francisco, CA · In Person · Full... ...systems where these signals originate, and across... ...ll see quickly when a dataset you've built allows them... ...pipelines, or evaluation operations. You'... ...equity We regularly benchmark compensation against...SuggestedFull timeFlexible hoursVizcom
San Francisco, CA5 days ago$80 per hour
...San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel ,... ...Summers , and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type:... ...frontier AI coding agents to complete and evaluate complex data engineering tasks....SuggestedContract workSummer workRemote workMercor
San Francisco, CA10 days ago- ...built the fastest-growing AI data business in history. We... ...lab researchers to create evaluations, publish benchmarks, and push the boundary of... ...institutions Work together with engineers, scientists, operators,... ...inference, evaluation datasets, and agent-facing data....SuggestedFull timeWork at officeLocal areaRemote workFlexible hours
Apply
San Francisco, CA3 days ago - ...looking for a Senior Machine Learning Engineer to join our Search & Intelligence... ..., agentic systems, models, evaluation frameworks, and data platforms that power Atlassian’s AI... ...define quality, build evaluation datasets and benchmarks, analyze model behavior, and use results...SuggestedWork at officeLocal area
Atlassian
San Francisco, CA5 days ago - ...Cartesia is hiring a Research Engineer, Data Infrastructure to build datasets and systems powering pretraining for foundation models. You will develop scalable, high-throughput data pipelines and collaborate with research to optimize data mixtures for model quality....SuggestedWork at officeVisa sponsorship
$155k - $210k
...function at Zipline with the data they need to optimize... ...As a Senior Analytics Engineer, you are the owner of... ...infrastructure and datasets that directly inform... ...aircraft availability signals into a single fleet health... ...harness: add evaluation tests, ground-truth datasets...Local areaFlexible hours3 days per weekEarly shiftZipline
San Francisco, CAa month ago$130k - $220k
...AI Systems · Benchmarking · San Francisco Member of Technical... ...how modern AI is evaluated, understood, and... ...shape the frameworks, datasets, and metrics that the... ...on problems that most engineers never get to touch.”... ...work Experience with data analysis and building...$130k - $220k
...leading independent AI benchmarking and insights company. They help engineers, enterprises, investors... ...best described as an AI Evaluation Engineer / Technical... ...benchmarking methodologies and datasets that improve evaluation... ...analysis, and data visualizations that help...Full timeWorldwideAurora Jobs ApS
San Francisco, CAmore than 2 months ago$150.4k - $225.3k
...Software Engineer - Generative Data Platform, Evaluation San Francisco, California, United States Software and Services Apple's machine learning features... ...ideas, text, and images into large-scale generated datasets. We don't produce the datasets ourselves: ML...Work experience placementRelocation$120k - $160k
...Data EngineerLos Angeles; New York; Remote; San FranciscoEDO is... ...combining immediate engagement signals with world-class decision... ...EDO Press.The RoleAs a Data Engineer at EDO, you will join a team... ...and analyzing a wide range of datasets, including advertising occurrence...Full timeWork experience placementWork at officeImmediate startRemote workEDO
San Francisco, CA3 days ago- ...Data Engineer Role at StuutStuut is transforming accounts receivable for B2B companies—making... ...translate messy data into clean, reliable datasets that drive product decisions, customer... ...built in from day oneBuild the event and signal pipelines that turn product interactions...Full time
Stuut
San Francisco, CA3 days ago - ...the largest proprietary radiology report datasets in the world, our AI has helped uncover... ...Why Join Us: We are seeking a Senior Data Engineer to join our engineering team. With our... ...validation, monitoring, and access controls. Evaluate and recommend data technologies and...Full timeWork at officeRemote workFlexible hours
Engg
San Francisco, CA2 days ago $180k - $220k
...Data EngineerSan Francisco (Hybrid)At You.com, we are building... ...and useful.Our team includes engineers, researchers, product builders... ...systemsCreate and manage curated datasets to support analytics,... ...each candidate and applicable evaluation criteria.If AI-assisted note-...Full timeImmediate startRemote workWork from homeY.O.U.
San Francisco, CA4 days ago- ...locations for team members for our Data Platform roles are our San... ..., we’re looking for a Data Engineer to join our Data Foundations... ...rich data opportunity, with signals spanning our full product suite... .... It owns the foundational datasets and data models that define the...WorldwideHome officeFlexible hours
Superhuman
San Francisco, CA4 days ago - ...locations for team members for our Data Platform roles are our San... ...its business? As a Data Engineer on the Finance & Revenue team... ...own the pipelines, models, and datasets that power revenue reporting... ...customer, and product-usage signals into reliable, decision-grade...WorldwideHome officeFlexible hours
Superhuman
San Francisco, CA4 days ago - ...Senior Data EngineerSan FranciscoRZR Global is an AI-driven company... ...of contextual bidding signals and proprietary behavioral models... ...is seeking a talented Data Engineer to join our growing engineering... ...data engineering initiatives.Evaluate and introduce new data...
RZR Global Inc.
San Francisco, CA4 days ago - ...Data Engineering RoleAs our first dedicated data engineering hire, you'll own the full data stack... ...raw event streams into clean, trusted datasets.Pipeline orchestration. Build and... ...short: live game economics, real-time risk signals, and in session player behavior.Reverse...
Triumph
San Francisco, CA3 days ago - ...HybridWe’re looking for a talented Sr. Data Engineer to join our new Data Platform product team... ...multi-step agent workflowsBuild the evaluation harness before the feature — define what... ...setting experience is preferred.Large Datasets: Skilled in processing and extracting value...Ongoing contractFull timeCasual workWorldwideFlexible hours
SS&C Technologies
San Francisco, CA19 hours ago $185.5k - $232k
...Senior Data EngineerFormation Bio is a tech and AI driven pharma... ...the PositionAs a Senior Data Engineer at Formation Bio, you will... ...clinical operations, drug asset evaluation, business development,... ...with Data Science on training datasets and ML use cases, and with human...Work at officeLocal areaRelocation3 days per weekFormation Bio (Formerly TrailSpark)
San Francisco, CA1 day ago$227.33k - $312.58k
We’re looking for a Staff ML Data Engineer to join Procore’s AI & Frontier Models organization... ...transform, and operate on large‑scale datasets that move from experimentation to... ...opportunity to shape how Procore builds, evaluates, and deploys frontier models by ensuring...Full timeWork at officeLocal areaImmediate start3 days per weekProcore Technologies
San Francisco, CA4 days ago$295k
...Data Engineer, Hardware & Consumer Devices | OpenAI Careers Data Engineer, Hardware &... ...scalable data pipelines, platforms, and datasets that support hardware development, manufacturing... ..., reliability, returns, and failure signals. Prepare for scale: Design systems...Work at officeLocal areaImmediate startRelocation packageFlexible hours- ...largest real-time compensation dataset with deep expertise in AI and... ...of Forbes AI 50 use Pave to benchmark compensation.The future of... ...Partners, and Craft Ventures.The Data Team @ PaveAs part of the... ...product managers and software engineers to translate product needs into...Flexible hours
Trove Information Technologies
San Francisco, CA5 days ago - ...professional profiles, and signals from 100+ labor... ...Silicon Valley engineering team — a small,... ...and unstructured datasets. • Design experiments and benchmarks to measure model accuracy... ...— to real-world data pipelines. •... ...detection. • Build evaluation frameworks for AI-...Work at officeVisa sponsorship
Hop
San Francisco, CAa month ago $150k - $250k
...ForAt Distyl, Research Engineers build the bridge... ...system behavior, build evaluation frameworks, and... ...of AI systemsDevelop datasets, preference signals, evaluation suites,... ...modeling, synthetic data, evals, or related post... ...not just whether a benchmark improvesAI Systems Mindset...Work at office3 days per weekDistyl AI
San Francisco, CA2 days ago- ...believe culture can be engineered – but when it falls... ...We're looking for a data engineer to help us turn... ...queryable, and trustworthy datasets that power our... ...queryable for ML training, evaluation, and debugging Turn... ...data into the derived signals, annotations, and aggregates...Local areaFlexible hours
- ...As a Senior Data Engineer at Regard, you will own the design, development, and production deployment... ...with engineering teams, ensuring datasets are trustworthy and production-ready... ...validation, deployment, and ongoing performance evaluation Hybrid Work | Location | Work...Work at officeLocal areaHome officeVisa sponsorshipRelocation package
$159.2k - $301.6k
About the roleAdobe’s Security Data Platform team builds and... ...analytics.As a Staff Security Data Engineer, you will own full-system... ...Preserve source fidelity and make datasets useful for investigations,... ...through technical assessment.Evaluate and introduce useful platform...Full timeTemporary workLocal areaWorldwideAdobe Systems
San Francisco, CA1 day ago$123.7k - $254.67k
...recruiting process here.Job Title: Software Engineer II, Data Analytics and EngineeringIntro: We’re... ...trusted, self-service access to datasets, insights and metric investigations across... ....Strong track record of critically evaluating and verifying AI-assisted work through...Work at officeLocal areaRemote workRelocationRelocation packagePinterest
San Francisco, CA1 day ago- ...Machine Learning Engineer to join our Growth... ...behavioral and product signals into scalable ML... ..., engineering, data science, analytics... ...features, models, evaluation frameworks, and decisioning... ...; building datasets, features, and... ...datasets and evals to benchmark systems operating...Temporary workWork at officeLocal area
Atlassian
San Francisco, CA5 days ago $112k - $269k
SummaryYelp engineering culture is driven by our values... ...for turning raw data into valuable signals and building ML systems... ...textual and visual datasets.Support the... ...prompt engineering and evaluation.A Bachelor’s Degree... ...to external market benchmark data, geographic location...Work experience placementLocal areaYelp
San Francisco, CA5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Signal, Data & Evaluation Engineer: Benchmarks & Datasets. Be the first to apply!
Related searches
- data engineer machine learning San Francisco, CA
- aws data engineer San Francisco, CA
- data systems engineer San Francisco, CA
- big data developer San Francisco, CA
- sr data engineer San Francisco, CA
- big data cloud engineer San Francisco, CA
- finance data engineer San Francisco, CA
- junior big data engineer San Francisco, CA
- entry level data engineer San Francisco, CA
- sr information security engineer San Francisco, CA