Member of Technical Staff - ML Evaluation & Benchmarking
Arena Physica
Arena Physica is hiring a Member of Technical Staff for Evaluations to lead the design, verification, and automation of evaluation pipelines for hardware-focused AI systems. You will craft analyses, calibrate scoring, and report findings to CTO and partners. The role spans infrastructure, statistics, and domain knowledge, with collaboration across electrical engineering and product teams. This is a high-impact role based in the New York area. #J-18808-Ljbffr
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Member of Technical Staff - ML Evaluation & Benchmarking in New York, NY vacancy
- ...Francisco or New York · Full-time Member of Technical Staff, Research Competitive salary +... .... Develop behavioral‑fidelity benchmarks. Train and evaluate agents against human data.... ...Qualifications Led research in agents, ML, simulation, or cognitive science....SuggestedFull time
Primitive Labs AI Inc.
New York, NY2 hours ago $200k - $300k
...Member of Technical Staff, Product, Core Product at Profound — New York, NY... ...monitoring and competitive benchmarking to agent-driven automation... ...fundamentals with a spike in data, AI/ML systems, development,... ...creating prototypes, writing SQL, evaluating model outputs, building...SuggestedFull timeWork at officeImmediate startNeolife Updates
New York, NY4 days ago$200k - $350k
...their most consequential data, evaluation, and post-training... ...The Role We are hiring a Member of Technical Staff, Evals to help define how frontier... ...customers to develop new benchmarks, reward and verifier... ...we're looking for PhD in ML/AI, computer science, data...SuggestedWork at officeFlexible hoursHandshake
New York, NY1 day ago- ...Req ID: JR112532 Member of Technical Staff, Agentic AI Architect Our vision... ...performance profiling and evaluation to turn massive data and compute... ...evaluation harnesses to benchmark agent accuracy, reliability... ...spanning large-scale data/ML pipelines and/or production...SuggestedLocal areaRelocation
Micron Technology
New York, NY4 days ago - ...Member of Technical Staff $1,000,000-$1,750,000 NYC (Hybrid) Company Overview: Join a confidential, high-performing... ...scalable deployments Own the full stack of ML workflows: data ingestion, training, evaluation, deployment, and monitoring Improve platform...SuggestedFull time
People In AI
New York, NY2 days ago $400k
...Remote | Member of Technical Staff, Research Engineering — $400,000–$800,000/year We are... ...in reinforcement learning, ML-oriented data systems, evaluation infrastructure, and scalable experimentation... ...model performance Develop benchmarking frameworks measuring capability,...Full timeRemote work24-Mag Llc
New York, NY3 days ago- ...team closes the gap between benchmark performance and useful performance... ...model usage into rigorous evaluations, targeted training data, and... ...models. This is a hands-on technical role at the intersection of... ...evaluation pipelines or large-scale ML systems A track record of...Work at officeVisa sponsorship
Reflection
New York, NY2 hours ago $250k - $300k
...precision inference useful. Evaluate weight and KV-cache quantization... .... Maintain reproducible benchmarks and numerical checks across supported... ...on it. Deep experience in ML inference, GPU programming,... ...budgets. Evidence of technical depth matters more to us than...Local areaVisa sponsorshipRelocation packageFlexible hoursSonder
New York, NY2 hours ago$200k - $300k
...ée Lauder, and Walmart - is hiring a Member of Technical Staff, Product, Core Product in New York,... ...fundamentals with a spike in data, AI/ML systems, development, design, or enterprise... ...to create prototypes, write SQL, evaluate model outputs, build dashboards, create...Full timeWork at officeNeolife Updates
New York, NY2 hours ago$200k - $300k
...Commure, and Paradigm. The technical problems are concrete: find... ...About the role As a Member of Technical Staff in Machine Learning, you will... ...labels, and keep held-out evaluation cases separate from training... ...'re looking for Strong ML fundamentals and hands‑on implementation...Full timeH1bWork at officeVisa sponsorshipRelocation packageSolstice
New York, NY2 hours ago$250k - $350k
...available. Responsibilities Design and own benchmarks, evals and dataset taxonomies setting the... ...autonomy Tech stack: Python, PyTorch, LLMs, Evaluation Frameworks, Data Pipelines Source: PhD in ML/CS/CV or quantitative science (strongly preferred...Work at officeVisa sponsorshipFlexible hoursdeCircle
New York, NY5 days ago- ...and scenarios that push agents past static benchmark behavior. You’d design tasks, tools, validators, reward signals, and evaluation harnesses that test meaningful capabilities... ...building with PyTorch, FastAPI, and modern ML infrastructure. You can work at a high level...
Embedding VC
New York, NY3 days ago - ...task classification, capability profiles, evaluation harnesses, and the cost governor that... ...evaluation, and cost control. Strong applied ML or systems background with rigorous... ...pricing behaviour. Skepticism about benchmarks and the discipline to construct better...
Dipp AI Technologies, Inc.
New York, NY2 hours ago $270k - $350k
...AI economy — and the company is hiring a Member of Technical Staff, Coding Agents to help define how frontier coding agents are benchmarked and improved. Based in New York, NY,... ...labs on their most consequential data, evaluation, and post-training challenges. The team...Full timeNeolife Updates
New York, NY4 days ago$200k - $320k
Job Title: Member of Technical Staff, Robotics Research Job Type: Full-time Location: Remote The Role... ...the datasets, collection methods, and evaluation frameworks that power next-generation... ...multimodal learning, embodied AI, or applied ML research. Strong understanding of...Full timeRemote work$300 per month
...racing to deploy the most advanced applied ML at production scale. Our investors... ...Delangue and many other operators/technical leaders. "Basis is on the frontier of... ...Lead, OpenAI The Work Being a Member of Technical Staff at Basis means you'll face changing problems...Work at officeShift workgetbasis.ai
New York, NY2 hours ago$180k - $280k
...building the software to fix it. As a Member of the Technical Staff at Finch, you'll own critical... ...real cases and operator feedback into benchmarks that continuously improve agent quality... ..., and changing case information. Evaluate advances in agentic AI and rapidly ship...Work at officeRemote workFlexible hours1 day per weekFinchlegal
New York, NY3 days ago$180k - $250k
...regression beats a neural net. You’ll ship ML that touches hardware: vision for... ...software yourself and through the other technical staff you coordinate on-site and at HQ. Write... ...notebook to production: feature pipelines, evaluation, monitoring, and retraining ~...Work at officeAIC
New York, NY2 hours ago- ...production. You'll turn validated methods, benchmarks, and evaluation infrastructure from our research team... ...experience building and shipping AI/ML-powered products, with deep... ...user impact and skilled at making smart technical trade-offs to deliver value quickly.Ownership...Contract workWork at officeImmediate start
C R O S B Y
New York, NY5 days ago $400 - $800 per month
...Role Overview Advance the evaluation and development of frontier coding agents by designing the benchmarks, methodologies, and data systems used to measure and improve... ...emerging model capabilities. Contribute to technical reports, benchmark studies, and client-facing...Full timeRemote workSaidGig
New York, NY1 day ago- ...Pro Integrate LLC is seeking a Member of Technical Staff, Enterprise AI, for a remote role. You will... ...surface failure modes, design datasets and evaluation protocols, run rapid experiments, and... ...contribute to research artifacts and benchmarks in a fast-iteration environment. #J-1...Remote work
Pro Integrate LLC
New York, NY4 days ago - ...needs to be correct every time, at scale, under pressure. As a Member of Technical Staff, you'll own critical pieces of this system end-to-end. You'... ...with FHIR, HL7, or clinical data standards Background in ML/AI systems or real-time inference pipelines Prior...
Convera AI, Inc.
New York, NY4 days ago - ...PyTorch and backed by AMD, Atreides, Benchmark Capital, Index Ventures, Lightspeed,... ...run on real fleets at real scale. As a Member of Technical Staff on the Systems Infrastructure team,... ...proficiency in Python Experience building and evaluating real systems — not only simulation —...InternshipLocal areaFlexible hours
Fireworks AI
New York, NY4 days ago $100k - $300k
...company with a flat organizational structure, where every team member is empowered to actively contribute. Leadership roles are earned... ...learning papers into production-ready code. Build robust model evaluation frameworks. Disseminate the results by co-authoring research...WorldwideAtaraxis AI
New York, NY1 day ago$160k - $300k
...ll be working on Build our evaluation systems. Because we can't... ...Experience testing non‑deterministic, ML, or LLM‑based systems, or the... .... You can explain technical risk to non‑technical people,... ...as part of the early team. We benchmark against top‑tier startups and...Permanent employmentH1bVisa sponsorshipRelocation packageNight shiftSolstice
New York, NY2 hours ago$125k - $190k
...Founding AI Engineer / Member of Technical Staff Location: New York, NY, United States Department... ...a proven track record of shipping ML/AI systems to real users and the ability... ...design sophisticated data pipelines and evaluation frameworks. You will collaborate...Permanent employmentFull timeTemporary workWork at officeRelocation packageAdvatix
New York, NY5 days ago- ...these technologies for enterprises, and we’re looking for a senior member for the Agent Code team. You’ll be at the forefront of research... ...on existing and new RL algorithms. Hillclimb on existing benchmarks and design new ones that reflect the needs of our enterprise...Full timeWork at officeLocal areaRemote workHome office
Cohere Inc.
New York, NY4 days ago - ...architectures, but from better data.As a member of the Data Team, your mission... ...the data used to train and evaluate our models meets a high bar... ...in Python and building ML / LLM workflows. Must be comfortable... ...to clearly articulate complex technical concepts across teamsWhat We...Work at officeVisa sponsorship
Reflection AI
New York, NY3 days ago $180 per hour
...ideas into scalable training, evaluation, and data-generation... ...training strategies. Establish benchmarking frameworks to assess model capability... ...frameworks. Experience with ML-oriented data design,... ...machine learning models. Clear technical communication and writing skills...Full timeRemote work- ...red-teaming and adversarial evaluation pipeline for Reflection’s models... ...scalable, automated safety benchmarks that evolve alongside our model... ...in AI Safety. Deep technical understanding of LLM safety,... ...evaluation pipelines or large-scale ML systems. Experience with Reinforcement...Relocation package
Reflection AI
New York, NY3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Member of Technical Staff - ML Evaluation & Benchmarking. Be the first to apply!
Related searches
- IT help desk technician New York, NY
- IT assistant New York, NY
- technical support representative New York, NY
- work from home technical support specialist New York, NY
- user support analyst New York, NY
- product support analyst New York, NY
- help desk administrator New York, NY
- remote support technician New York, NY
- support technician New York, NY
- decision support analyst New York, NY