LLM Systems Engineer: AI Agents & Evaluation
ByteDance
ByteDance is hiring in Seattle for roles within the Applied Machine Learning Ark team. You will help design, develop, and operate MaaS platforms using LLMs and multimodal models, spanning training, inference, prompt engineering, and evaluation pipelines. The team builds end-to-end solutions, analyzes large-scale log data, and collaborates across research, platform, and product groups to drive product improvements. A strong background in ML and Python is essential. #J-18808-Ljbffr ByteDance
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the LLM Systems Engineer: AI Agents & Evaluation in Seattle, WA vacancy
- ...team. You will work on end-to-end LLM service platforms, from evaluation to production integration,... ...users. You will design evaluation systems for LLM-based agents, build benchmarks, and analyze logs... ...hands-on experience in large-scale AI systems and data-driven feedback...SuggestedInternship
$42.75 per hour
...Learning Ark team combines system engineering and machine learning... ...Large Language Model (LLM) service platforms... ...alignment, and intelligent agent systems. Beyond model... ...Language Models and AI Agent systems to join... ...Responsibilities Design evaluation systems for LLM-based...SuggestedHourly paySummer workInternship- Atlassian is seeking a Systems Engineer to build observability for AI-assisted software development. The role includes designing pipelines and working directly with Git to collect AI-generated code data. Ideal candidates will have systems engineering experience, particularly...Suggested
$124k - $280k
...in data and analytics engineering focus on leveraging advanced... ...implementing advanced AI and ML solutions to... ...algorithms, models, and systems to enable intelligent decision... ...prompt engineering for LLM outputs- Developing... ...with team members. We evaluate these factors...SuggestedFull timeH1b- ...join as a Machine Learning Engineer Intern focused on E-commerce... ...The role emphasizes LLMs, agents, and retrieval systems applied to complex logistics... ...Interns will contribute to AI-native capabilities,... ..., workflow automation, and evaluation frameworks within the TikTok...SuggestedInternship
- TikTok is seeking a Machine Learning Engineer Graduate for 2027 start in... ...Logistics team. You will build AI-native capabilities for logistics, including agents, LLM model development, and integration... ...integration with forecasting and decision systems. The role requires PhD-level...
$185k - $230k
...and passionate Senior Software Engineer to join our highly skilled... ...sophisticated enterprise search system, spearheading the design, development... ...the best user experience. Evaluate and optimize algorithm... ...major advantage. About Otter.ai We are in the business of shaping...- ByteDance Networking is seeking talented researchers and engineers to design and deploy high-speed network technologies supporting AI/LLM applications at massive scale. You will design and develop platforms for monitoring, analysis and diagnosis of large-scale AI networks...
$180k - $225k
Remitly, Inc. in Seattle is seeking a Senior AI Native Software Engineer to orchestrate AI agents in building complex production-grade systems. You'll lead efforts in rebuilding legacy systems and guide teams on effective AI tools. The ideal candidate has proficiency in...$236k - $339.2k
...this new era, we seek AI-native thinkers across... ...intersection of autonomous agents and large language... ...will build the models and systems that turn enterprise data... ..., including:Arctic LLM: Enterprise foundation... ...coordinationEstablish evaluation standards for reliability...Shift work$200.8k - $251k
A leading AI technology company in San Francisco seeks a team member to build and optimize a machine learning... ...for large language models. Candidates should have system optimization experience and solid software engineering skills, particularly in tools like CUDA and Pytorch...Full time- We are hiring an AI Research Scientist (New Grad... ..., self‑improving AI systems — building agents that reason, code,... ...strategies Build and evaluate autonomous research agents... ...experience in LLM post‑training, fine‑tuning... ...researchers and engineers building the next generation...
$114.1k - $268.18k
...KPMG is currently seeking a Manager, SAP AI & Agent Governance to join our Advisory... ..., and human oversight requirements Evaluate risks associated with AI-enabled business... ...Bachelor'sdegree in business, information systems, engineering, technology, or relatedfield preferred...Full timeH1bLocal areaKPMG
Seattle, WA7 days ago$148.5k - $293k
...complete and complex framework, system or product. Defines processes... ...a Systems Design/Architecture Engineer and play a pivotal role in... ...structured mentoring programs.Evaluate new technologies and determine... ...ideas into impact, powered by AI and driven by human ingenuity....Full timeTemporary workWork at officeLocal areaWorldwide$127.4k - $191.2k
...opportunityUnity's Vector AI team builds the machine learning systems that decide which ads... ...the world's leading game engine. Recommendation and ranking... ...doingDesign, build, and evaluate next-generation ranking and... ...reinforcement learning, LLM post-training or alignment...Full timeInternshipWork at officeWorldwideShift work- ...DC, United StatesProducts - Engineering /Fulltime /HybridOver 50,000... ...Product Engineer /Staff SW Systems EngineerExperience : 8 to 14... ...Python,TypeScript or JavaScript,AI coding agents Claude Code, Copilot, Cursor... ...workflows where you direct, evaluate, and iterate on agent-...Full time
$90k - $180k
...ResponsibilitiesAI Systems & Agentic WorkflowsBuild agentic AI services (planning,... ..., memory, tooling, evaluation, and guardrails for... ...design reviews and engineering best practices. Mentor... ...experience with agent frameworks (e.g., LangGraph... .... Familiarity with LLM and embedding...Full timeTemporary workPart time$130.4k - $195.6k
...content into other game engines and 3D environments —... ....We’re building systems that allow real-time 3... ...orchestration by future AI-driven tools and systems... ...training, data collection, evaluation loops, agents, tool use, or model-... ...tool interfaces, LLM agents, multimodal models...Full timeWork at officeRemote workWorldwide$145k - $200k
...RoleWe are seeking an experienced Software Engineer to join a newly-formed team focused on developing and deploying advanced AI systems to the tactical edge. This role involves training, evaluating, and deploying LLM agents to edge hardware for interfacing with sensors...Full timeWork experience placementWork at officeRemote workWork from homeRelocation package- ...Irvine team is where embodied AI meets real robots, real... ...aware, reliable, field-ready AI systems that solve the hardest problems... ...architectures, combining rigorous engineering with learning systems proven... ...processing, model training, evaluation, and post-processing...Local area
FieldAI
Seattle, WA13 days ago - ...Description Senior Software Engineer Job Type: Contractor... ...to support an AI training project by creating... ...environments that evaluate AI models on complex software... ...verification systems and golden reference solutions... ...solutions. Evaluate AI agents' ability to reason...Remote jobFor contractors
$168.1k - $227.4k
Lead our pioneering AI initiative at AWS and define the future of AI agents. Shape how businesses leverage... ...& orchestration, agent evaluation, or knowledge... ...projects requiring multiple engineers, balancing business goals... ...) of new and existing systems experience- Experience...InternshipFlexible hoursDay shift$146.88k - $220.32k
...Are: We are a small engineering team at the Allen Institute for AI working on AI for... ...understanding the system, not blaming individuals... ...We are building an agent platform for... ...something breaks, and the evaluation tooling that tells... ...shipping LLM agents to production...Contract workWork at officeLocal areaFlexible hoursWeekend work- Expedia Group is seeking a senior AI/ML Engineer to design and deploy agentic, multi-step AI systems. You will architect autonomous agents, integrate LLMs with tools and data sources, and define evaluation frameworks for reliable, scalable performance. You will mentor engineers...
$152k - $240k
...effortlessly. Brex’s AI-native automation and... ...need to grow your career.Engineering at BrexEngineering at... ...Brex is about building systems that scale with speed... ...ll doWe're building AI agents to automate and... ...and data sources.Define evaluation frameworks, success metrics...Work at officeRemote workWork from home$200.1k - $270.6k
Principal Engineers provide technical leadership at Amazon... ...architecturally sound systems that are aligned with... ...direction for agentic AI systems that power Kiro... ...distributed systems and AI agent design, defining how... ...workflows across multiple LLM models with low latency...InternshipFlexible hours$134.25k - $214.8k
...matter.Your ImpactAre you an engineer who gets excited about the... ...making complex distributed systems observable — not just... ...engineeringExperience with agentic AI tooling or building LLM-powered developer... ...information from applicants to evaluate candidates for employment....Work experience placementWork at officeRemote work- ...Performance Consulting is seeking an Engineer to support the development, integration, sustainment, and evaluation of software, databases, data... ..., interoperability with DoD systems, data processing, technical... ...architecture, automation, analytics, or AI‑assisted methods. Experience...For contractorsFlexible hours
H2 Performance Consulting
Seattle, WA1 day ago $90.1k - $191.8k
...the world! The Data Labeling Engineering team designs, builds, and operates... ...of ML teams to train and evaluate the models behind advanced... ...leadership, and work directly on systems that unblock the next... ...and latency goals. Champion AI‑assisted engineering Use and...Work experience placementLocal areaWork from homeFlexible hours- ByteDance in Seattle is seeking a Senior Research Engineer/Scientist for Storage for LLM to design and maintain a high-performance KV cache layer for LLM inference across GPUs and nodes. This role focuses on reducing latency, increasing throughput, and lowering cost for...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to LLM Systems Engineer: AI Agents & Evaluation. Be the first to apply!
Related searches
- healthcare systems engineer Seattle, WA
- system engineer contract Seattle, WA
- senior linux systems engineer Seattle, WA
- systems engineer Seattle, WA
- computer system validation engineer Seattle, WA
- distributed systems engineer Seattle, WA
- space systems engineer Seattle, WA
- visual systems engineer Seattle, WA
- director systems engineering Seattle, WA
- operating system engineer Seattle, WA