Staff AI Evaluation Architect
Uncover
Uncover is building a central evaluation framework to measure model quality across teams. You will design reusable pipelines, benchmarks, and baselines, and build visualization tools to turn results into actionable insights for stakeholders. The role demands strong software engineering, experience with data/evaluation pipelines at scale, and the ability to translate research outputs into standardized metrics across domains. You’ll own the end-to-end delivery from requirements to deployment. #J-18808-Ljbffr Uncover
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Staff AI Evaluation Architect in Brooklyn, NY vacancy
- Role Overview As the AI Native Staff Architect, you will help define the future of our platform by balancing long-term architectural vision with practical execution. Working with engineering, product, and business leaders, you'll promote modern engineering practices, foster...SuggestedRemote jobWork at officeLocal areaFlexible hours
$129k - $152k
About the Role: At Fetch, we’re building AI and automation systems that make our... ...reliability, and measurable impact. As a Staff AI Evaluation Lead, you’ll own automation and... ...workflows or teams Design scalable solutions: Architect end-to-end workflows integrating...SuggestedFull timeRemote work$80 per hour
A leading AI project specialist company is seeking experienced professionals for project-based opportunities. Candidates will focus on reviewing coding tasks, writing comprehensive tests, and analyzing AI model behaviors in a flexible environment. With rates up to $80/...SuggestedFlexible hours- ...pipelines, training at scale, and real-time inference. You will own a unified system, data, architecture, training, inference, and evaluation, pushing the limits of current capabilities. The role requires hands-on experience training large models, multi-node scaling, and...Suggested
$170k - $200k
...Project, an education nonprofit, seeks an architect to build an end-to-end generative video... ...do Responsibilities Design the generative AI pipeline to create deterministic lessons-... ...quality assurance systems with rubric-based evaluation and ground-truth verification Design...SuggestedRemote workWorkman Labs
Brooklyn, NY3 days ago$61k - $101k
...leading safe adoption of enterprise-authorized AI capabilities within architecture... ...metrics, and architectural guardrails. We evaluate emerging capabilities for enterprise... ...ITSM Marketing Network Security Architect CMDB More: We are JPMorganChase,...Full timeJ.P. Morgan
Jersey City, NJ2 days ago$142.6k - $261.5k
...to build a better working world. ServiceNow– ServiceNow AI Architect Manager In the digital economy, it takes more than good... .... Exercise judgment in selecting methods, techniques, and evaluation criteria for obtaining results. Provide direction and feedback...Summer holidayWorldwideFlexible hoursEY
Secaucus, NJ4 days ago$171.6k - $392.1k
...to build a better working world. ServiceNow – ServiceNow AI Architect Senior Manager In the digital economy, it takes more than... ...a clients' enterprise capabilities with ServiceNow or evaluating how AI can streamline service delivery. Wherever you find yourself...Summer holidayWorldwideFlexible hoursEY
Hoboken, NJ4 days ago$90 - $92 per hour
AI Strategy & Enterprise Architect Birmingham, AL (Hybrid Preferred) Contract-to-Hire $90 - $92 per hour ABOUT THE ROLE Our client is seeking an experienced... ...AI and emerging AI technologies across the organization Evaluate platforms, frameworks, and cloud solutions to recommend...Hourly payPermanent employmentContract work- ...Utility Structures are seeking a highly motivated and innovative AI Solutions Architect to lead the identification, development, and implementation... ...standards and specifications. Responsibilities: Identify, evaluate, and implement AI solutions that improve manufacturing,...
- Job Description The Gen AI Architect will leverage advanced Generative AI models and Azure OpenAI services to develop innovative solutions... ...requirements into scalable AI-driven technical solutions. Evaluate and select appropriate AI/ML tools, cloud services, frameworks...Permanent employmentContract workLocal area
- ...Systems Engineer to design, test, and improve prompts powering AI agents across enrollment, learning, and student support. You’ll... ...translate educational goals into reliable agent behavior and build evaluation frameworks to prove effectiveness. You will collaborate with...
$85.12 - $95.4 per hour
...thinking organization, We are currently seeking a Distinguished AI Engineer to join our team in Jersey City, New Jersey (US-NJ), United... ..., and guardrails. LLMOps, AI gateways, model-serving strategy, evaluation platforms, observability, governance, and operating standards....Temporary workFlexible hours$87k
HCLTech is seeking an experienced AI Architect to lead the design and implementation of next-generation Artificial Intelligence solutions... ..., inference costs, latency, and operational KPIs. Establish evaluation frameworks for model quality and reliability. Implement...Local area$240k - $315k
Applied AI Architect, Strategic Enterprise Tech About Anthropic Anthropic’s mission is to create... ...journey, from discovery to initial evaluation through deployment. You will need to coordinate... ...hybrid policy: Currently, we expect all staff to be in one of our offices at least 25%...Work at officeVisa sponsorshipFlexible hours- Job Description - Enterprise Agentic AI Architect Day to Day job Duties: As an AI Architect, You will design and build agentic AI systems... ...architectural patterns for agent governance, observability, evaluation, and safety controls in enterprise environments. Define...Work experience placementWork at officeRemote workFlexible hours
$170k - $200k
Gen AI Architect Date: 2 Sept 2026 Company: QualityAI Country/Region: US Are you interested in working with the World’s leading AI-first... ..., tools, and frameworks to standardize model deployment, evaluation, and governance. Customer Engagement & Solutioning Partner with...Casual workLocal areaFlexible hours$176.72k - $265.08k
..., highly technical, and Hands-on Senior Principal Platform & AI Architect with 10+ years of experience to lead the design, development,... ...goals and conduct personnel duties for team (e.g. performance evaluations, hiring and disciplinary actions)Provide strategic influence...Full time- ...for people like you. As a Senior Principal Architect at JPMorgan Chase within Global... ...-first adoption of enterprise-authorized AI capabilities within the work environment... ...accelerate architecture analysis, option evaluation, and delivery of strategic global solutions...
- LangChain, Inc. is seeking a Deployed Engineer to work on production-grade AI agents for customer environments. You will co-architect and co-build agent systems, design POCs, and guide evaluations, ensuring robust, scalable deployments. You’ll partner with engineering,...
$180k - $240k
...Senior $180k - $240k About the Role We are seeking a Senior AI Agent Architect to design and deploy autonomous, multi-agent systems using... ...LLMs with external enterprise databases and APIs Define evaluations, safety guardrails, and reinforcement loops for agent tasks...Full time$125k - $160k
...looking for a motivated Knowledge Architect Manager to support these... ...and research look like in an AI-enabled practice. MSK is looking... ...analysis, and cost justification. Evaluate new AI and research tools as... ...research. Guide attorneys and staff across all levels of AI...Contract work- QualityAI is seeking a Gen AI Architect to lead design and delivery of enterprise-grade AI solutions powered by LLMs, embeddings, and RAG... .... You will guide cross-functional teams, define tech roadmap, evaluate frameworks (LangChain, LangGraph, OpenAI Agents SDK), ensure compliance...
- Step into the role of a Principal Architect at JPMorganChase and become a driving force behind... ...that power the firm's most critical AI and platform capabilities.As a Principal... ...awareness of data sensitivityAbility to evaluate and integrate AI-enabled capabilities into...
$142.32k - $213.48k
...ProfessionalCompany: CitiThe Senior Python Architect/Developer is responsible for establishing... ...problems/projects through in-depth evaluation of complex business processes, system processes... .... Our automated processing and AI do not involve relying on automatic or autonomous...Full timeWork experience placement$200k - $300k
AI Architect, Principal Frontline AI Engineer - HealthcareAbout Brillio: Brillio is one of the fastest growing digital technology service... ....• Experience with AI governance, security, monitoring, evaluation frameworks, and responsible AI practices.• Healthcare industry...- Mitchell Silberberg & Knupp seeks a Knowledge Architect Manager to lead the evolution of our research and knowledge management into an AI-powered legal practice. This hands-on... ...with the CAIO on tool strategy and evaluation; MSK offers a competitive salary and comprehensive...
- ...companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM... ...role and responsibilities We are seeking a Senior AI Architect - Enterprise Integrations to join our growing AI practice. As...Full timeTemporary workPart timeWorldwide
IBM
Jersey City, NJ2 days ago - ...JPMorgan Chase & Co. is seeking a Senior Principal Architect to own target-state architectures for data platforms spanning Reference Data... ...and trade data. You will drive modernization, governance, and AI adoption across a firmwide portfolio and interact with senior leaders...
- ...JPMorgan Chase & Co. in Jersey City is seeking a Senior Principal Architect to shape our target state architecture and strategy within... ...governance, delivering durable frameworks, and driving adoption of AI capabilities with proper validation and data sensitivity controls...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff AI Evaluation Architect. Be the first to apply!
