Principal AI Architect
Yochana
AI Architecture & System Design
• AI system architectures: multi-agent orchestration layers, RAG pipelines, hybrid retrieval systems (knowledge graphs + vector search), text-to-SQL engines, and real-time inference APIs.
• Define and own technical blueprints for new AI products - from data ingestion and embedding pipelines through to response generation, evaluation, and production monitoring.
• Solve hard engineering problems: latency, precision/recall trade-offs, context window management, hallucination mitigation, and cost-efficient LLM usage at scale.
• Make deliberate, well-documented architecture decisions with clear trade-off analysis (build vs. buy, framework selection, deployment topology).
Implementation
• Write production-quality code - Python, SQL, API services - across the full AI lifecycle: data qualification, model training, evaluation, containerised deployment, and API serving.
• Build and own reusable, framework-quality components (chunking pipelines, retrieval layers, agent tool-calling modules) that accelerate team velocity.
• Own CI/CD pipelines, Docker-based deployment, and production telemetry for AI services.
AI Market Intelligence & Technology Strategy
• Track and evaluate the AI landscape - new LLMs, agentic frameworks (LangGraph, Google ADK, CrewAI, AutoGen), retrieval methods, fine-tuning techniques, and emerging tooling.
• Translate AI market trends into actionable roadmap inputs - surfacing opportunities for step change capability improvements before competitors do.
Cross-Functional Technical Partnership
• Partner closely with Product, Data Science, and Platform Engineering to align AI architecture with product direction, data constraints, and infrastructure capabilities.
• Communicate complex technical trade-offs clearly to non-technical stakeholders - translating architecture decisions into business impact narratives.
Must-Have Experience
• 12+ years of hands-on experience in AI/ML engineering and data science, with significant depth in production system delivery.
• Deep, working expertise in LLM application development: LangChain, LangGraph, tool-calling agents, RAG, prompt engineering, embedding pipelines, and hybrid retrieval.
• Proven track record architecting and shipping multi-agent systems, knowledge graph-powered retrieval (Neo4j or equivalent), and real-time inference APIs.
• Strong ML fundamentals: XGBoost, deep learning, NLP, time-series forecasting, propensity modelling, experimental design, and causal inference.
• Experience delivering AI systems in regulated industries (financial services, cybersecurity, healthcare) with SOX, GDPR, or SOC 2 compliance awareness.
• Expert-level Python and SQL; fluency with GCP, AWS, Docker, FastAPI, BigQuery, FAISS, and CI/CD tooling.
Technical Depth
• Ability to design hybrid retrieval architectures that balance precision (graph traversal) and semantic recall (vector similarity), with reranking layers - not just off-the-shelf RAG.
• Hands-on experience reducing LLM inference latency in production (e.g., redesigning pipelines from multi-minute to sub-30-second response times).
QUALIFICATIONS
• Master's or PhD in Computer Science, Operations Research, Statistics, or a related quantitative field
• AWS Certified Machine Learning Engineer or GCP Professional ML Engineer certification.
• Completion of an AI Strategy or AI Governance programme.
• Prior experience at a data science / ML services firm, enterprise SaaS, or fintech - where you shipped AI to external customers, not just internal tools.
• Hands-on experience with Snowflake Cortex or comparable enterprise LLM deployment platforms.
• Open-source contributions to AI/ML tooling, published technical writing, or conference presentations.
• AI system architectures: multi-agent orchestration layers, RAG pipelines, hybrid retrieval systems (knowledge graphs + vector search), text-to-SQL engines, and real-time inference APIs.
• Define and own technical blueprints for new AI products - from data ingestion and embedding pipelines through to response generation, evaluation, and production monitoring.
• Solve hard engineering problems: latency, precision/recall trade-offs, context window management, hallucination mitigation, and cost-efficient LLM usage at scale.
• Make deliberate, well-documented architecture decisions with clear trade-off analysis (build vs. buy, framework selection, deployment topology).
Implementation
• Write production-quality code - Python, SQL, API services - across the full AI lifecycle: data qualification, model training, evaluation, containerised deployment, and API serving.
• Build and own reusable, framework-quality components (chunking pipelines, retrieval layers, agent tool-calling modules) that accelerate team velocity.
• Own CI/CD pipelines, Docker-based deployment, and production telemetry for AI services.
AI Market Intelligence & Technology Strategy
• Track and evaluate the AI landscape - new LLMs, agentic frameworks (LangGraph, Google ADK, CrewAI, AutoGen), retrieval methods, fine-tuning techniques, and emerging tooling.
• Translate AI market trends into actionable roadmap inputs - surfacing opportunities for step change capability improvements before competitors do.
Cross-Functional Technical Partnership
• Partner closely with Product, Data Science, and Platform Engineering to align AI architecture with product direction, data constraints, and infrastructure capabilities.
• Communicate complex technical trade-offs clearly to non-technical stakeholders - translating architecture decisions into business impact narratives.
Must-Have Experience
• 12+ years of hands-on experience in AI/ML engineering and data science, with significant depth in production system delivery.
• Deep, working expertise in LLM application development: LangChain, LangGraph, tool-calling agents, RAG, prompt engineering, embedding pipelines, and hybrid retrieval.
• Proven track record architecting and shipping multi-agent systems, knowledge graph-powered retrieval (Neo4j or equivalent), and real-time inference APIs.
• Strong ML fundamentals: XGBoost, deep learning, NLP, time-series forecasting, propensity modelling, experimental design, and causal inference.
• Experience delivering AI systems in regulated industries (financial services, cybersecurity, healthcare) with SOX, GDPR, or SOC 2 compliance awareness.
• Expert-level Python and SQL; fluency with GCP, AWS, Docker, FastAPI, BigQuery, FAISS, and CI/CD tooling.
Technical Depth
• Ability to design hybrid retrieval architectures that balance precision (graph traversal) and semantic recall (vector similarity), with reranking layers - not just off-the-shelf RAG.
• Hands-on experience reducing LLM inference latency in production (e.g., redesigning pipelines from multi-minute to sub-30-second response times).
QUALIFICATIONS
• Master's or PhD in Computer Science, Operations Research, Statistics, or a related quantitative field
• AWS Certified Machine Learning Engineer or GCP Professional ML Engineer certification.
• Completion of an AI Strategy or AI Governance programme.
• Prior experience at a data science / ML services firm, enterprise SaaS, or fintech - where you shipped AI to external customers, not just internal tools.
• Hands-on experience with Snowflake Cortex or comparable enterprise LLM deployment platforms.
• Open-source contributions to AI/ML tooling, published technical writing, or conference presentations.
Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Principal AI Architect in San Jose, CA vacancy
- Cornelis Networks, Inc. is looking for a Senior Principal ASIC Design Engineer to develop world-class SoCs for high-performance computing and AI networking. The successful candidate should have extensive experience in digital design, specifically with Verilog/System Verilog...PrincipalRemote job
- Proofpoint is seeking a Principal ML Architect to lead the design and development of next-generation AI systems focused on cybersecurity. In this role, you will leverage advanced machine learning techniques, including LLMs and SLMs, to create intelligent security solutions...Principal
$172.8k - $304.9k
...Job Title SoC Architect for Next Generation AI Products in Datacenter Job Description Qualcomm is looking for an experienced SoC architect to work on the next generation AI products in the datacenter. We are looking for a data center engineer whose expertise...PrincipalWork experience placement$175k - $350k
TylSemi is seeking a Power Architecture Lead for PMIC in San Jose, CA. In this role, you will define and oversee the IVR architecture, ensure optimal performance across power domain specifications, and collaborate with foundry partners. Qualified candidates should possess...Principal- ...products that accelerate next-generation computing experiences-from AI and data centers, to PCs, gaming and embedded systems. Grounded... ...career. THE ROLE: We are seeking a Robotics AI Architect to define and scale next-generation Physical AI systems ,...Principal
- ...AI Solutions Architect Key Responsibilities Design comprehensive AI solutions integrating the latest AI technology developments. Lead architectural discussions with clients across AI PC, Edge, Data Centre, and Public Cloud. Drive AI and cloud architecture...PrincipalWork at office
- Advanced Micro Devices is looking for a Principal Engineer in Santa Clara, CA to lead AI infrastructure development, define GPU architecture specifications, and drive performance gains in ML systems. The role involves leading innovative techniques, collaborating with stakeholders...Principal
- ...Chance to be perm?: Yes Performance Expectations: Technical Hiring Criteria (Must Haves) • Top 3 Required skills: Python, LLM, AI agent frameworks (Google ADK, LangChain • Years of experience in each of the must-have skills: 8 • Any Certifications required:...PrincipalPermanent employmentWork at officeRemote work3 days per week
$190k - $280k
MixMode is seeking a Principal Architect in Santa Clara to accelerate AI application performance through innovative hardware and software solutions. The role involves analyzing ML workloads and collaborating with product and hardware teams to enhance inference accelerators...Principal$212k - $386.3k
...Principal AI Architect, App Store Data Apple's App Store is the world's largest and most innovative app marketplace, home to over 1.5 million apps and serving more than half a billion customers every week across all Apple devices. Since the App Store launched in 20...PrincipalRelocation$254k - $349.25k
...agent-centric cybersecurity. We protect how people, data, and AI agents connect across email, cloud, and collaboration tools.... ...in execution and impact Role Overview We are seeking a Principal ML Architect to lead the design and development of next-generation AI...PrincipalFlexible hours- ...next-generation computing experiences—from AI and data centers, to PCs, gaming and... ..., we advance your career. THE ROLE As a Principal Engineer, you will spearhead the next generation... ..., expert and data parallel dimensions Architect memory‑efficient training systems...PrincipalRemote work
$150k - $200k
Principal AI/ML Architect Job in USA 2025 (USD 150,000 to 200,000) Are you ready to elevate your career in artificial intelligence and machine learning? Mogi I/O is hiring a Principal AI/ML Architect to help lead innovative solutions in the industrial and automotive sectors...PrincipalFull time$225k - $245k
A leading tech company is seeking a Principal AI/ML Engineer to lead the technical strategy for AI systems' safety against misuse. Responsibilities include architecting defenses against harmful outputs and guiding evaluation strategies. The ideal candidate has over 7 years...Principal$254k - $349.25k
...agent-centric cybersecurity. We protect how people, data, and AI agents connect across email, cloud, and collaboration tools.... ...Exceptional in execution and impact Role Overview We are seeking a Principal ML Architect to lead the design and development of next-generation AI...PrincipalFlexible hours- ...global supply chain has access to the Flash memory it needs to keep our world moving forward. Job Description An AI Interconnect Architect defines and engineers high-speed networking and communication systems for AI Inference infrastructure which include servers...PrincipalTemporary workRemote workFlexible hoursShift work
- ...Matrix, we are focused on unleashing the potential of generative AI to power the transformation of technology. We are at the... ...headquarters 3 days per week. The role: Digital Design Engineer, Micro-Architect, Principal What you will do: As part of this team, you will be...PrincipalWork experience placement3 days per week
$219k - $351k
Principal engineer, AI Serving Framework Architect (Software) San Jose, California, United States Please Note: To provide the best candidate experience amidst our high application volumes, each candidate is limited to 10 applications across all open jobs within a 6-month...PrincipalWork at officeFlexible hours$272k - $431.25k
...Do you want to help drive the development of CPU technology for architectures used for artificial intelligence (AI), agentic workloads, deep learning (DL), high-performance computing (HPC), cloud service providers (CSP), gaming, virtual reality, and autonomous vehicles...Principal$270k - $300k
...Piper Companies is seeking an ASIC Architect to join a fast-growing innovator in AI infrastructure, for an onsite permanent position in Saratoga, CA . The ASIC Architect will be l eadin g the definition, modeling, and implementation of high-performance ASIC architectures...PrincipalPermanent employment$200k - $351k
A leading technology company in San Jose is seeking a Senior Principal, Design Engineering specialized in power solutions. Responsibilities include architecting power systems for networking and storage solutions, with a focus on multi-tiered designs. The ideal candidate...Principal$210k - $260k
...Astera Labs (NASDAQ: ALAB) provides rack-scale AI infrastructure through purpose-built connectivity solutions. By collaborating... ...requirements. Discover more at Role Overview As a Sr. Principal DSP Architect, you will be the technical visionary leading the definition...PrincipalFlexible hours$200k - $240k
A leading semiconductor firm is seeking a Principal Engineer for Photonics Reliability in San Jose, CA. The role involves conducting detailed failure analysis investigations and implementing corrective actions to enhance product reliability. Candidates should have a Master...Principal$232k - $368k
...and amazing people. We are seeking a Senior Power and Performance Architect to influence, innovate and drive next generation power... ...2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering...Principal$191.53k - $286.9k
...Senior Principal Engineer Marvell's semiconductor solutions are the essential building... ...our world. Across enterprise, cloud and AI, and carrier architectures, our innovative... .... Strong background designing or architecting complex ASICs, ideally in networking or...PrincipalPermanent employmentInternshipWork from home- ...Principal ASIC Architect Sunnyvale, CA About our company - Tensordyne (formerly Recogni) AI is reshaping our world, performing cognitive tasks once unique to humans—perceiving across modalities, learning quickly, and solving complex problems. Tensordyne builds...PrincipalRemote work
- A pioneering tech company based in the US is seeking a Principal Packaging Engineer to drive the development of next-generation packaging technologies for photonic systems. You will lead the creation of innovative packaging solutions and conduct advanced modelling. The...Principal
- Qualcomm is hiring a Principal Robotics SLAM and Positioning Lead in Santa Clara to own the architecture of their SLAM and positioning services. This role focuses on developing core services around localization and mapping while ensuring quality and performance in autonomous...Principal
$147k - $237.5k
A leading cybersecurity firm in Santa Clara is seeking a Principal SQA Engineer to join their SASE SD-WAN team. The role focuses on validating technologies and contributing to product quality. Candidates should have 8+ years in software quality assurance, strong skills...Principal$118.2k - $185k
Vistance Networks, Inc. is looking for a Principal Engineer - WiFi in Sunnyvale, California. You'll design and develop cutting-edge WiFi features, optimize system performance, and collaborate with cross-functional teams. Ideal candidates should have a Bachelor’s or Master...Principal
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Principal AI Architect. Be the first to apply!
Related searches
- principal San Jose, CA
- senior principal cloud computing engineer San Jose, CA
- principal architect San Jose, CA
- principal data scientist San Jose, CA
- principal cloud computing engineer San Jose, CA
- senior principal scientist San Jose, CA
- principal
- principal game designer
- principal network administrator
- principal advisor


