Principal Engineer, AI & Agentic Systems
BICP
BICP, a San Diego-based consulting firm specializing in AI, Data & Analytics, is seeking an experienced Principal Engineer, AI & Agentic Systems to join our team in an embedded capacity with one of our strategic clients in the apparel and retail industry. We will consider remote candidates with the right blend of experience however candidate must be willing to be onsite 1 week a month initially. This is a hands-on and technical leadership role for someone who can establish platform direction, write production code, and turn business needs into reliable AI applications.
You’ll work with engineering and business leaders to identify valuable opportunities, choose the right approach, and deliver solutions that teams can use and trust. The scope includes AI assistants, agent-driven workflows, enterprise knowledge retrieval, and reusable services that support future development.
*** PLEASE NOTE: We are open to remote candidates however must be available to be onsite once a month for the first several months.
What You’ll Do
- Own key architectural decisions for the AI platform, including technology selection, build-versus-buy recommendations, and reusable engineering patterns.
- Design and ship AI applications and agentic workflows using LLMs, tool calling, planning, memory, and human review where appropriate.
- Connect AI solutions to enterprise APIs, databases, SaaS applications, and business systems.
- Develop orchestration and retrieval capabilities that support multiple applications and use cases.
- Establish evaluation, testing, monitoring, and observability practices to measure accuracy, safety, reliability, and business results.
- Incorporate security, access controls, governance, and operational guardrails into solution design.
- Move promising prototypes into production and improve them through usage data and stakeholder feedback.
- Help engineers strengthen their capabilities through technical reviews, pairing, and practical tooling.
- Assess whether a problem calls for an agentic approach, conventional automation, or another solution.
What You Bring
- 10+ years of software engineering experience, including at least 2 years building and operating AI systems in production.
- Hands-on experience deploying agentic systems that use planning, reasoning, memory, and tools to complete business workflows.
- Strong Python skills and experience developing APIs, backend services, and cloud-native applications.
- Experience creating AI evaluation frameworks and production monitoring that assess quality, reliability, safety, and operational performance.
- A strong understanding of LLM platforms and orchestration frameworks, with the ability to evaluate tradeoffs and select technologies based on the problem.
- Experience integrating applications with enterprise data and systems.
- The ability to clarify ambiguous requirements, explain technical decisions, and collaborate effectively with business stakeholders.
- A bachelor’s degree in computer science, engineering, AI, data science, or a related field, or equivalent practical experience.
Additional Experience We Value
- RAG, embeddings, vector databases, and enterprise knowledge retrieval.
- Enterprise AI platforms such as Azure AI Foundry, Google Gemini Enterprise, Microsoft Agent 365, or Copilot Studio.
- Azure, AWS, or Google Cloud.
- AI governance, security, compliance, and observability.
- MLOps, model lifecycle management, and deployment pipelines.
- Retail, apparel, ecommerce, direct-to-consumer, or other consumer businesses.
About BICP
BICP is a San Diego-based AI, Data & Analytics consultancy that helps organizations close the gap between strategic priorities and the specialized capabilities required to execute them. Our Forward Deployed practitioners work alongside client teams, bringing deep expertise across AI/ML, Data, Analytics, Product, and Architecture to move initiatives from idea to implementation, production, and measurable business impact.
$197.5k - $264.39k
...are proud to be an outlet for opportunity and for personal growth and success. Job Description As a Principal AI Engineer specializing in Agentic Systems, you will own designing, building and scaling AI-powered applications, agentic systems and intelligent tooling...SuggestedCasual work- ...breakthrough optical interconnect technology for data centers, AI, and high-performance computing. Their proprietary... .... Role Summary We are looking for a Senior or Principal Systems Integration Engineer to own system-level bring-up, calibration,...Suggested
- ...interconnect technology for data centers, AI, and high-performance computing.... ...Summary We are looking for a Senior or Principal Embedded Firmware Engineer to develop the control firmware and... ...platforms and evaluation systems. This is a hands-on role focused on...Suggested
$204k - $284k
...Principal Radar Systems Engineer Dayton, OH STR is focused on delivering National Defense capabilities by driving mission-focused strategies to develop advanced technology systems enabling enduring products and solutions focused on achieving the customer vision....SuggestedFull timeWork experience placementLocal areaNight shift$155k - $193k
ResponsibilitiesMaxLinear is seeking a Principal Systems Engineer to join our team. In this role, you will focus on the following:Leading cross-functional... ..., hardware, and systems teams supporting data center and AI infrastructureArchitect and analyze optical systems,...SuggestedWork experience placementLive in- ...Principal Information Security Engineer Callaway Golf Company is a premium golf equipment, gear and apparel... ...general network/DevSec Ops practices. AI & LLM Threat Modeling: identify... ...Understanding how LLM's. vector databases, and agentic frameworks function. DE&I and...
$127.5k - $236.5k
...solutions connecting the space, air, land, sea and cyber domains in the interest of national security. Job Title: Manager, Systems Engineering Job Code: 46927 Job Location: Carlsbad, CA Job Schedule: 9/80 9/80: Employees work 9 out of every 14 days - totaling...Local areaFlexible hours- ...re looking for a hands-on Infrastructure Engineering Manager to lead Blooio's hardware operations... ..., networking, software development, and AI-powered automation alongside engineering... ...engineering, network engineering, or systems administration ~ Experience leading or...Remote work
- ...Venus Aerospace is revolutionizing rocket engine propulsion. With the first generational... ...future of propulsion. The Role: Principal Propulsion Engineer As a Principal Propulsion... ...Propellant & Pressure Management (PPMS) systems in mass and volume constrained...Full time
$260k - $315k
...manufactured implant. As Head of Software Engineering, you’ll lead the organization behind that... ..., design automation, and manufacturing systems on the aprevo platform. You’ll own delivery... ...leaders and reporting to the Chief AI and Product Officer.ResponsibilitiesLead...For contractors$150k - $205k
ResponsibilitiesMaxLinear is seeking a Principal Analog EDA Engineering Architect to join our team. In this... ...you will focus on the following:Drive AI adoption in analog IC design to... ...is alive and well. We are a fabless system-on-chip product company, striving to...Work experience placementLive in$135k - $198k
ResponsibilitiesMaxLinear is seeking a Principal Engineer Memory Architect to join our team. In this role, you will focus on the following:Design... ...architecture expertise, circuit-level understanding, and system-level awareness to deliver high-performance, power-efficient,...Work experience placementLive in- ...interconnect technology for data centers, AI, and high-performance computing.... ...Summary We are looking for a Senior or Principal Hardware Engineer to own the high-speed electrical... ...work closely with internal photonics, systems, packaging, and test teams to translate...
- ...interconnect technology for data centers, AI, and high-performance computing.... ...Summary We are looking for a Senior or Principal Hardware Engineer to own the mixed-signal, power, and... ..., firmware, photonics, packaging, and systems teams to develop robust electrical control...Local area
$160k - $191k
ResponsibilitiesMaxLinear is seeking a Principal Product Engineer to join our team. In this role, you will focus on the following:Own ICs from design... ...entrepreneurial spirit is alive and well. We are a fabless system-on-chip product company, striving to improve the world’s...Work experience placementLive in$184k - $341.5k
...national security. Job Title: Senior Scientist, Division Chief Engineer, Tactical Data Links (TDL) Job Code: 45421 Job Location:... ...digital communications, RF/waveforms, networking, processing, and systems integration. Coordinate senior technical leaders; establish...Local areaFlexible hours$180.5k - $285k
Principle Communications Engineer One team. Global challenges. Infinite opportunities. At Viasat, we're on a mission to deliver connections... ...to deliver high-quality, ground-breaking communications systems for government customers. These projects entail system engineering...$144k - $227.5k
...governing change management processes to improve system stability Design and optimize scalable... ...Partner cross-functionally with engineering teams, service providers, and technology... ...term success Drive innovation through AI and automation, identifying, evaluating,...All shiftsShift workNight shift- ...Aerospace is revolutionizing rocket engine propulsion. With the first... ...of propulsion. The Role: Principal Propulsion Engineer As a... ...& Pressure Management (PPMS) systems in mass and volume constrained... ...use artificial intelligence (AI) tools to support parts of the...Permanent employmentFull timeRemote workFlexible hours
$105.5k - $196.5k
...interest of national security. Job Title: Manager, Field Engineering Job Code: 44816 Job Location: Carlsbad, CA Job Schedule... ...: Comprehensive knowledge and experience with Link 16 TDL systems and products (MIDS, STT, BATS) Familiarity with ICDs, platform...Local areaFlexible hours- ...cycles counts and rental software management system Qualifications: Bachelor’s... ...management, email and internet search engines Basic Math and standard English grammar... ...: The Above description covers the principal duties and responsibilities of the job....Full timeFor contractorsFor subcontractorWork at officeFlexible hours
$170k - $220k
...Job Description Job Description About the Role: We are seeking a Principal Firmware Engineer to provide technical leadership for the firmware across our clients cardiovascular system. The ideal candidate will have extensive experience architecting and developing...$75k
...about club floors and rooms; communicate with employees, members, and the public Ability to access and operate company computer systems in order to prepare documents, enter data, and read reports from data base or email system. Ability to bend routinely and repetitively...Daily paidFull timePart timeFlexible hours- A brief overviewThe Sr. / Principal Validation Engineer fulfills a critical role in GMP operations, supporting cleaning validation activities for... ...individual must also monitor facility / utility validated systems, plan, write and execute validation/ qualification protocols...
$181k - $242k
...are proud to be an outlet for opportunity and for personal growth and success. Job Description We're looking for a Principal Data Engineer to design, build, and scale the data pipelines that bring data into our platform and make it ready for analytics. You'll work...Casual work$220.9k - $291.5k
...will be responsible for setting up and leading a high-performance engineering design team in the US & India for Cisco’s Client Optics Group (... ...). A significant new focus for this role is the deployment of AI tools, when possible, to enhance productivity, quality, and...Full timeTemporary workWork experience placementLocal areaFlexible hours$126.32k - $173.69k
A brief overviewThe Sr. Engineering Project Manager leads the implementation of large‑scale production processes and equipment at Bachem... ...management methodologies to capital projects. This role oversees system start‑up activities and ensures that engineers and operations...For contractors$120k - $140k
...Interact with, and provide support for Sales EVP, Sales Leadership Team, Direct Sales Representatives and ATEC Sales Agents. Manage data, systems and reporting across SAP, Power BI and SalesForce.com. Assist in the planning, execution and training of new systems and processes....Full timeWork at officeLocal areaVisa sponsorshipWork visa$86.5k - $216k
...the interviewing process. Ensure that new employees are properly onboarded and trained.• Ensure that staff has the proper materials, systems access and training to complete job responsibilities. Provide oversight for the execution of the training plan, SOP review and...Full timePart timeImmediate startWorldwide$146.1k - $206.01k
...Job Description We're looking for a Senior Manager, Data Engineering to lead and grow our data engineering team while staying close... ...architectural decisions across pipeline design, ingestion patterns, and system reliability Represent data engineering in cross-functional...Casual work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Principal Engineer, AI & Agentic Systems. Be the first to apply!
- principal engineer Carlsbad, CA
- data center chief engineer Carlsbad, CA
- chief engineer Carlsbad, CA
- director software engineering Carlsbad, CA
- engineering director Carlsbad, CA
- hotel chief engineer Carlsbad, CA
- general engineer Carlsbad, CA
- principal developer Carlsbad, CA
- healthcare systems engineer Carlsbad, CA
- operating system engineer Carlsbad, CA





