Senior AI Engineer

Job Description

Position Overview: We are seeking a Senior AI Engineer to build and support production AI solutions involving document intelligence, LLMs, retrieval, evaluation, and agentic AI. This is a hands-on engineering role for someone who has built, operated, measured, and improved AI systems in production, not someone whose experience is primarily research, notebooks, or proof-of-concepts.

The client operates a live document-intelligence platform: multimodal large language models extract structured records from high volumes of unstructured documents, hybrid retrieval reconciles those records against large reference catalogs, and human reviewers adjudicate low-confidence output through internal applications.

We are seeking one senior individual contributor to own the applied-AI and data-science layer end to end — extraction modeling, retrieval quality, and the evaluation and experimentation infrastructure that governs both.

Key Responsibilities: • Build and operate production AI solutions for document extraction and information retrieval. • Develop reliable Python services supporting LLM and AI applications. • Build evaluation and experimentation frameworks to measure AI quality and guide production decisions. • Perform structured dataset analysis, error analysis, and model comparison. • Develop retrieval capabilities using dense/sparse retrieval, vector search, ranking, and reranking. • Support agentic AI and AI orchestration initiatives. • Build reliable event-driven services and safely manage application/schema changes. • Ensure sensitive and regulated data is appropriately protected across AI systems.

We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here.

Skills and Requirements

• Expert Python: 5+ years of production Python experience, including services the candidate personally operated; experience with async development, typing, Pydantic, packaging, and profiling. • Production AI: Hands-on experience with both production document extraction and retrieval at scale. • AI Evaluation: Personally built an evaluation framework/harness for a non-deterministic system and used the results to make a meaningful production decision. • Measurement & Statistics: Strong understanding of hypothesis testing, confidence intervals, sample sizing, precision/recall/F1, accuracy, variance, and model calibration. • Dataset & Error Analysis: Experience with sampling, annotation, dataset versioning, label quality/agreement, and identifying performance issues hidden within specific data populations. • Production Engineering: Experience with event-driven architectures, idempotency, deduplication, replay/backfill, schema/API evolution, and safe rollout/rollback practices. • Regulated Data: Experience appropriately handling sensitive/personal data, including classification, minimization, redaction, retention, least-privilege access, and secrets management. • Agentic AI / AI orchestration experience in production. • Experience managing LLM cost, token usage, and caching. • Online experimentation such as A/B testing, shadow deployments, or similar comparison approaches. • Drift detection and traditional ML experience. • Fine-tuning, adapter training, distillation, or systematic prompt optimization. • Large-scale analytical/data experience. • Experience with FERPA, HIPAA, GDPR, PCI-DSS, or SOX environments. • GCP and Google ADK experience would be a strong plus. • ML experience is beneficial. • Higher education experience is a plus, but not required.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to View email address on click.appcast.io.