Senior AI Engineer

Senior AI Engineer

Sciene runs a production agentic AI platform: a config-driven engine where every product is an agent with its own identity, skills, tools, and quality gates, executing ReAct loops against real business data. The Senior AI Engineer will design, build, and operate these agentic systems — from prompt and context engineering, through tool and integration design (including MCP), to the evaluation harnesses and deterministic guardrails that keep LLM output trustworthy at scale. You will own features across the full model lifecycle: shipping new AI products, benchmarking models against each other with LLM-as-judge evaluation, hardening outputs with validators and enforcers, and monitoring quality and cost in production over time.

Requirements:

• Strong software engineering fundamentals in Python, including modern async Python — this role builds production services, not notebooks
• Hands-on experience building LLM-powered applications in production: agents / tool use / function calling, prompt engineering, RAG, and structured outputs
• Experience with at least one major LLM provider API (OpenAI, Anthropic, Google) and an understanding of the trade-offs between models and providers
• Experience with FastAPI (or an equivalent modern web framework) and Pydantic
• Understanding of how to evaluate AI systems: offline evals, LLM-as-judge, regression benchmarks, and quality metrics beyond "it looks right"
• Familiarity with cloud platforms (we run on Azure — Container Apps, Key Vault, Container Registry) and containerized deployment with Docker
• Experience with databases in production (we use MongoDB and Databricks SQL warehouses)
• Solid testing habits (pytest or similar) and comfort with CI/CD pipelines
• Excellent problem-solving and analytical skills, and the autonomy expected of a senior engineer: you own a problem end to end — from framing to shipped, monitored outcome — and are accountable for the result, not just the merge
• Bachelor's degree or higher in Computer Science, Artificial Intelligence, or a related field — or equivalent practical experience

Preferred Qualifications:

• Experience with the Model Context Protocol (MCP) or similar agent-integration standards
• Experience with observability stacks: OpenTelemetry, Grafana, Loki, structured logging
• Experience with Databricks beyond SQL (Delta Sharing, jobs, model serving)
• Track record of shipping something from 0 to 1 in a fast-moving environment where priorities shift often
• Experience with classification pipelines and fine-tuning where they beat prompting
• Contributions to open-source AI tooling

What You'll Do:

• Design, develop, and ship agentic AI products on our platform: agent identities, reusable skills, tool integrations, and structured outputs — often with zero code changes thanks to our config-driven architecture, and with platform-level code changes when the engine itself needs to grow
• Do serious prompt and context engineering: system prompt assembly, context injection from live data sources, thread/memory management, and structured output design with Pydantic schemas
• Build and extend agent tools that query Databricks, MongoDB, and external systems, and integrate agents with the broader ecosystem via the Model Context Protocol (MCP) — both exposing our platform as an MCP server and bridging external MCP servers
• Work across multiple LLM providers (OpenAI, Anthropic, Google, and emerging providers) through our in-house gateway with fallback chains and circuit breaking — choosing the right model per task on quality, latency, and cost
• Own output quality: design deterministic enforcers, validation quality gates, and LLM-as-judge evaluators; run model benchmarks and use the results to drive model and prompt decisions
• Operate what you ship: instrument services with OpenTelemetry, monitor them in Grafana, and improve reliability, latency, and cost efficiency in production
• Collaborate with cross-functional teams — data engineers, software engineers, and product — to deliver end-to-end AI solutions
• Stay current with the fast-moving agentic AI landscape (new models, protocols, evaluation techniques) and pragmatically bring the good parts into the platform
• Document and maintain the codebase, ensuring code quality and adherence to best practices

*This is a PJ contract based in Brazil.