AI software engineer

AI Software Engineer

The Fitch Group Emerging Technology AI group is seeking an AI Software Engineer to be part of a team that will be dedicated to build and support Generative AI, Machine learning, Deep Learning and Data science solutions across the organization. The position could be based out of our Chicago or NY offices. Fitch is building a unified platform that supports both internal Centers of Excellence (CoE) Python-based agent frameworks for complex, pro-code use cases and no-code/low-code solutions such as Microsoft Copilot Studio and UiPath. This platform will centralize governance, evaluation tooling, and LLMOps capabilities so teams across Fitch can confidently design, deploy, and operate AI agents. We're seeking a contractor AI Software Engineer with deep Python and cloud computing experience to design, implement, and operationalize this platform end-to-end.

What We Offer:

• This will be a high impact role with significant visibility where the candidate will work on some flagship Fitch products
• The candidate will have an excellent opportunity to work in the cutting-edge field of Generative AI and LLMOps
• Fitch promotes an excellent work culture and is known for providing a good work life balance

We'll Count on You To:

• Platform architecture and development
• Design and build a modular platform that integrates Python-based agent frameworks (e.g., LangChain, Semantic Kernel, custom orchestration) and no-code/low-code tools (e.g., MS Copilot Studio).
• Implement core services: agent orchestration, prompt/policy management, connector/tool registry, configuration store, and secure secrets handling.
• Develop APIs/SDKs/adapters to support multiple model providers and runtimes; enable multi-tenant use by various teams.
• LLMOps, evaluation, and observability
• Stand up experiment tracking and evaluation pipelines (offline/online, human-in-the-loop, rubric/golden set scoring).
• Build telemetry for prompt usage, latency, cost, safety incidents, grounding/accuracy metrics, and drift detection.
• Enable A/B testing, canary releases, and rollback strategies for agents and prompts.
• Governance, security, and compliance
• Embed policy enforcement for PII handling, data residency, model/provider allowlists, and approval workflows.
• Implement role-based access control (RBAC), audit logging, and change management for agent lifecycle.
• Support model risk management artifacts (model cards, decision logs, evaluation reports).
• Reliability, CI/CD, and infrastructure
• Develop secure-by-design services: least-privilege access, secrets rotation, content filtering, isolation of external tools.
• Build robust CI/CD, automated tests, IaC, and environment promotion; ensure high availability and resilience.
• Integrations and data grounding
• Create connectors to enterprise systems (APIs, data lakes, knowledge bases) and retrieval pipelines (RAG) with vector search.
• Integrate with Microsoft ecosystem (Copilot Studio) while leveraging AWS-first infrastructure and services.
• Collaboration and enablement
• Partner with CoEs, security, compliance, and product teams to translate requirements into platform features.
• Produce documentation, usage guides, templates, and reference implementations for pro-code and no-code users.
• Support onboarding, troubleshoot issues, and drive adoption across Fitch teams.

What You Need to Have:

• 5+ years of software engineering experience building production-grade platforms or distributed systems
• Expert-level Python skills, including building services, SDKs, and integrations
• Strong AWS experience container orchestration (EKS/Fargate), serverless (Lambda), API Gateway, Step Functions, S3, DynamoDB/RDS, Secrets Manager/KMS, CloudWatch/CloudTrail, IAM
• Hands-on with LLM/agent frameworks and providers (e.g., LangChain, Semantic Kernel, OpenAI/AWS Bedrock/Anthropic SDKs) and prompt/agent lifecycle management
• Proven LLMOps experience: experiment tracking, offline/online evaluation, telemetry, cost/safety monitoring, and A/B testing
• Security and governance expertise: RBAC/ABAC, audit logging, policy enforcement, PII controls, approvals workflows
• Data retrieval/RAG proficiency: vector databases (e.g., OpenSearch vector, pgvector, Pinecone), embedding pipelines, grounding strategies
• CI/CD and IaC: GitHub Actions/AWS CodePipeline, Terraform/CloudFormation; testing frameworks and secure supply chain practices
• Strong communication skills; ability to collaborate with cross-functional stakeholders

What Would Make You Stand Out:

• Experience integrating Microsoft Copilot Studio, or Power Platform into enterprise ecosystems
• Familiarity with evaluation methods for LLMs (assertion tests, rubric scoring, golden datasets, red-teaming, human review loops)
• Observability stack experience (OpenTelemetry, Prometheus/Grafana) and cost/usage analytics
• Background in model risk management or regulated environments
• Knowledge of multi-tenant architecture and guardrails for internal platforms