Agentic AI & Observability Lead

Job Title: Agentic AI & Observability Lead
Job Location: Mclean, VA/Plano TX
Job Type: Contract

Job Description:

• Design, develop, and deploy AI/Agentic AI applications and autonomous agents that interact with enterprise systems, APIs, data platforms, and operational tools.
• Design and build MCP Servers to securely expose enterprise tools, APIs, data sources, and operational capabilities to AI agents.
• Develop reusable MCP tools, resources, and integrations , including authentication, authorization, error handling, logging, and monitoring.
• Build backend services, APIs, integrations, and automation using Python and/or JavaScript .
• Implement application and platform observability using the OpenTelemetry framework , including logs, metrics, and traces.
• Implement and troubleshoot distributed tracing across microservices and distributed application environments.
• Build advanced log analytics, dashboards, visualizations, and operational insights using Kibana .
• Analyze telemetry and operational data to identify anomalies, application failures, performance bottlenecks, and recurring operational patterns.
• Design and integrate real-time and near-real-time data streams for telemetry, analytics, and AI-agent use cases.
• Develop and deploy solutions in Kubernetes-based containerized environments .
• Design and integrate solutions with AWS services and cloud-native architectures .
• Build and maintain CI/CD pipelines for automated build, testing, security validation, and deployment.
• Automate application, infrastructure, and service provisioning using Python or JavaScript .
• Apply enterprise security, compliance, identity, access-control, secrets-management, and data-protection requirements across agent and observability solutions.
• Work with application development, SRE, DevOps, platform, security, and architecture teams to onboard applications and implement observability and AI-driven operational capabilities.
• Develop production-quality solutions with appropriate scalability, resiliency, performance, security, and monitoring .

Required Technical Skills

Agentic AI & Development

• Strong experience developing AI agents / Agentic AI solutions .
• Hands-on experience designing and building MCP Servers and integrating MCP clients, tools, resources, APIs, and enterprise data sources.
• Strong programming experience in Python and/or JavaScript .
• Experience developing REST APIs, backend services, integrations, and automation frameworks.

Observability & Analytics

• Strong hands-on experience with OpenTelemetry (OTel) .
• Experience implementing and analyzing distributed traces, logs, and metrics .
• Strong experience with Kibana dashboards and log analytics .
• Ability to correlate telemetry across distributed systems for troubleshooting and root-cause analysis.
• Experience working with high-volume telemetry and data streams .

Cloud & Platform Engineering

• Strong experience with Kubernetes , containers, and microservices architectures.
• Hands-on experience with AWS and cloud-native services.
• Experience developing and maintaining CI/CD pipelines .
• Experience with infrastructure/application provisioning and automation using Python or JavaScript .

Security & Compliance

• Understanding of secure application and API development.
• Experience implementing authentication, authorization, RBAC, secrets management, encryption, and secure communication .
• Ability to develop solutions aligned with enterprise security, governance, audit, and compliance requirements .

Preferred Qualifications

• Experience integrating AI agents with observability, incident management, DevOps, or enterprise operational platforms .
• Experience building agent-based solutions for incident triage, root-cause analysis, log analysis, remediation, and operational automation .
• Knowledge of SRE, AIOps, SLIs/SLOs, reliability engineering, and automated remediation .
• Experience working with large-scale distributed and microservices-based enterprise applications.
• Understanding of secure AI-agent architectures, including tool permissions, guardrails, identity propagation, auditability, and controlled access to enterprise systems .