Senior AI Software Engineer & Developer

All Jobs > Senior AI Software Engineer & Developer

Fully Remote Remote - Washington DC Metro Area (DMV), DC

Full-time

Description

NOTE: This opportunity is full-time employment position only (no 1099 or C2C engagements, or third parties or staffing agencies, please). The candidate MUST be a U.S. Citizen and Permanent Resident (Green Card holder). This is a remote opportunity; candidate must be based in the U.S.; have resided in the U.S. for at least 3 years in the past 5 years; ET time zone work schedule.

Daily Responsibilities

• Serve as a senior, hands-on full-stack AI engineer and technical authority, leading the technical strategy, design, and delivery of large-scale mission-critical AI systems supporting federal programs.
• Serve as the primary technical authority, defining AI and application architecture across multiple programs.
• Establish enterprise modernization roadmaps aligned to mission outcomes, compliance, and scalability.
• Lead architecture for distributed, cloud-native, and hybrid AI systems.
• Define and enforce reference architectures, standards, and reusable frameworks.
• Drive cross-program technical decision-making to ensure interoperability, security, and long-term sustainability.
• Advise senior federal stakeholders (SES-level and above) on AI adoption, modernization, and risk management.
• Lead design, development, and deployment of advanced AI solutions using Python as the primary development language, including large language models (LLMs) and foundation models, Retrieval-Augmented Generation (RAG) systems, agentic workflows, and orchestration frameworks.
• Architect and implement scalable ML systems and services built on Python-based frameworks and APIs.
• Build full-stack AI applications end to end, from user-facing interfaces to back-end services, APIs, and data layers.
• Integrate AI and LLM capabilities into existing enterprise applications and legacy platforms (e.g., content management, case management, and records systems) via APIs, middleware, and event-driven patterns.
• Define and implement distributed training strategies (GPU/TPU clusters, parallelization, optimization).
• Oversee full ML lifecycle in partnership with the senior data scientist: data pipelines, feature engineering, training, evaluation, deployment, and monitoring.
• Drive model optimization techniques (quantization, distillation, caching) to improve performance and cost.
• Establish robust MLOps practices leveraging Python-driven automation, pipelines, and tooling.
• Stand up the enterprise CI/CD-to-AI/MLOps pipeline, beginning with time-boxed proofs of concept and MVP implementations that mature into production systems.
• Serve as SME in federal AI policy (e.g., NIST AI RMF, OMB M-25-21 and M-25-22, Executive Order 14179).
• Define and operationalize Responsible AI frameworks, including model validation and evaluation, bias mitigation and fairness, and explainability, auditability, and safety.
• Ensure compliance with FISMA, FedRAMP, NIST 800-53, privacy, and Section 508 requirements.
• Lead large-scale modernization initiatives (e.g., legacy-to-cloud, microservices transformation, including Python-based refactoring and re-platforming efforts).
• Define repeatable modernization frameworks and accelerators.
• Oversee DevSecOps pipelines, CI/CD automation, zero-trust architectures, and secure software supply chain practices.
• Ensure delivery of resilient, high-availability systems in regulated federal environments.
• Lead multiple concurrent engineering efforts across integrated teams.
• Provide technical leadership to architects, engineers, and DevSecOps specialists, including establishing Python coding standards and engineering best practices.
• Mentor senior engineers and technical leaders; elevate engineering excellence and code quality.
• Support technical strategy in proposals, captures, and client engagements.
• Contribute to thought leadership (whitepapers, architecture patterns, platform strategy).
• Executive communication skills with experience influencing senior leaders.

Requirements

• Must of a U.S. Citizen or Permanent Resident (Green Card holder), as mandated by our government client.
• Must be able to complete/pass/hold at a minimum a Public Trust Investigation / background check. An active clearance (e.g. Public Trust, Secret, or higher) is preferred.
• Must be based / reside in the U.S.
• 12+ years of software engineering experience combining senior technical leadership, hands-on expertise in Python-based AI/ML systems (including large language models), and ownership of enterprise architecture, governance, and innovation.
• 8+ years of applied AI/ML experience, including building and deploying production systems (LLMs, generative AI, and large-scale or distributed model systems).
• Expert-level Python development experience, including designing production-grade ML systems, data pipelines, and microservices-based architectures.
• Hands-on experience with ML frameworks (PyTorch, TensorFlow, JAX) and distributed training (DeepSpeed, FSDP, Horovod).
• Full-stack engineering skills, including front-end frameworks, back-end services, RESTful APIs, microservices, and cloud-native deployment (e.g., containers, Kubernetes).
• Deep experience with cloud platforms (Azure, AWS, GCP), including FedRAMP environments and designing AI systems in cloud-native, distributed environments.
• Deep expertise in machine learning and deep learning, particularly transformer-based models and LLMs.
• Hands-on experience with LLM application stacks, including orchestration frameworks (e.g., LangChain, LlamaIndex, Semantic Kernel), embeddings, vector databases, and prompt engineering.
• Experience with AI platforms and architectures (e.g., AWS Bedrock, Azure OpenAI Service, Google Vertex AI, RAG, agents).
• Proven success delivering enterprise-scale systems and modernization programs.
• Strong background in microservices, APIs, distributed systems, and DevSecOps practices.
• Experience managing GPU-based infrastructure or high-performance ML environments.
• Demonstrated ability to translate AI research into production system.
• Proven ability to integrate AI capabilities into existing and legacy enterprise systems (e.g., legacy CMS or COTS platforms) using APIs, middleware, connectors, and event-driven architectures.
• Strong understanding of large-scale data systems and ML evaluation methodologies.
• Experience working with sensitive data, including PII safeguards such as anonymization, masking, and data loss prevention.
• Experience with enterprise integration technologies, including REST/SOAP services, message queues, ETL pipelines, and SQL/NoSQL databases.
• Proficiency with managed generative AI services (e.g., AWS Bedrock, Azure OpenAI Service, Google Vertex AI) and integrating frontier models such as GPT, Claude, and Gemini.
• Demonstrated ability to own solutions end to end — from discovery and prototyping through production deployment, integration, and ongoing support.
• Ability to balance strategic vision with deep hands-on technical execution.
• Excellent analytical skills, attention to detail, and strong problem-solving abilities.
• Excellent communication and collaboration skills to communicate complex analytical insights to executive and non-technical stakeholders. Ability to translate ambiguous business questions into analytical solutions.
• BS or MS degree (preferred) in engineering, data science, computer science, statistics or related field.

The following experience is PREFERRED

• Experience with federal civilian agencies preferred.

Compensation decisions depend on a wide range of factors, including but not limited to skill sets, experience and training, security clearances, licensure and certifications, and other business and organizational needs. $155k -$189k.