AI/ML Software Engineer

AI/ML Software Engineer

Location: Administrative Office of the Courts (AOC) - Annapolis, Maryland

Role: The AI/ML Software Engineer will build software tools that incorporate artificial intelligence and machine learning techniques to automate narrowly defined tasks with high accuracy, assist internal users with their job functions, and improve the experience external users have when interacting with user-facing systems. The work includes developing AI-enabled solutions for automation, knowledge retrieval, document processing, and data analysis within established technical constraints.

Duties: The AI/ML Software Engineer will:

System Design & Collaboration

• Design and build software systems that integrate AI/ML techniques to automate tasks, assist internal users, and improve user-facing services
• Work within established constraints related to infrastructure, programming languages, and model selection
• Contribute to technical decision-making related to data processing, retrieval strategies, and system integration
• Collaborate with team members to define agent architectures, workflows, and system design decisions
• Evaluate and select appropriate approaches for assigned tasks, including determining when to use LLM-based versus non-LLM techniques

Testing, Evaluation & Quality Assurance

• Assist in the design and implementation of testing and evaluation pipelines for AI/ML systems
• Develop unit and integration tests for AI-enabled workflows and data pipelines
• Generate and use synthetic data to support evaluation and benchmarking efforts
• Contribute to improving system performance, including accuracy, latency, and cost efficiency

Deployment & Operations

• Support deployment of AI/ML applications within a hybrid cloud environment
• Work with containerized applications to ensure reliable deployment and updates
• Optimize systems for environments with limited computational resources, including minimal GPU availability

General Responsibilities

• Deliver production-grade systems aligned with defined requirements while supporting iterative improvement of evolving tools
• Document system designs, workflows, and technical decisions as required
• Stay informed on relevant advancements in AI/ML and apply them where appropriate within project constraints

Education:

• Bachelor of Science in Engineering, Computer Science, Data Science, Mathematics, or a related technical field is required

Required Experience and Skills:

• At least 3 years' experience in data science, machine learning, or applied AI development
• At least 3 years' experience in software engineering, architecture, or web development
• Experience building AI/ML-enabled software systems, including LLM-based and non-LLM approaches
• Experience with Python for developing backend services, APIs, middleware, and data pipelines
• Experience with SQL and relational databases (e.g., PostgreSQL)

Preferred Skills:

• Experience working with large language models, including API-based integration and local deployment
• Experience with RAG systems, including embeddings, vector similarity, reranking, and graph-based retrieval
• Familiarity with testing and evaluation pipelines for AI systems, including the use of synthetic data
• Experience validating AI-generated outputs and integrating AI tools into production systems
• Experience with Git, Docker, containerization, and service-oriented architectures
• Knowledge of hybrid cloud environments and distributed systems
• Experience designing multi-agent or task-oriented AI systems
• Familiarity with graph databases, asynchronous processing, and backend queues
• Knowledge of classical NLP techniques alongside LLM-based approaches