AI/ML Software Engineer

Job Title: AI/ML Software Engineer
Location: Remote (On-site support may be required)
Duration: 05 years

Job Summary:
The AI/ML Software Engineer will build software tools that incorporate AI/ML techniques to automate narrowly defined tasks with high accuracy, assist internal users with their job functions, and improve the experience external users have when interacting with the Maryland Judiciary. This includes, but is not limited to, RPA work, building or refining chatbots, incorporating AI/ML into reporting tools, building llm agents for knowledge retrieval, deep research, translation, transcription, redaction, document analysis, document generation, agentic coding, and data processing.



1. System Design & Collaboration:
a. Work within established constraints regarding infrastructure, programming languages, and model selection
b. Contribute to technical decision-making related to data processing, retrieval strategies, and system integration
c. Collaborate with team members to define agent architectures, workflows, and system design decisions
d. Evaluate and select appropriate approaches for given tasks, including determining when to use LLM-based versus non-LLM techniques
e. Designing and building software systems that integrate AI/ML techniques to automate tasks, assist internal users, and improve user-facing services.

2. Testing, Evaluation, and Quality Assurance
a. Assist in the design and implementation of testing and evaluation pipelines for AI/ML systems
b. Develop unit and integration tests for AI-enabled workflows and data pipelines
c. Generate and utilize synthetic data to support evaluation and benchmarking efforts
d. Contribute to improving system performance, including accuracy, latency, and cost efficiency

3. Deployment & Operations:
a. Support deployment of AI/ML applications within a hybrid cloud environment
b. Work with containerized applications to ensure reliable deployment and updates.
c. Optimize systems for environments with limited computational resources, including minimal GPU availability

4. General Responsibilities:
a. Deliver production-grade systems aligned with defined requirements, while supporting iterative improvement of evolving tools
b. Document system designs, workflows, and technical decisions as required
c. Stay informed on relevant advancements in AI/ML and apply them where appropriate within project constraints
5. In addition, the resource will complete the deliverables listed below by Purchase Order year. The estimated level of effort for each deliverable may vary based on its complexity and may be adjusted as needed, including extension beyond the applicable Purchase Order year
a. Year 1
(1) Internal Chatbot Refinement: UI improvements, user history & feedback - 240 hours.


i Deliverables: Application code + Docker build; user profile & history DB; test cases & privacy/compliance pipeline. "UI improvements, enable user history and feedback..."


(2) External Chatbot Development: Initial conversational bot (non-analytical) - 480 hours.


i Deliverables: Application code + Docker build; conversation DB; test cases & compliance pipeline; UX/agency scoring. "Goal is conversational but not analytical. Must point users to resources..."


(3) RPA: Local LLM analysis tools with batching - 240 hours.


i Deliverables: Application code; integration documentation; usage & process reporting.


(4) Knowledge Retrieval (RAG & Search): Improve vector/hybrid search & case mgmt integration - 520 hours.


i Deliverables: Comparative RAG results; agent code/prompts; test pipeline; recommendations for knowledge store updates.


(5) Translation: MD-specific terminology & guidelines - 80 hours.


i Deliverables: Translation agent code/prompts; test cases & pipeline.


(6) Transcription: Refine deployment based on feedback - 160 hours.


i Deliverables: Comparative pipeline results; updated code/prompts; test cases & pipeline.


(7) Redaction (PII & Sensitive Data): Build detection agent - 240 hours.


i Deliverables: Application code; test cases & pipeline for PII/sensitive data identification.


b. Year 2
(1) Internal Chatbot Refinement: Personalization & workflow integration - 160 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(2) External Chatbot Development: Improvements from feedback - 160 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(3) RPA: Reporting pipelines & analytics; automate tasks - 160 hours.


i Deliverables: Application code; integration docs; usage & process reporting.


(4) Knowledge Retrieval: Expand case mgmt integration & permission-based indexing - 240 hours.


i Deliverables: Comparative RAG results; agent code/prompts; test pipeline; knowledge store recommendations.


(5) Deep Research Capabilities: graphRAG for case-statute interaction - 520 hours.


i Deliverables: Web/file crawler to graph format; agent code/prompts; test pipeline; knowledge store recommendations.


(6) Translation: Quality & accuracy improvements - 80 hours.


i Deliverables: Translation agent code/prompts; test cases & pipeline.


(7) Transcription: Improve diarization & speaker ID - 160 hours.


i Deliverables: Transcription agent code/prompts; test cases & pipeline.


(8) Redaction: Accuracy improvements & workflow integration - 160 hours.


i Deliverables: Application code; test cases & pipeline.


(9) Document Analysis: NLP + graphRAG to reduce token use - 320 hours.


i Deliverables: Scripts for universal intermediary format; extraction scripts to graph store.


c. Year 3
(1) Internal Chatbot Refinement: Improvements from feedback - 160 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(2) External Chatbot Development: County-specific deployments & scalability - 200 hours.


i Deliverables: County data ingestion; agent/bot revisions; onboarding; county-specific test pipeline.


(3) RPA: Case update review & flagging - 320 hours.


i Deliverables: Application code; integration docs; usage & process reporting.


(4) Knowledge Retrieval: Centralize statutes/ordinances for counties - 320 hours.


i Deliverables: Document data sources; standardization recommendations; supporting code/database.


(5) Deep Research Capabilities: Combine retrieval + case mgmt for complex research - 320 hours.


i Deliverables: Agent code/prompts; test pipeline; knowledge store recommendations.


(6) Translation: Expand language support - 80 hours.


i Deliverables: Translation agent code/prompts; test cases & pipeline.


(7) Transcription: Expand language support - 80 hours.


i Deliverables: Transcription agent code/prompts; test cases & pipeline.


(8) Document Analysis: Extract structured data from forms - 160 hours.


i Deliverables: Extraction scripts; standardized process proposal; supporting tools.


(9) Document Generation: Initial PDF generation features - 320 hours.


i Deliverables: PDF generation scripts; agent code/prompts; test pipeline.


d. Year 4
(1) Internal Chatbot Refinement: Low-code custom agent builder - 320 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(2) External Chatbot Development: Workflow-integrated chatbot - 320 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(3) RPA: Expand AI-enhanced reporting & automation - 200 hours.


i Deliverables: Application code; integration docs; usage & process reporting.


(4) Knowledge Retrieval: Finetune embeddings & small LLM - 320 hours.


i Deliverables: Finetuning scripts; model weights.


(5) Deep Research Capabilities: Limited web access for research - 240 hours.


i Deliverables: Agent code/prompts; test pipeline; knowledge store recommendations.


(6) Transcription: Finetune model for opt-in user voices - 240 hours.


i Deliverables: Voice finetuning scripts; model weights.


(7) Document Generation: Expand document types & form-filling - 320 hours.


i Deliverables: Document/presentation generation scripts; agent code/prompts; test pipeline.

e. Year 5
(1) Internal Chatbot Refinement: Improvements from feedback - 160 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(2) External Chatbot Development: Feature expansion - 160 hours.


i Deliverables: Application code + Docker build; test cases & compliance pipeline.


(3) RPA: Integrate retrieval & research into workflows (with manual review) - 480 hours.


i Deliverables: Application code; integration docs; usage & process reporting.


(4) Knowledge Retrieval: Open some retrieval capabilities to public bot - 320 hours.


i Deliverables: Application code; integration docs; usage & process reporting.


(5) Transcription: Integrate into court recording systems - 320 hours.


i Deliverables: Application code; integration docs; usage & process reporting.


(6) Document Generation: Modify user documents using RPA/retrieval/research - 520 hours.


i Deliverables: Document-modification agent code/prompts; test pipeline.



Required Education:
Bachelor of Science in Engineering, Computer Science, Data Science, or Mathematics, or a related field (as determined by the Client).


Required Skills:
At least three (3) years' experience in data science, machine learning, or applied AI development.
At least three (3) years' experience in software engineering, architecture, or web development.
Experience with:
(1) SQL and relational database systems (e.g., PostgreSQL)
(2) Fine-tuning small language models or embedding models
(3) Contributing to or maintaining open-source software projects
(4) Graph databases or graph extensions (e.g., Neo4j, Apache AGE)
(5) Designing and implementing multi-agent or task-oriented AI systems
(6) Embedding models, vector similarity, re-ranking, and graph retrieval techniques in RAG systems
(7) Version control systems (e.g., Git), containerization technologies (e.g., Docker), and service-oriented architectures
(8) Collaborating with large language models (LLMs), including both API-based integration and local deployment
(9) Validating AI-generated outputs, mitigating hallucinations, and integrating AI tools into production service pipelines



Ability to:
(1) Understand data structures, algorithms, and clean coding principles
(2) Select and apply appropriate techniques (LLM and non-LLM) based on task requirements
(3) Develop and improve testing and evaluation pipelines for AI systems, including use of synthetic data
(4) Demonstrate proficiency in Python, including the ability to develop production-grade backend services, APIs, middleware, and data pipelines.
(5) Design and implement AI/ML systems that operate effectively on complex, inconsistent, or evolving datasets while balancing accuracy, latency, and cost (token consumption)
(6) Collaborate with team members to define system architecture, agent workflows, and data pipelines while working in constrained environments, including limited GPU availability and predefined infrastructure



Knowledge of:
(1) Hybrid cloud environments and distributed system considerations
(2) Threading, asynchronous processing, and queues in backend servers
(3) React and Microsoft Teams Toolkit for developing chatbot user interfaces
(4) Non-llm data analysis techniques for structured, semi-structured, and unstructured data
(5) Classical natural language processing (NLP) techniques in addition to LLM-based approaches
(6) Data science and LLM-related libraries in Rust or other performance-oriented programming languages