Data Engineer

Open to Tysons Corner, VA or Rockville, MD locations The Data Engineer works with moderate supervision across two equally weighted domains: (1) large-scale data pipeline development processing market events in a cloud environment, and (2) design and development of agentic AI systems including LLM-powered regulatory data assistants, MCP servers, and agent harness architectures. This position contributes to overall product quality throughout the software development lifecycle. Qualifications Experience building data pipelines using Apache Spark (PySpark preferred) and SQL Experience with SQL query engines (Hive, Trino/Presto, or similar) and cloud data platforms (AWS S3, EMR, Lambda) Practical experience building LLM-powered agent systems Hands‑on experience with at least one agent framework: LangChain, LangGraph, AWS Strands, or equivalent Proficiency in Python for data engineering and automation Responsibilities Build and maintain ETL/ELT pipelines using Apache Spark, Hive, and Trino across S3‑based data lake environments. Develop and optimize SQL for large‑scale datasets, including window functions, multi‑table joins, and complex aggregations. Build and engineer big data systems (EMR‑on‑EC2, EMR‑on‑EKS) and develop solutions on analytical platforms (SageMaker, Domino, Dataiku). Participate in data quality monitoring, anomaly detection, and production incident investigation. Develop AI agent systems using AWS Bedrock and agent frameworks (Strands Agents SDK, LangChain/LangGraph, or equivalent). Build agent‑harness architectures combining LLM reasoning with deterministic execution, including skill‑ and RAG‑based SQL generation and structured output validation. Implement agent memory, context management, and tool integration (MCP servers, API connectors, data catalog lookups) across data platforms. Build evaluation frameworks for agent accuracy, including paraphrase robustness, routing precision, and structural consistency. Stay informed of advances in LLM frameworks (LangGraph, Google ADK, AWS Strands) and emerging AI capabilities. Write clean, well‑tested code and contribute to CI/CD pipelines and infrastructure‑as‑code on AWS. Ensure secure handling of sensitive and regulated data across both data pipelines and AI agent outputs, including auditable execution traces. Adhere to organizational standards for secure software development practices, governance, and technology policies. Partner across teams, communicate technical information effectively to both technical and non‑technical stakeholders, and maintain clear documentation. Actively learn from senior team members and contribute to continuous improvement initiatives, fostering collaboration, innovation, accountability, and technical excellence. #J-18808-Ljbffr Optomi