Data Engineer

About Us ESimplicity is a modern digital services company that partners with government agencies to improve the lives and protect the well-being of all Americans, from veterans and service members to children, families, and seniors. Our engineers, designers, and strategists cut through complexity to create intuitive products and services that equip federal agencies with solutions to courageously transform today for a better tomorrow. Purpose and Scope eSimplicity is seeking a skilled and motivated Data Engineer to support a federal financial regulatory agency's enterprise data engineering program. In this role, you will design, build, and maintain scalable ETL and data sourcing pipelines within the agency's AWS-based enterprise data platform, making a wide range of financial datasets: structured, semi-structured, and unstructured, sourced from commercial vendors, self‑regulatory organizations, and internal agency filing systems, reliably available for regulatory examinations, enforcement, analytics, and policymaking. This role emphasizes data quality, performance, security, and conformance to enterprise ETL standards, and supports preparing datasets to be AI‑ready for advanced analytics including machine learning and Retrieval Augmented Generation. Successful candidates will be eligible to hold a U.S. Federal Public Trust security clearance (Moderate Risk). This position is contingent upon contract award. Responsibilities Design, develop, ingest, and maintain well‑architected data pipelines that retrieve data from external feeds (APIs, SFTP, FTP, web scraping, Direct Connect) and internal agency sources into the data lake landing zone and downstream curated zones. Develop production‑grade ETL workflows using AWS Glue, PySpark, Python, Lambda, and EMR, integrated with a shared ETL common library and orchestrated via Amazon Managed Workflows for Apache Airflow (MWAA). Load data accurately and optimally into S3 zones (Parquet, ORC, Iceberg), relational datastores (PostgreSQL, Redshift, Oracle), NoSQL databases, and knowledge bases/vector stores, preventing duplicate loads and maintaining data integrity and traceability across all lifecycle stages. Implement schema enforcement, XSD validation, data quality checks, error handling, and automated SNS notifications; ensure all production jobs populate ETL Load Reports and Gap Reports through static and dynamic ETL metadata. Develop semantic‑layer objects (tables, views, materialized views) that ensure complete data coverage, optimized query performance, and consistent application of business logic. Develop XML parsing/shredding logic for high‑volume regulatory filings using Glue PySpark, supporting schema evolution and batch processing per program standards. Design pipelines with query performance in mind and support rollback, reload, and date‑range reprocessing capabilities without manual intervention. Support self‑service ETL development by other agency teams through standardized, reusable components aligned with program standards, and facilitate the transition of externally developed ETL jobs into the Data Engineering team's production support. Create and maintain required engineering artifacts, including business requirements, ETL design documents, mapping documents, data models, data dictionaries, deployment references, operations and maintenance guides, and test plans. Deploy code through automated CI/CD pipelines using CloudFormation templates, following agency release, security, and governance processes. Provide operational support for production jobs, including rapid identification and resolution of failed jobs and performance issues; participate in on‑call/after‑hours support for production outages and emergencies as part of a team rotation. Collaborate with Data Officers, Data Stewards, SMEs, data providers, and IV&V teams to understand requirements and deliver user‑accepted solutions; engage closely with the Product Owner and cross‑functional teams to provide timely updates and resolve issues. Leverage AI‑assisted development tools to accelerate coding, optimize workflows, and enhance code quality while adhering to security and performance standards. Work in Agile teams; drive iterative delivery, joint problem‑solving, and continuous improvement, including participation in sprint planning and program increment planning. Requirements Required Qualifications Bachelor's degree in Data Science, Computer Science, Engineering, or a related technical discipline. OR; In lieu of a degree, four additional years of related experience is required Minimum of 5 years of related data engineering experience developing and deploying data pipelines in production. Hands‑on experience developing ETL pipelines in AWS using Glue, Spark/PySpark, Lambda, and S3. Proficiency in Python (adhering to PEP8) and strong SQL skills, with experience integrating data from relational databases. Experience processing structured and semi‑structured data formats, including JSON, XML, CSV, Avro, and Parquet. Experience with workflow orchestration (Airflow/MWAA preferred). Experience with relational databases (PostgreSQL, Redshift, or Oracle) and lake table/file formats (Iceberg, Parquet, ORC). Experience with Git/GitHub version control and Agile methodologies. Strong attention to detail with a commitment to delivering high‑quality and accurate work; excellent written and verbal communication skills. Ability to obtain and maintain a Moderate Risk Public Trust clearance; residing in the United States Desired Qualifications Experience with financial/regulatory datasets, public company filings data, or other high‑sensitivity federal data. Experience with CI/CD, CloudFormation/Infrastructure‑as‑Code, and automated testing in a federal or FedRAMP environment. Familiarity with Zero Trust security principles, FISMA, NIST800‑53, Section508 of the Rehabilitation Act, and the Privacy Act of 1974. Experience with knowledge bases, vector stores, Amazon Bedrock, or OpenSearch supporting AI/ML and RAG workloads. AWS certifications (e.g., AWS Certified Data Engineer) are advantageous. Working Environment eSimplicity supports a hybrid work environment operating within the Eastern time zone so we can work with and respond to our government clients. Expected hours are 9:00AM to 5:00PM Eastern unless otherwise directed by your manager. Occasional travel for training and project meetings. It is estimated to be less than 5% per year. Benefits eSimplicity offers a comprehensive benefits package, including medical, dental, and vision coverage, 401(k) retirement benefits, paid time off, paid holidays, life and disability insurance, and additional wellness and employee support programs. Eligibility may vary based on employment status and applicable plan terms. Reasonable Accommodation eSimplicity is committed to providing reasonable accommodations to qualified individuals with disabilities during the application and hiring process. Applicants who need assistance or accommodation should contact Human Resources. Equal Employment Opportunity eSimplicity is an Equal Opportunity Employer, including disability and protected veteran status. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran status, disability, or any other legally protected status. #J-18808-Ljbffr