Data Engineer

We are seeking an experienced Databricks Data Engineer to support a mission-critical federal program. The ideal candidate will have hands-on experience building and maintaining scalable data pipelines and data platforms using Databricks, Apache Spark, Python, and SQL.

This role will work with large, complex datasets in a secure environment and collaborate with data engineers, software engineers, data scientists, and mission stakeholders to make data accessible, reliable, and usable for analytics and operational requirements.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark
• Build ETL/ELT workflows for ingesting, transforming, and processing structured and unstructured data
• Develop data processing solutions using Python, PySpark, and SQL
• Build and optimize Delta Lake / Lakehouse architectures
• Integrate data from APIs, databases, cloud storage, and other enterprise data sources
• Implement data quality, validation, governance, and monitoring processes
• Optimize Spark workloads and Databricks jobs for performance and scalability
• Support automated deployment of data pipelines through CI/CD
• Work with cloud services and modern data architectures in AWS or Azure
• Collaborate with data scientists and analysts to prepare datasets for analytics, reporting, and machine learning
• Troubleshoot data pipeline, integration, and performance issues
• Maintain technical documentation for data pipelines, architectures, and processes
• Operate within federal security and information assurance requirements
• Required Qualifications
• Active TS/SCI clearance
• Bachelor's degree in Computer Science, Engineering, Information Technology, Data Science, or a related technical field, or equivalent professional experience
• 3+ years of professional data engineering or software engineering experience
• Hands-on experience with Databricks
• Strong experience with Apache Spark / PySpark
• Strong programming experience with Python

Strong SQL skills

• Experience developing ETL/ELT data pipelines
• Experience working with large-scale or distributed datasets
• Knowledge of data modeling, data warehousing, and modern data lake/lakehouse architectures
• Experience with Git and modern software development practices
• Preferred Qualifications
• Experience with Databricks Delta Lake
• Experience with Unity Catalog
• Experience with Databricks Workflows or Jobs
• Experience with AWS services such as S3, Glue, Lambda, Redshift, EMR, or IAM
• Experience with Azure services such as Azure Data Lake Storage, Azure Data Factory, Synapse, or Azure DevOps
• Experience with orchestration tools such as Apache Airflow
• Experience with Terraform or other Infrastructure-as-Code tools
• Experience with Docker and Kubernetes
• Experience implementing CI/CD pipelines
• Knowledge of data governance, lineage, access controls, and data security
• Databricks certifications such as Databricks Certified Data Engineer Associate or Databricks Certified Data Engineer Professional
• Previous experience supporting DoD, Intelligence Community, or other federal government customers

Clearance:

Applicants selected will be subject to a security investigation and may need to meet eligibility requirements for access to classified information. TS/SCI clearance required.

Benefits: Gridiron offers a comprehensive benefits package including medical, dental, vision insurance, HSA, FSA, 401(k), disability & ADD insurance, life and pet insurance to eligible employees. Full-time and part-time employees working at least 30 hours per week on a regular basis are eligible to participate in Gridiron’s benefits programs.

Gridiron IT Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status or disability status.

Gridiron IT is a Women Owned Small Business (WOSB) headquartered in the Washington, D.C. area that supports our clients' missions throughout the United States. Gridiron IT specializes in providing comprehensive IT services tailored to meet the needs of federal agencies. Our capabilities include IT Infrastructure & Cloud Services, Cyber Security, Software Integration & Development, Data Solution & AI, and Enterprise Applications. These capabilities are backed by Gridiron IT's experienced workforce and our commitment to ensuring we meet and exceed our clients' expectations.