Senior Data Engineer (Databricks)

Senior Data Engineer (Databricks)

Join Sky Solutions, LLC - Pioneering Digital Transformation Excellence!

Sky Solutions is a trusted partner in government contracting, empowering federal agencies with cutting-edge IT solutions. Our expertise spans AI-first enablement, delivering innovative solutions and advanced cybersecurity, fortifying digital ecosystems against emerging threats and digital transformation through low-code/no-code development, cloud solutions, and agile methodologies. We simplify complex processes and deliver breakthrough results for federal clients across healthcare, financial services, and national security.

Location: Tysons Corner, VA (Hybrid / Remote)

Duration: Fulltime

Position Summary

The Senior Data Engineer will support a mission-critical data ecosystem built on Databricks and AWS. This role will be responsible for designing, implementing, optimizing, and governing enterprise-scale data pipelines while enabling secure, compliant, and high-performance analytics across IRS programs. The ideal candidate possesses deep expertise in Databricks, Delta Lake, Unity Catalog, data governance, and cloud-native data engineering practices.

Key Responsibilities

• Design and develop scalable data pipelines using Databricks and AWS services.
• Build and maintain Medallion Architecture data solutions.
• Develop Delta Live Tables (DLT) and Auto Loader pipelines.
• Implement Change Data Capture (CDC) solutions.
• Configure and administer Unity Catalog.
• Enable Delta Sharing and Clean Room capabilities.
• Support Databricks Genie and AI-powered analytics initiatives.
• Optimize workloads using: Photon, Liquid Clustering, Z-Ordering.
• Implement RBAC and ABAC security controls.
• Support data quality monitoring, observability, and lineage tracking.
• Develop dashboards and reporting solutions using Power BI, Tableau, and Databricks SQL.
• Participate in architecture reviews and technical design sessions.
• Collaborate with cloud engineering and cybersecurity teams to ensure compliance.

Required Qualifications

• Bachelor's degree in computer science, Information Systems, Engineering, or related field.
• 8+ years of Data Engineering experience.
• 3–5+ years of Databricks experience.
• Strong experience with: Databricks, Delta Lake, Spark, Python, SQL, AWS.
• Experience implementing enterprise data governance frameworks.
• Experience with CI/CD pipelines and Infrastructure as Code.
• Strong understanding of cloud security principles.

Preferred Qualifications

• Databricks Professional or Advanced Certification.
• AWS Certification.
• Federal Government experience.
• Experience supporting FedRAMP High environments.
• IRS or Treasury experience.
• Experience with Informatica EDC/Axon.

Sky Solutions Benefits & Culture at a Glance

• Healthcare & Wellness
  • Medical, dental, and vision coverage tailored to your needs.
  • Supplemental options like life insurance, disability, HSA/FSA available.
• Time Off & Work-Life Balance
  • Starting at 15 PTO days per year, increasing with tenure.
  • Plus 11 paid holidays to recharge and spend time with loved ones.
• Professional Growth & Development
  • Annual education stipend.
  • Access to thousands of online courses.
  • Formal mentorship programs for career advancement.
• Inclusive Culture
  • "One Team" mentality focused on collaboration and belonging.
  • Recognized as a Great Place to Work® for six years straight.

At Sky Solutions, LLC, we welcome applications from all qualified candidates who share our vision and values.