Data Engineer

Position Description

SiloSmashers, Inc. is a leading management and IT consulting firm, serving as a trusted advisor and solutions partner to a diverse range of government, business, and not-for-profit organizations. Applying proven expertise and methodologies, we help our customers align and mobilize every aspect of their operations, strengths, and resources to achieve Peak Performance and increased business value today and through the years ahead.

Position Overview

SiloSmashers is hiring for a Data Engineer to support digital transformation efforts for a government customer. The selected candidate will serve as part of a team responsible for architecting, developing, optimizing, and maintaining high-performance data ingestion, transformation, streaming, and analytical pipelines within Databricks-enabled Lakehouse environments. This role is expected to collaborate closely with the federal technical teams, and cross-task platform, SRE, governance, privacy, and development personnel.

Required Skills
• Foundational experience designing and implementing Databricks Lakehouse solutions, including:

o Basic Delta Lake table design, schema evolution, and familiarity with medallion architecture (bronze/silver/gold).

o Hands on experience with Databricks Workflows/Jobs, Auto Loader, COPY INTO, Structured Streaming, and introductory Databricks SQL Warehouse usage. Understanding of common performance considerations (e.g., cluster sizing, partitioning, basic caching strategies).

o Practical experience building batch or streaming pipelines using Python and PySpark, along with cloud native services in AWS or Microsoft Azure.
• Experience implementing secure cloud data integrations or platform configurations such as external locations, storage credentials, secret scopes, or identity federation basics.
• Experience working with Git based workflows and CI/CD tooling (e.g., GitHub Actions, Azure DevOps), plus familiarity with Databricks Repos, Asset Bundles, or basic Terraform/IaC concepts.
• Introductory experience or interest in AI/ML enabled engineering, such as Databricks Model Serving, Genie, Agent Bricks, or equivalent cloud native ML platforms.

Mandatory Qualifications & Requirements
• Ability to obtain and maintain a Public Trust clearance.
• Eight (8) or more years of experience (2 of which much be within the last five (5) years) in data engineering, ETL/ELT development, or distributed data processing within cloud environments or platforms (e.g. AWS, Azure, GCP, Snowflake, Databricks, etc.).
• Bachelor's degree in computer science, information systems, engineering, or related field; or four (4) or more years of equivalent experience within the last six (6) years in lieu of a degree.
• Databricks Data Engineer Associate Certification.
• At least one foundational cloud certification (e.g., AWS Cloud Practitioner, AWS Developer Associate, Azure Fundamentals, or equivalent).
• Preferred but not required: Databricks Lakehouse Architect, Generative AI Associate/Professional, or related ML/AI certifications.
• This position requires regular, physical attendance at customer sites.