Sr. Data Engineer

Role Description

Assist and lead the design, development, and maintenance of scalable, high-performance data solutions that support analytics, AI, and business intelligence across the organization.

• Assist in designing and documenting technical requirements for data flows between diverse operational systems and the data warehouse.
• Support the end-to-end development of ETL/ELT processes using Apache Spark on Databricks.
• Contribute to the ingestion of data into data lakes and warehouses.
• Implement Delta Lake for transactional integrity.
• Help orchestrate both batch and streaming data pipelines.
• Collaborate with cross-functional technology teams on extract, transform, and load data from a variety of sources.
• Build and optimize data models.
• Maintain software applications aligned with business needs.
• Explore and integrate emerging AWS technologies to enhance data engineering capabilities.
• Write complex SQL queries for data validation and transformation.
• Manage Databricks workspaces, clusters, and jobs.
• Support data quality initiatives and ensure compliance with governance and security policies.
• Participate in the development and operationalization of machine learning and generative AI applications.
• Assist in troubleshooting production workflows and refactoring legacy systems.
• Participate in code reviews and mentor junior engineers.
• Requires a strong understanding of modern data architecture, a proactive approach to problem-solving, and a passion for building reliable, scalable, and innovative data solutions.
• 100% telecommuting allowed from a home office anywhere in the U.S.

Qualifications

• Bachelor’s Degree in Computer Science or related field of study.
• Five (5) years of experience in the job offered or a related occupation.

Requirements

• Experience in data engineering with Databricks and AWS.
• Experience with Apache Spark, SQL, and Python or Scala.
• Experience with Delta Lake, Unity Catalog, and cloud data warehouses (e.g., Redshift).
• Experience with data modeling, ETL/ELT processes, and data governance.
• Experience with orchestration tools including Gitlab or Bitbucket and CI/CD practices.
• Experience in collaborating with cross-functional teams to deliver data solutions using Python and Databricks in Agile environments.
• Experience with design and optimizing scalable data pipelines with Python and Spark in Databricks.
• Experience with developing modular, reusable components in Databricks notebooks and workflows.
• Experience in implementing data validation frameworks to ensure pipeline accuracy and consistency.

Benefits

• Competitive compensation package as part of Total Rewards.
• Base pay determined by performance, experience, skills, equity, and geographical markets.
• Other compensation may include an annual bonus or long-term incentive opportunities.