Senior Data Engineer

Job Description

Job Description

We are looking for an experienced Senior Data Engineer to help design and enhance scalable data platforms that support critical business operations in Houston, Texas. This role is ideal for someone who enjoys building reliable pipelines, improving data accessibility, and working with modern big data technologies in a service-focused environment. The successful candidate will play a key role in shaping data architecture, enabling analytics, and maintaining high standards for performance, quality, and governance.

Responsibilities:
• Design, build, and optimize large-scale data pipelines that collect, transform, and deliver data from multiple sources.
• Develop robust ETL workflows using Python and Spark to support reporting, analytics, and operational data needs.
• Implement and maintain data solutions across distributed processing environments, including Hadoop and Azure Databricks.
• Create streaming and batch data integrations using Kafka and related technologies to ensure timely and dependable data movement.
• Collaborate with analysts, engineers, and business stakeholders to translate requirements into scalable technical solutions.
• Monitor pipeline performance, troubleshoot data issues, and apply improvements that increase reliability and efficiency.
• Establish data quality controls, validation processes, and documentation to support governance and long-term maintainability.
• Contribute to platform enhancements and technical initiatives, including changes to data infrastructure or internal processing frameworks when needed.• 7+ years of experience in data engineering or a closely related field.
• Strong hands-on expertise with Python for data processing, automation, and pipeline development.
• Advanced experience working with Apache Spark in large-scale data environments.
• Solid knowledge of Apache Hadoop and distributed data processing concepts.
• Experience building or supporting real-time and batch integrations using Apache Kafka.
• Proven background in designing and maintaining ETL processes for complex data ecosystems.
• Familiarity with Azure Databricks and cloud-based data engineering best practices.