Senior Data Engineer

7+ years of experience as a data engineer having configured and built pipelines within Databricks Strong Python and Pyspark coding skills Experience with SQL when it comes to data manipulation, analysis, and performance tuning Experience with Dataproc, Dataform and Big Query for data pipeline development Understands data governance and best practices Familiar with Git and CI/CD pipelines for automated deployments Bachelor's Degree in Computer Science or similar Plusses Knowledge of Infrastructure-as-code (IAC) Familiar with DBT and Machine Learning AI concepts Experience with Kafka and Confluent for data streaming Day-to-Day Insight Global is looking for a Senior Databricks Engineer to work onsite for one of our client's in the Kansas City area. In this role, we are looking for this person to be responsible for the design, build, and maintenance of scalable data pipelines and platforms that support enterprise analytics, reporting, and data-driven decision-making. This role is responsible for developing ETL/ELT workflows using technologies such as PySpark, SQL, Kafka Confluent, DBT, Dataform, and Dataproc, while integrating data from both cloud and on-premises systems. The ideal candidate will collaborate closely with business, product, and technical teams to translate requirements into efficient data solutions, optimized data models, and high-performing data warehouse structures. Responsibilities include ensuring data quality, governance, lineage, monitoring, and pipeline reliability, as well as troubleshooting and optimizing performance within Azure environments. Additionally, this individual will build REST API integrations, support downstream analytics and AI/ML initiatives, and contribute to CI/CD, automation, and lightweight web application development where data services interact with front-end systems. #J-18808-Ljbffr Insight Global