Senior Data Engineer

Brooksource is hiring a Senior Data Engineer for a 12-month contract-to-hire role based on-site in Kansas City, MO . In this position, you’ll design and build dependable data pipelines that turn raw inputs into clean, consumable datasets for enterprise analytics and downstream AI/ML consumption. This role focuses on hands-on ETL/ELT work, data modeling, and keeping pipelines operating reliably in production through monitoring, validation, and governance practices. Responsibilities Design, develop, and maintain scalable and reliable data pipelines and platforms for enterprise analytics and reporting. Work with business and technical teams to understand data requirements and translate them into data engineering solutions. Create ingestion frameworks using Kafka Confluent and integrate data from multiple cloud and on-premises sources . Build ELT/ETL pipelines using PySpark , SQL , Dataform , DBT , and Dataproc . Design and manage data models, schemas, and data warehousing structures for performance and scalability. Ensure data quality, consistency, and integrity using validation and monitoring mechanisms. Implement data governance practices including data lineage, metadata management, privacy, and compliance. Develop automated monitoring and alerting to support reliability, availability, and performance. Collaborate with cross-functional teams to support downstream reporting, analytics, and AI/ML model consumption. Follow best practices for version control, documentation, and CI/CD using Git and infrastructure-as-code tools. Provide operational support for data pipelines and troubleshoot performance issues proactively. Continuously optimize data processing, query execution, and resource utilization in Azure environments. Build and maintain RESTful API integrations to support data ingestion and synchronization with web applications and enterprise systems. Contribute to lightweight web application development efforts when pipelines interface directly with front-end or full-stack components. Requirements Bachelor Degree in Computer Science or related field and 8 years related professional experience (required). Associate Degree and 10 years related professional experience (required). High School Diploma/GED and 12 years related professional experience (required). Strong programming skills in Python and PySpark for large-scale data processing. Proficiency in SQL for data manipulation, analysis, and performance tuning. Experience with Dataform , Dataproc , and BigQuery for pipeline development and orchestration. Understanding of data governance principles and implementation practices. Experience using Git for version control and CI/CD pipelines for automated deployments. Experience building and consuming RESTful APIs , including enterprise integrations such as OAuth2 , OpenID Connect , webhooks, pagination, rate limiting, retry logic, and error handling. Some hands-on web application development experience (full-stack or front-end). Candidates must be legally authorized to work permanently in the country where this position is located, without time limitations, restrictions, or the need for work sponsorship. Technologies Python, PySpark, SQL Dataform, DBT, Dataproc, BigQuery Kafka, Confluent Azure, Git RESTful APIs OAuth2, OpenID Connect, webhooks Infrastructure as Code RESTful API integrations Nice to Have Experience configuring and building pipelines within Databricks . Working knowledge of Infrastructure as Code (IaC) for cloud resource management and automation. Familiarity with DBT , Machine Learning , and AI concepts. Hands-on experience with Kafka and Confluent for real-time data streaming. Knowledge of Cloud Scheduler and Dataflow for automation and workflow management. Full-stack application development experience using Node.js , TypeScript , React , and Express . Experience designing contract-first APIs using OpenAPI , Orval , and Zod . Proficiency with React , Vite , Tailwind CSS , shadcn/ui , and TanStack Query . Experience with PostgreSQL and ORM frameworks (Drizzle ORM preferred). Experience integrating engineering, construction, ERP, or project management systems. Experience with metadata-driven applications, configurable mappings, and schema transformation frameworks. Benefits Dental insurance Health insurance Vision insurance Additional Benefits & Perks Competitive medical, dental, vision, Health Savings Account, Dependent Care FSA, and supplemental coverage. 401k plan with company match and fully vested after eligibility, plus paid time off, sick time, and paid company holidays. Employee Assistance Program (EAP) including virtual counseling, financial services, legal services, and life coaching. Compensation: USD 55 to 60 per hourly. Pay disclaimer: The pay range is a general guideline only and not a guarantee of compensation. Factors considered include responsibilities, education, experience, knowledge, skills, abilities, internal equity, alignment with market data, applicable bargaining agreements (if any), and other law. #J-18808-Ljbffr