Big Data Engineer

Big Data Engineer

Immediate need for a talented Big Data Engineer. This is a Fulltime opportunity with long-term potential and is located in Detroit MI (Hybrid). Employee benefits include, but are not limited to, health insurance (medical, dental, vision), 401(k) plan, and paid sick leave (depending on work location).

Key Responsibilities:

• Design, develop, and maintain scalable big data pipelines and ETL workflows using AWS Glue and Python.
• Build and optimize Spark-based Glue jobs for efficient data processing and transformation.
• Develop and manage real-time data streaming solutions using Apache Kafka.
• Create and support event-driven serverless applications using AWS Lambda.
• Design, optimize, and maintain Client data warehouse solutions, including schema design, query optimization, and performance tuning.
• Write advanced SQL queries for large-scale data transformation, reporting, and analytics.
• Manage and integrate AWS cloud services such as S3, CloudWatch, SES, and IAM.
• Implement CI/CD pipelines and infrastructure automation using Terraform and Git Actions.
• Monitor data workflows and troubleshoot issues related to data quality, system performance, and scalability.
• Collaborate with cross-functional teams including developers, analysts, and business stakeholders to deliver data-driven solutions.
• Ensure data security, reliability, and operational efficiency across cloud-based data platforms.
• Participate in code reviews, technical discussions, and continuous improvement initiatives.
• Work effectively in a fast-paced hybrid environment with minimal supervision.

Key Requirements and Technology Experience:

• Strong Python experience (hands-on coding required)
• AWS Glue experience with Spark-based Glue jobs
• Apache Kafka experience for real-time streaming
• Experience with Client data warehousing
• Advanced SQL skills
• Experience with AWS Lambda
• Experience with Terraform
• Experience with Git Actions / CI-CD
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.
• Ability to work in a fast-paced, dynamic environment with minimal supervision.
• Spark optimization experience
• Real-time data pipeline architecture
• Event-driven/serverless architecture
• AWS certification
• Experience in enterprise-scale data engineering
• Strong communication/client-facing experience
• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field is preferred