Big Data Engineer

Purple DriveBig Data Engineer Share Contractual Sunnyvale, CA | Bentonville, AROverview:Key ResponsibilitiesDesign, develop, and maintain big data pipelines leveraging Hadoop, Spark, and Kafka.Develop and optimize J2EE backend services for large-scale data processing and integration.Program in Python, Scala, and Java to build robust data and backend solutions.Architect and implement cloud-native solutions on GCP and Azure platforms.Design and maintain data models, data warehousing solutions, and BI tools to enable data-driven decision making.Implement and manage workflow orchestration frameworks such as Automic and Airflow for data pipeline automation.Collaborate with cross-functional teams including data scientists, analysts, and DevOps engineers to deliver end-to-end data solutions.Ensure data quality, security, and scalability across all platforms.Troubleshoot and optimize performance of large-scale distributed systems.Required Skills & QualificationsBachelor's/Master's degree in Computer Science, Engineering, or related field (or equivalent experience).6–8 years of experience in big data engineering and backend development.Expertise in Hadoop ecosystem (HDFS, MapReduce, Hive, Spark) and Kafka.Strong programming skills in Python, Scala, and Java.Hands-on experience with J2EE backend development.Proven experience with cloud services (GCP, Azure) for data engineering and application deployment.Proficiency in data modeling, warehousing concepts, and BI tools.Experience with workflow orchestration tools such as Airflow, Automic.Strong problem-solving and debugging skills in large-scale distributed environments.Preferred Skills (Nice to Have)Familiarity with containerization and orchestration (Docker, Kubernetes).Knowledge of streaming frameworks (Spark Streaming, Flink, Storm).Exposure to CI/CD pipelines and DevOps practices.Experience in Agile/Scrum development environments.