Big Data Engineer

Big Data Engineer (Mid-Level) - Remote | Experian | 12+ Month Contract

About the Role
Join Experian as a Big Data Engineer and drive the development, optimization, and support of scalable, high-performance data platforms in the AWS Cloud. Leverage your expertise to build robust data pipelines and API-driven microservices powering mission-critical systems. This is a fully remote, long-term contract offering you autonomy and technical growth on innovative projects.

Responsibilities


- Design, build, and maintain large-scale batch and real-time data pipelines using Spark, Hadoop, and AWS EMR
- Develop, deploy, and optimize API-based microservices leveraging Java, Scala, and AWS API Gateway
- Implement data ingestion, transformation, and automation workflows with PySpark, Snowflake, and Airflow
- Tune and scale data processing systems for maximum performance and reliability
- Collaborate cross-functionally to deliver high-quality, production-ready solutions in a cloud-first environment
- Troubleshoot and resolve complex issues in Linux-based environments using shell scripting and Python
- Ensure best practices in code versioning and CI/CD processes for continuous integration and delivery

Required Skills and Experience


- 7+ years designing, developing, and testing Big Data solutions (batch and APIs)
- Deep hands-on AWS experience: EMR, EC2, ECS, S3, Step Functions, API Gateway, Airflow
- Proficiency in Scala, PySpark, Java (REST APIs, SOAP web services)
- Strong knowledge of Hadoop, HDFS, Spark, MapReduce, and Cassandra
- Experience with performance tuning in Spark, Hadoop, and EMR environments
- Solid Linux administration skills with shell scripting and Python automation
- Proven track record building data pipelines and transformations in Snowflake using Spark

Preferred Skills


- Familiarity with DevOps tooling: GitHub, GitLab, Bitbucket, IntelliJ, PyCharm
- Experience with CI/CD: Maven, Gradle, Jenkins, Artifactory, AWS CodeCommit, CloudFormation

Benefits


- Fully remote work environment - collaborate from anywhere in the U.S.
- Work with cutting-edge AWS and Big Data tools
- Opportunity to own challenging, high-impact projects
- Long-term contract with potential for future engagement
- Expand your expertise with a global leader in data-driven solutions

How to Apply
Ready to shape the future of data engineering at Experian? Submit your resume today to take the next step in your Big Data engineering career!