Data Engineer

Data Engineer

Key responsibilities for the Data Engineer:

• Optimize data pipelines that read from Kinesis and Write into Oracle improve exception handling for schema/data type mismatches and implement alerting for data quality issues.
• Build services to read/write data to NetApp object store using the S3 API.
• Create analytical datasets in Parquet or Iceberg format.
• Develop housekeeping scripts for cleanup, archiving, and compression of logs, Object store files, etc.

Core skill requirements:

• Strong Python/Java and data engineering expertise.
• Experience building and deploying data pipelines using frameworks like Spark, Flink and understanding of distributed data systems.
• Hands-on experience with real-time streaming architectures, REST APIs, and database connectors.
• Proficient with Relational Databases (e.g. Oracle).
• Experience working with Amazon S3 APIs for data storage and integration.

Preferred (nice-to-have):

• Experience with Apache Iceberg.

About the company

Avance Consulting is a global leader in innovative talent solutions for diverse industries. Since 2007, we have been serving nearly a third of Fortune 500 companies and most of the top 100 technology companies worldwide.

Our team of 700+ experts offers Engineering Services, Information Technology, Digital and Executive Search solutions from our offices in North America, UK, Europe and APAC.

We understand the local markets and cultures and tailor our services to meet the needs of our clients. Our clients value our flexibility and agility, which enable us to deliver high-quality results at any scale across the globe.

This allows us to work with numerous industries at varying scale across the globe while ensuring the excellent service standard that our clients have come to expect.

At Avance you can learn new skills and technologies, have more advancement opportunities and career growth, being a part of our productive atmosphere, and a positive reinforcement culture.