Datastage Developer /Python

Datastage Developer /Python

Columbus, Ohio(Hybrid)

Video Interview

Long Term Contract

JD:

Can be in anyone of these HNB footprints:

• Columbus, OH
• Dallas, TX
• Schaumburg, IL
• Charlotte, NC
• Indianapolis, IN
• Denver, CO
• Pittsburgh, PA
• Detroit, MI
• Waterloo, IA

MUST have python and datastage

Develop Shell scripts for automation of the data flow process and ETL orchestration pipeline.

Implement ETL transformations using DataStage, Hadoop Big data technologies like Spark-Scala/Spark-Java, Hive, or Pig based on the business use case.

Develop One-time/On-demand data Extractions from the data marts in the database systems for the business users for analytical purposes.

Direct Podium for end-to-end data management.

Develop automation and ETL jobs to schedule data processing utilities using reusable ETL components and frameworks of the data pipelines.

Implement Data profiling, Data quality, and Data standardization for all the workflows based on EDM standards

(Enterprise Data Management)

Build Hadoop Data Lake for analytics using Hadoop echo system.

Migrate tables from traditional RDBMS into Data Lake using Sqoop.

Transfer real-time data from different data sources into HDFS systems using Kafka producers, consumers, and Kafka brokers.

Use cloud-native development, micro-services architecture, operations, and governance framework to gather required technical data for purposes of formulating recommended customization of AWS.

Perform Unit and regression testing, prepare unit test case documents for comparing the results with business requirements.

Ensure the technical implementations are in line with the requirements and specifications.

Help Quality Analyst Team in preparing a comprehensive test plan to ensure proper functioning of the application for the zero-defect delivery for the continuous integration and deployment to the production.

Prepare Technical Specification documents that explain the technical solutions and list the technologies to be used while implementing the solution.

Prepare data models to be certain that the data objects are represented accurately by the ETL and Data engineering software and application tools.

Participate in business requirement meetings, perform requirement, and impact analysis with Business Analysts.

Coordinate with business users to understand the data sources/entities.

Design the on-prem or cloud data warehousing models using ETL tools such as IBM Infosphere Data Stage, Oracle, Python, and cloud platforms such as Snowflake and AWS (Amazon Web Services)

Create Technical design documents as part of project deliverables.

Analyse Teradata databases to design extract strategies into Big data and cloud platforms.