ETL Developer

Datastage Etl Software Upgrade Project

MMB’s DataStage ETL tool supports the transfer of data from the PeopleSoft source system (Financial Management Solutions (FMS), Human Capital Management (HCM), Enterprise Learning Management (ELM)) to the EPM Data Warehouse which agency users across State government use for reporting. The current DataStage software operates on Windows 2008 physical machines which are no longer supported; therefore MNIT/MMB has undertaken an upgrade of the DataStage software, which includes moving the servers to supported server operating systems.

The upgrade project is underway and requires the migration of approximately 1,700 server jobs to bring it to completion. With the exception of a relatively small number of jobs that still need to be migrated to the DataStage V11 Test instance, the server jobs have generally already been migrated to the DataStage V11 Test instance, and now need to be tested/fixed and migrated to the DataStage V11 Production instance. The contractor resources under this engagement will perform these migration tasks, supplementing State staff members who will also be performing this work. State staff will provide project management. The contractor resources will work closely with the Data Warehouse Team lead along with other project team members.

The metadata database is Oracle 19C and the Data Warehouse is Oracle 12C. DataStage Version 11.7 on Red Hat is up and running and has already been configured.

During the production batch process, data is moved from the source Oracle databases into a staging layer in the Warehouse database and then moved to a multi-dimensional layer used for reporting using Oracle Business Intelligence Enterprise Edition (OBIEE). The Warehouse nightly batch takes about 10 hours to complete each night. Each upgraded job will need to be performance tuned so that it takes the same or less time to load when compared to DataStage V 8.5.

Tasks

• Test migrated ETL jobs, including data validation and data integrity testing
• Fix issues found during migration or testing
• Create new ETL jobs and update existing ETL jobs
• Migrate ETL jobs from DataStage V8.5 server to DataStage V11.7 server
• Document processes and details as directed by team lead
• Assist with implementation of ETL jobs into the production batch process as needed
• Performance tune ETL jobs as needed
• Provide knowledge transfer to State staff as needed
• Perform other related duties as assigned

Minimum Qualifications

• Three (3) years’ experience in DataStage ETL developer role(s)
• Two (2) years’ experience with Oracle PL/SQL development

Desired Qualifications

• Experience with DataStage Server and Parallel jobs, Routines, hash files, and/or sequences
• Experience testing and/or performance tuning ETL jobs
• Experience with batch scheduling tools, e.g., Automic, Control-M, etc.