Data Engineer

Location: West Chester,PA-- 100% ONSITE FROM DAT1

Duration: 6 months

Requisition ID: 10934365

Note: PLEASE SUBMIT ONLY QUALITY PROFILES MATCHING TO THE JOB DESCRIPTION

Role: Data Engineer

Skills: Digital : BigData and Hadoop Ecosystem - MapR~Digital : Databricks~Digital : PySpark Experience Required: 6- 8 Years

Role Descriptions:

Big Data| Spark| AWS| Databricks| Docker| Kubernetes and other Data Warehouse technologies Strong Knowledge on Spark| Scala| Python| PySpark AI Strong in Teradata SQL concepts Good Knowledge on Bigdata Technologies like|Kafka|.

Experience in Version Control tools like| GITHUB as well as experience in Production deployment process.

Hands-on development with AWS services: Glue| Lake Formation| S3| DynamoDB| SQS| SNS| Aurora| Lambda| Step Functions| API Gateway| IAM| CloudWatch| Kinesis| Elasticsearch| Airflow| Event Bridge| Grafana Basic knowledge in below skill sets:

a.SQL /RDBMS

b.Linux Shell Scripting

c.Data Warehousing concepts

The candidate must have experience in Docker and containerization technologies.

They must have knowledge about container orchestration tools like Kubernetes and logging/monitoring toolsStrong Communication"

Essential Skills:

Interacting with IT Business Analysts and / or our business partners to acquire the information necessary for product design.

Following written requirements to help design databases and develop dashboards to support our customers.

Translating raw data from multiple sources (ETL) into a single dataset at aggregate levels necessary to meet project needs.

Suggesting alternative ways to group| filter and sort aggregated data for optimal presentation and performance possibilities.

Providing front-line production support by analyzing customer questions| evaluating data sets to explain end results or quickly adjust processes based on consumer demands.

Working independently and with groups while engaging in multiple initiatives simultaneously in a fast-paced IT environment.

Resolving business process conflicts as they relate to project specifications.

Reporting project statuses to IT Management staff as required.

Review current environment that is used for product launch. Design and recommend the go forward architecture using Spark| Python and Teradata.

Migrate to and implement the new rules and jobs to support Operational Intelligence Set up and configure alerting and monitoring Create STTMs| ERDs| data dictionaries and other knowledge artifacts for the NDW Base environment The person should have the ability to leverage BTEQ scripts| familiar with Informatica| UC4/Atomic and obviously strong SQL skills."