Data Engineer

Role: Data Engineer
Location: West Chester,PA-- 100% ONSITE FROM DAT1
Duration: 6 months
Experience Required: 6- 8 Years

Note: PLEASE SUBMIT ONLY QUALITY PROFILES MATCHING TO THE JOB DESCRIPTION

Skills: Digital : BigData and Hadoop Ecosystem - MapR~Digital : Databricks~Digital : PySpark

Role Descriptions:
Big Data| Spark| AWS| Databricks| Docker| Kubernetes and other Data Warehouse technologies
Strong Knowledge on Spark| Scala| Python| PySpark AI
Strong in Teradata SQL concepts
Good Knowledge on Bigdata Technologies like|Kafka|.
Experience in Version Control tools like| GITHUB as well as experience in Production deployment process.
Hands-on development with AWS services: Glue| Lake Formation| S3| DynamoDB| SQS| SNS| Aurora| Lambda| Step Functions| API Gateway| IAM| CloudWatch| Kinesis| Elasticsearch| Airflow| Event Bridge| Grafana
Basic knowledge in below skill sets:
a.SQL /RDBMS
b.Linux Shell Scripting
c.Data Warehousing concepts
The candidate must have experience in Docker and containerization technologies.
They must have knowledge about container orchestration tools like Kubernetes and logging/monitoring toolsStrong Communication"

Essential Skills:
Interacting with IT Business Analysts and / or our business partners to acquire the information necessary for product design.
Following written requirements to help design databases and develop dashboards to support our customers.
Translating raw data from multiple sources (ETL) into a single dataset at aggregate levels necessary to meet project needs.
Suggesting alternative ways to group| filter and sort aggregated data for optimal presentation and performance possibilities.
Providing front-line production support by analyzing customer questions| evaluating data sets to explain end results or quickly adjust processes based on consumer demands.
Working independently and with groups while engaging in multiple initiatives simultaneously in a fast-paced IT environment.
Resolving business process conflicts as they relate to project specifications.
Reporting project statuses to IT Management staff as required.
Review current environment that is used for product launch. Design and recommend the go forward architecture using Spark| Python and Teradata.
Migrate to and implement the new rules and jobs to support Operational Intelligence Set up and configure alerting and monitoring
Create STTMs| ERDs| data dictionaries and other knowledge artifacts for the NDW Base environment
The person should have the ability to leverage BTEQ scripts| familiar with Informatica| UC4/Atomic and obviously strong SQL skills."