Senior Bigdata Engineer

Position: Senior Data Engineer


Location: Jersey City, NJ (2-3 times per week)

Responsibilities


Work with developers and architects to advise on standards and best practices.


Good understanding of Database architectures and data models.


Creating and maintaining technical documentation.


Actively work with onshore team. -Exposure of working in Agile model.

Skills, Must have


Exp range 7-9 years.


Very good knowledge on ETL Architecture and development using Abinitio/Informatica.


Very good knowledge on Data warehouse Concepts and Architecture.


Very good knowledge on doing Big Data ETL operations using Syncsort integrated platform.


Very Good Knowledge on SQL concepts (Oracle, MySQL).


Knowledge on Snowflake DB/Mongo DB/No SQL.


Good exposure on Enterprise Schedule (like Control-M).


Should have exposure to Extensive Operations Support on the above-mentioned applications.


Good knowledge in scripting skills using Python/Unix.


Cloud environment (AWS, Azure, or GCP) is a plus


Familiarity with other big data technologies (Hadoop, Spark, Kafka, etc.).


Experience with version control systems (Git, SVN, etc.) and CI/CD pipelines.


Coordinate with the Systems Engineering team to plan and execute system upgrades of existing warehouse application tools.


Maintain the software development disciplines including version control using Git repository.

Nice to have


Insurance domain experience.


Certifications or training in Data Bricks or PySpark.


Good understanding of AWS data services such as Lambda, EMT, Athena, Glue, Redshift.


Preparing Dashboards and metrics collection using tools such as Grafana, Datadog.


Automation of monitoring frameworks for key business jobs.