Senior, Data Engineer

Senior, Data Engineer

Position Purpose

Provide a brief description as to the primary purpose of this job (no more than three to five sentences).

Primary Duties/Responsibilities

• Develop and operate data management tools, monitoring data flows, data quality, data cleansing and data processing
• Create and document logical data integration strategies for data flows between disparate systems and the enterprise data warehouse/data lakes
• Collaborate with different stakeholders (engineers, data stewards) to collect required data from internal and external systems
• Work in an Agile environment that focuses on collaboration and teamwork
• Improve and extend existing data infrastructure services
• Monitor production job schedule and correct job failures in a timely manner

Qualifications

Education / Experience / Background - MS degree in Computer Science or computer related field from an accredited institution. 5+ years hands proven experience as a Data Engineer or similar role. 5+ years of strong experience building, running and maintaining datalake(s) and warehouse(s) in a cloud environment.

Knowledge / Skills / Abilities

• More than 4 years of experience developing with Python
• 4+ years performing with production environments in a DevOps culture managing code composed of multi-developer teams, following industry best practices
• 4+ years SQL development experience
• Experience with data modeling
• 4+ years bash scripting experience
• Strong experience with Git, CI/CD (preferably GitLab) and Docker
• Experience deploying and running services in Cloud Big Data platforms such as BigQuery and Snowflake
• Strong experience with GCP services
• Experience designing and building data pipelines using tools like Apache Beam, CDAP (Data Fusion) or other ETLs
• Knowledge with CDC design patterns and their challenges
• Experience with DAG workflows orchestration such as Apache Airflow
• Experience with NoSQL databases is a plus (i.e Firestore, MongoDB)
• Experience designing and developing APIs is a plus (i.e using FastAPI, Flask)
• Abilities: Being able to work in a large company with different stakeholders. Embrace mentorship through design sessions, code reviews, and community building. Take ownership and support solutions you develop. Value collaboration with other members of the team. Have a product mindset. Good communication.

Required Certification / Licenses / Training - (Nice to have) Google Cloud Data Engineer

Physical Requirements / Work Environment

Hybrid mode (2 days a week in the office)