Principal Data Engineer

Data Engineer PositionBuilds data and reporting platforms in Cloud using Big Data technologies -- Hadoop, HBase, MongoDB, and Cassandra. Performs coding in Java, Scala, and Python. Automates Continuous Integration/Continuous Delivery (CI/CD) pipelines, using Git and Jenkins. Performs Extract, Transform, Load (ETL) data integration and develops data movement design patterns, using Informatica. Uses business knowledge to translate the vision for divisional initiatives into business solutions by developing complex or multiple software applications and conducting studies of alternatives. Analyzes and recommends changes in project development policies, procedures, standards, and strategies to development experts and management. Collaborate with business teams to gather and document functional requirements for data ingestion in Data Lake. Proactively engage with analysts/data modelers to understand requirements and deliver high-quality solutions.Primary Responsibilities:Participates in architecture design teams.Defines and leads enterprise-level data architecture strategies.Develops and implements advanced data engineering frameworks.Recommends development testing tools and methodologies and reviews and validates test plans.Responsible for QA readiness of software deliverables.Develops comprehensive documentation for multiple applications or subsystems.Establishes full project life cycle plans for complex projects across multiple platforms.Responsible for meeting project goals on-time and on-budget.Advises on risk assessment and risk management strategies for projects.Plans and coordinates project schedules and assignments for multiple projects.Acts as a primary liaison for business units to resolve various project/technology issues.Provides technology solutions to daily issues and technical evaluation estimates on technology initiatives.Advises senior leadership on data infrastructure and technology solutions.Mentors junior engineers.Performs independent and complex technical and functional analysis for multiple divisional initiatives.Develops innovative solutions to support scalable, secure, and high-performance data environments.Education and Experience:Bachelor's degree in Applied Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and five (5) years of experience as Principal Data Engineer (or closely related occupation) designing, building and maintaining data pipelines using Snowflake, Python, and Azure services for analytics within the financial services industry.Or, alternatively, Master's degree in Applied Computer Science, Engineering, Information Technology, Information Systems, or a closely related field (or foreign education equivalent) and three (3) years of experience as a Principal Data Engineer (or closely related occupation) designing, building and maintaining data pipelines using Snowflake, Python, and Azure services for analytics within the financial services industry.Skills and Knowledge:Demonstrated Expertise ("DE") creating enterprise data lakes used for analytics, Artificial Intelligence (AI), and Machine Learning (ML) capabilities on AWS, Microsoft Azure, or Snowflake, using big data processing techniques and solutions – Snowflake or Python; building data movement patterns using ETL/ELT tools; and migrating data between heterogeneous datastores using Snowflake, Python, Informatica, Azure Data Factory, or AWS services.DE designing complex dimensional models using ER Studio Data Architect with appropriate company taxonomy including Client/Customer, Account, Product and Supply Chain; and creating conceptual, logical and physical models according to relevant standards using single shared unique identifier across all common entities to deliver scalable and high performing solutions.DE designing, developing, and supporting data driven solutions and data warehouses, using Snowflake, AWS, Microsoft Azure, Python, or Azure Data Factory; creating data visualization using BI Reporting tools -- Tableau or Microsoft Power BI; deploying code changes using Continuous Integration/Continuous Delivery (CI/CD) tools – GitHub or Azure Pipelines; and establishing data governance, cataloging, and change management practices using Alation or Collibra.DE creating projects stories, backlogs, and references using enterprise tools – JIRA, Confluence, or SharePoint -- to execute project deliverables; and providing project management for Scrum teams -- creating and project managing business, design, development, and test stories -- according to Agile methodologies.