Data Engineer

What we are looking for

• Education & Experience
• Education: Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field (or equivalent professional experience).
• Core Experience : Proven experience developing end-to-end data pipelines extracting/transforming/loading data from REST APIs, relational databases, cloud storage, and flat files.
• Snowflake Proficiency : Demonstrated hands-on experience with virtual warehouses, streams, tasks, stages, Snowpipe, secure data sharing, and performance optimization.
• SQL : Advanced SQL development skills with the ability to write complex queries, tune performance, and optimize large-scale workloads.
• Data Modeling : Experience with dimensional modeling techniques (star schemas, fact tables, dimension tables).
• Cloud & DevOps :
• Familiarity with cloud-based data ecosystems, particularly Microsoft Azure .
• Managing source code and CI/CD pipelines using Git and Azure DevOps (or similar).
• Soft Skills & Practices :
• Strong analytical, problem-solving, and detail-oriented mindset.
• Excellent verbal and written communication skills; ability to collaborate in a fast-paced environment with evolving priorities.
• Knowledge of data integration best practices, data governance, and enterprise data management.
• Preferred / Nice-to-Have
• Certifications : SnowPro, Azure Data Engineer Associate, or equivalent cloud data platform certifications.
• Advanced Tech : Big data technologies, machine learning, data science platforms, or advanced analytics.
• BI Tools : Power BI, Tableau, or similar visualization platforms.
• Methodologies : Agile delivery frameworks and DevOps practices.
• Preferred Tech Stack Summary
• Core : Snowflake, Python/PySpark, SQL
• Cloud & Orchestration : Azure Data Lake Storage (ADLS), Azure Data Factory, Azure Key Vault
• DevOps & Infrastructure : Git, Azure DevOps, CI/CD, Infrastructure as Code (Terraform preferred)
• Integration & Analytics : REST APIs, Power BI

Responsibilities

• Data Engineering & ETL Development
• Design, develop, and maintain scalable ETL/ELT pipelines using Python (PySpark) , Snowflake , and cloud-native technologies.
• Build reliable, efficient, and reusable data ingestion, transformation, and loading processes.
• Snowflake Data Platform & Warehousing
• Utilize Snowflake’s architecture to design, build, and optimize modern cloud data solutions.
• Implement and manage Snowflake objects (databases, schemas, tables, views, streams, tasks, stages, stored procedures).
• Leverage virtual warehouses, data sharing, time travel, and automated scaling to balance performance and cost efficiency.
• Apply dimensional modeling (star schemas, facts, dimensions) to build scalable enterprise data warehouses.
• Data Integration & Modeling
• Extract and ingest structured and semi-structured data from REST APIs, relational databases, SaaS apps, flat files, and cloud storage.
• Develop robust ingestion frameworks.
• Collaborate with data architects and stakeholders to create logical and physical data models aligned with business goals.
• Cloud Architecture & Standards
• Contribute to modern data platform concepts (data lakes, lakehouses, data mesh architectures, enterprise data catalogs).
• Support integration between Snowflake and cloud-native services (primarily Azure ).
• Establish and enforce data engineering standards and best practices.
• Quality, Governance & Security
• Implement automated data quality controls, validation frameworks, and monitoring processes.
• Support data governance initiatives and maintain adherence to organizational standards.
• Ensure data security, privacy, and regulatory compliance (access controls, masking policies, industry best practices).
• Optimization, Operations & Maintenance
• Monitor and optimize Snowflake workloads, ETL/ELT processes, and SQL queries to meet SLAs.
• Analyze warehouse utilization and recommend performance and cost-efficiency improvements.
• Monitor pipelines, diagnose performance issues, and implement long-term solutions; support production environments and incident resolution.
• Maintain comprehensive documentation for pipelines, flows, transformations, data models, and operational processes.
• Collaboration
• Partner with cross-functional teams (data architects, data scientists, analysts, business stakeholders) to understand requirements and provide technical expertise.