Data Engineer

Data Engineer

We are seeking an experienced Data Engineer with over 5 years of expertise in designing, building, and optimizing data pipelines and architectures. The ideal candidate will be skilled in using Azure Data Factory, Databricks, SQL databases, and data modeling to support scalable and efficient data solutions. This role is integral to enabling data-driven decision-making across the organization.

Key Responsibilities

• Data Pipeline Development: Design, build, and maintain scalable data pipelines using Azure Data Factory and Databricks.
• Data Integration: Integrate data from multiple sources into a centralized repository while ensuring data accuracy and reliability.
• Data Modeling: Create and optimize data models (conceptual, logical, and physical) to meet reporting and analytics needs.
• Database Management: Develop and maintain SQL databases, including writing complex queries, stored procedures, and performance tuning.
• Collaboration: Partner with data analysts, business stakeholders to understand requirements and deliver high-quality data solutions.
• Data Quality: Implement data validation, monitoring, and error-handling mechanisms to ensure data integrity.
• Optimization: Continuously optimize data architectures and pipelines for performance, cost-efficiency, and scalability.
• Documentation: Maintain comprehensive documentation for data pipelines, models, and systems.

Qualifications

Required Skills & Experience

• 5+ years of experience as a Data Engineer or in a related role.
• Expertise in Azure Data Factory and Databricks for ETL processes and data transformation.
• Strong proficiency in SQL and experience with managing and optimizing SQL databases.
• Hands-on experience with data modeling for analytics and reporting.
• Solid understanding of distributed systems and data architecture principles.
• Proven ability to manage and deliver data engineering projects independently and collaboratively.

Preferred Skills

• Experience with other Azure services like Azure Synapse Analytics, Azure Data Lake, or Azure Functions.
• Knowledge of Python or Scala for data engineering tasks.
• Familiarity with CI/CD pipelines for data workflows.
• Understanding of cloud cost management and optimization.

Key Competencies

• Problem-solving mindset with attention to detail.
• Excellent communication and collaboration skills.
• Proactive, with a strong sense of ownership and accountability.
• Ability to thrive in a fast-paced and dynamic environment.

Education

• Bachelor's degree in Computer Science, Data Science, Information Systems, or a related field. (Master's degree preferred but not required.)