Senior Data Engineer

Data Engineering Lead

SymphonyAI is at the forefront of enterprise AI solutions, delivering cutting-edge technology that fosters transformative outcomes for businesses. Our expertise in advanced AI and data analytics empowers organizations across a wide range of industries. The Customer Insights Professional Services team at SymphonyAI is committed to aiding clients in leveraging data integration, analytics, and insights to attain their strategic objectives. We collaborate intimately with our clients to provide solutions that enhance decision-making processes and drive business success.

Job Summary

We are seeking a lead professional with 7-8 years of experience in Data Engineering. The ideal candidate will possess a comprehensive skill set, including proficiency in SQL, data warehousing, Azure Data Factory, Kubernetes, Python, shell scripting, Azure pipelines, and Snowflake or similar technologies. Responsibilities include collaborating with Development, QA, and other stakeholders to manage multiple projects simultaneously.

About the Role

• Data Integration and Management: Lead initiatives focused on integrating, transforming, and managing large datasets from diverse sources, ensuring optimal availability and performance.
• Advanced SQL Queries: Analyze complex queries to resolve data issues; write ETL code to meet project requirements.
• Azure Data Factory: Leverage Azure Data Factory to orchestrate data workflows and pipelines, implementing best practices for cloud-based integration and processing.
• Azure Pipelines Deployment: Manage CI/CD pipelines using Azure Pipelines, automating and streamlining deployment processes.
• Kubernetes Deployment: Oversee deployment and management of data solutions within Kubernetes environments, ensuring application scalability, reliability, and security.
• Python and Shell Scripting: Develop custom scripts in Python and shell for tasks related to data processing, automation, and system integration.
• Project Leadership: Lead multiple projects simultaneously, coordinating among development, QA, and business teams; provide technical leadership and mentorship to junior team members.
• Requirement Analysis: Work with stakeholders to understand business requirements and design and execute builds to fulfill those needs.
• Configuration and Execution: Configure systems according to project specifications and execute data builds efficiently.
• API Configuration: Understand and configure APIs using Python code.

About You

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 7 or more years of experience in data engineering with a strong background in technical leadership.
• Proven expertise in SQL, data warehousing, and cloud environments (preferably Microsoft Azure).
• Hands-on experience with Azure Data Factory, Azure pipelines, Kubernetes, Python, shell scripting, and Snowflake or similar technologies.
• Experience handling extensive datasets and developing ETL code to import data into a datamart.
• Demonstrated ability to collaborate with cross-functional teams in a dynamic setting.
• Strong problem-solving and analytical capabilities.
• Retail domain experience is a plus.
• Excellent communication and interpersonal skills.

What We Offer

• Competitive salary and benefits package.
• Flexible hybrid working model.
• Opportunities for professional growth and development.
• Collaborative and inclusive work environment.
• Access to the latest technologies and tools.
• Opportunity to make a tangible impact on cutting-edge Retail/ CPG AI solutions.