Azure Data Engineer

Job Description

Role Descriptions:

\n

\n

\n
• Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.
\n
• Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or Scala
\n
• Implement and support Data Lake / Lakehouse architectures
\n
• Integrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.
\n
• Expertise in handling Data migration using API’s.
\n
• Apply data quality checks and support data governance basics (metadata, lineage, access controls)
\n
• Optimize Spark jobs for performance, reliability, and cost
\n
• Collaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managers
\n

\n

\n

Required Skills & Experience:

\n

\n
• 8+ years of experience in data engineering in azure using Azure Databricks.
\n
• Strong experience with Azure Data Lake, Databricks, ADF, Postgres.
\n
• Strong Data Engineering experience in Big Data (Cloudera)
\n
• Hands-on expertise in SQL, Python and PySpark or Scala
\n
• Proven experience with batch & real-time ingestion
\n
• Experience with Kafka and big data processing
\n

\n

\n

Added good to have experience with:

\n

\n
• Informatica PowerCenter
\n
• Informatica IDMC or equivalent cloud data integration tools
\n
• SQL Server, Oracle, Azure SQL
\n
• Cosmos DB
\n
• Unix/Linux
\n
• Maestro (or similar enterprise schedulers)
\n
• Data tools such as Precisely or equivalent
\n

\n

\n

Added Advantage

\n

\n
• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
\n
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD
\n
• Practical usage of Copilot for data engineering and development productivity
\n
• Exposure to Generative AI / GenAI use cases in data platforms and analytics
\n

\n