Azure Data Engineer

Azure Data EngineerCary NC – OnsitePay Rate -$65/hr-- $70/hr C2CDesign and build batch and streaming data pipelines using Azure, Databricks, and Kafka.Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or ScalaImplement and support Data Lake / Lakehouse architecturesIntegrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud.Expertise in handling Data migration using API's.Apply data quality checks and support data governance basics (metadata, lineage, access controls)Eligible to optimize Spark jobs for performance, reliability, and costCollaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managersRequired Skills & Experience:8+ years of experience in data engineering in azure using Azure Databricks.Strong experience with Azure Data Lake, Databricks, ADF, Postgres.Strong Data Engineering experience in Big Data (Cloudera)Hands-on expertise in SQL, Python and PySpark or ScalaProven experience with batch & real-time ingestionExperience with Kafka and big data processingAdded good to have experience with:Informatica PowerCenterInformatica IDMC or equivalent cloud data integration toolsSQL Server, Oracle, Azure SQLCosmos DBUnix/LinuxMaestro (or similar enterprise schedulers)Data tools such as Precisely or equivalentDiverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.