Azure Databricks Data Engineer

Azure Databricks Data Engineer Location Rartian NJ Onsite Role OverviewWe are looking for a highly motivated Data Engineer with strong expertise in Databricks and modern data engineering practicesThe ideal candidate should have handson experience in building scalable data pipelines notebooks implementing CICD processes and working with Delta Lake technologies The candidate must possess excellent Python programming skills a solid understanding of Unity Catalog and the ability to collaborate effectively within large crossfunctional teamsKey ResponsibilitiesDesign develop and maintain scalable ETLELT pipelines Notebooks using DatabricksBuild and optimize data processing solutions using PySpark and PythonImplement and manage data governance security and access controls using Unity CatalogWork with Delta Lake features including Delta Sharing Time Travel Change Data Feed CDF and data optimization techniquesDevelop and maintain CICD pipelines for Databricks deployments using industrystandard DevOps practicesPerform performance tuning and optimization of Databricks workloadsCollaborate with architects business analysts and stakeholders to understand data requirements and deliver solutionsSupport production deployments troubleshooting monitoring and performance improvementsFollow coding standards documentation and best practices for software development lifecycle SDLCRequired SkillsDatabricks StrongStrong understanding of Databricks architecture and ecosystemHandson experience with Delta Lake Delta Sharing Unity Catalog Structured Streaming Workflow orchestration Performance tuning and optimization Data governance and access managementExperience with Medallion Architecture Bronze Silver Gold layersProgrammingStrong handson coding experience in PythonExperience with PySpark for largescale data processingGood understanding of SQL and query optimizationDevOps CICDExperience implementing CICD processes for Databricks workloadsFamiliarity with Gitbased source control and branching strategiesExperience with deployment automation tools and DevOps pipelinesData Engineering ConceptsETLELT design patternsData modeling and data warehousing conceptsData quality monitoring and observability frameworksGood to Have SkillsAWS Cloud AWS Glue S3 IAM LambdaAzure Cloud Azure Data Factory ADF Azure Storage Azure Databricks Key VaultSoft SkillsExcellent verbal and written communication skillsStrong problemsolving and analytical abilitiesAbility to work effectively in a large global team environmentStrong stakeholder management and collaboration skillsSelfdriven proactive and able to work independently with minimal supervision"