Data Engineer

We are hiring a Data Engineer to join a modern cloud data engineering team responsible for building scalable data pipelines, cloud integrations, and enterprise data products. The ideal candidate will have strong expertise in Azure Data Engineering, Azure Databricks Lakehouse, Azure Data Factory (ADF), SQL, Spark, PySpark, and ETL development . Full Stack Engineers from leading technology companies are also encouraged to apply , provided they have strong hands‑on experience in Azure Data Engineering technologies and meet the required data engineering skill set. This role involves designing scalable cloud‑based data solutions, optimizing distributed data processing workloads, and collaborating with architects, analysts, and engineering teams to deliver enterprise‑grade data platforms. Key Responsibilities Design and develop scalable cloud‑based data pipelines. Build ETL/ELT workflows using Azure Data Factory and Databricks. Develop high‑performance PySpark and Spark SQL applications. Design data ingestion, transformation, and integration pipelines. Optimize Spark jobs for performance and scalability. Work with Azure Data Lake Storage and cloud‑based data storage solutions. Develop reusable, scalable, and fault‑tolerant data engineering solutions. Build streaming data pipelines using Kafka/Event Hub technologies. Apply best practices for indexing, partitioning, and query optimization. Participate in Agile development and DevOps processes. Collaborate with architects, analysts, and product teams. Prepare technical documentation and system design documents. Mentor junior engineers and participate in knowledge‑sharing sessions. Identify automation opportunities and improve engineering processes. Support production deployments, debugging, and performance tuning. Design low‑level technical solutions for multiple components. Required Technical Skills SQL PySpark Spark SQL ETL Development Kafka / Event Hub Parquet Data Streaming Performance Optimization Indexing Distributed Computing Idempotency Service‑Oriented Architecture (SOA)

CI/CD

Agile / Scrum Preferred Skills Experience building enterprise‑scale cloud data platforms. Strong understanding of Spark execution plans and DAG optimization. Experience optimizing large‑scale ETL workloads. Knowledge of distributed cloud computing concepts. Experience implementing automated data engineering solutions. Exposure to AI‑enabled or Agentic AI data engineering solutions. Strong documentation and technical communication skills. Experience mentoring junior engineers. Responsibilities Across SDLC Develop clean, reusable, and scalable code. Perform unit testing and code reviews. Debug production issues and perform root cause analysis. Create technical documentation. Execute release and deployment activities. Design Low‑Level Design (LLD) documentation. Follow coding standards and engineering best practices. Ensure quality, scalability, and maintainability of applications. Qualifications Bachelor's degree in Engineering, Computer Science, Information Technology, MCA, BCA, or equivalent. Strong analytical and problem‑solving skills. Experience working in Agile environments. Ability to collaborate with cross‑functional teams. Mandatory Requirements 1–3 years of Data Engineering experience Strong experience with Azure Databricks Lakehouse Hands‑on experience with SQL, PySpark, Spark SQL, and ETL Experience with Azure Data Lake Storage (ADLS) Knowledge of Kafka/Event Hub Experience with Azure DevOps and CI/CD Strong understanding of Indexing, Partitioning, and Performance Optimization Experience with Distributed Computing Concepts (Fault Tolerance, Idempotency, SOA) #J-18808-Ljbffr BrickRed Systems