Senior Data Engineer

Azure – Data Factory, Databricks, Synapse, ADLS Senior Data Engineer – 8+ Years Experience Job Type: Full-Time Work Model: Hybrid Experience: 8+ Years Employment: Full-Time Job Summary We are looking for an experienced Senior Data Engineer with 8+ years of hands‑on experience designing, developing, and maintaining scalable data platforms and data pipelines. The ideal candidate will have strong expertise in Python, SQL, ETL/ELT, cloud platforms, data warehousing, and distributed data processing . The candidate will work closely with Data Scientists, Software Engineers, Business Analysts, and other stakeholders to build reliable, high-performance data solutions and enable data-driven decision‑making. Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines. Build and optimize data processing workflows using Python and SQL. Develop data ingestion solutions from databases, APIs, files, and other sources. Design and maintain enterprise data warehouses, data lakes, and lakehouse architectures. Work with large datasets using distributed processing technologies such as Apache Spark/PySpark. Develop data models and optimize complex SQL queries for performance. Implement data quality, validation, monitoring, and error‑handling frameworks. Integrate data from structured and unstructured sources. Build and maintain batch and real‑time/streaming data pipelines. Work with cloud data platforms such as AWS, Azure, or GCP. Implement CI/CD and DevOps practices for data engineering workflows. Collaborate with cross‑functional teams to understand business and technical requirements. Troubleshoot pipeline failures, performance issues, and data quality problems. Ensure data security, governance, scalability, and compliance. Mentor junior and mid‑level data engineers and contribute to technical architecture decisions. Document data pipelines, architectures, processes, and technical solutions. Required Skills 8+ years of professional experience in Data Engineering or related roles. Strong programming experience with Python. Advanced SQL skills with experience in query optimization. Strong experience with ETL/ELT development and data pipeline architecture. Hands‑on experience with Apache Spark / PySpark. Experience with cloud platforms such as: AWS – S3, Glue, Redshift, EMR, Lambda Azure – Data Factory, Databricks, Synapse, ADLS GCP – BigQuery, Dataflow, Cloud Storage Strong understanding of data warehousing and dimensional data modeling. Experience with relational databases such as SQL Server, PostgreSQL, Oracle, or MySQL. Experience with Databricks and/or Snowflake is highly desirable. Knowledge of Apache Kafka or other streaming technologies. Experience with workflow orchestration tools such as Airflow, Azure Data Factory, or AWS Glue. Familiarity with Git, CI/CD, Docker, and DevOps practices. Strong understanding of data quality, governance, security, and metadata management. Preferred Qualifications Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field. Experience designing enterprise-scale data architectures. Experience with real‑time data processing and streaming. Experience with Terraform or Infrastructure as Code. Familiarity with Kubernetes and containerized workloads. Experience working in Agile/Scrum environments. Strong communication, analytical, and problem‑solving skills. Technical Environment Languages: Python, SQL, Scala/Java Big Data: Apache Spark, PySpark, Hadoop Cloud: AWS / Azure / GCP Data Warehousing: Snowflake, Redshift, Synapse, BigQuery Databases: SQL Server, PostgreSQL, Oracle, MySQL ETL/ELT: Databricks, AWS Glue, Azure Data Factory Streaming: Kafka, Spark Streaming Orchestration: Apache Airflow DevOps: Git, Jenkins/GitHub Actions, Docker, CI/CD Methodology: Agile/Scrum Work Arrangement This is a Full-Time Hybrid position. The selected candidate will be expected to work both remotely and from the designated office location based on business requirements. What We’re Looking For We are seeking a senior‑level engineer who can independently own data engineering projects, contribute to architecture and design decisions, build production‑grade pipelines, and collaborate effectively with technical and business teams. Skills: data,pipelines,sql,platforms,aws,processing,cloud,etl,azure,apache #J-18808-Ljbffr 7Th Sky Tech