Senior Data Engineer

Senior Data EngineerLocation: Atlanta, GAClient- Southern CompanyContract- 1 YearExperience Level: 10+ YearsPosition Summary:We are seeking an experienced Senior Data Engineer with a strong background in designing and developing robust, scalable, and high-performance data infrastructure. The ideal candidate will have 10+ years of experience in manipulating data, working with both relational and NoSQL systems, and building modern data pipelines across data lakes and warehouses. This role demands a deep technical skillset in SQL, data modeling, and data pipeline orchestration tools like Apache Spark, Hive, and Airflow.Key Responsibilities:Design, develop, and maintain scalable ETL/ELT pipelines to process and integrate data from disparate sources.Work with structured and unstructured data to build clean, efficient, and machine-readable datasets.Normalize databases and ensure that data structures align with application and business requirements.Collaborate with business analysts, data scientists, and software engineers to deliver data solutions aligned with company objectives.Design and implement data models that support reporting, analytics, and machine learning use cases.Migrate and optimize workloads in cloud-based environments (e.g., Azure, AWS, GCP).Monitor and optimize data pipeline performance, data quality, and reliability.Implement data governance best practices and ensure compliance with security and privacy regulations.Technical Skills & Expertise:Expert-level proficiency in SQL, including writing complex queries and optimizing performance.Deep understanding of data modeling techniques for both OLTP and OLAP systems.Hands-on experience with big data frameworks: Apache Spark, Hive, Airflow, DatabricksExperience with modern cloud data platforms: Azure Data Lake, Azure Synapse, AWS Redshift, or Google BigQueryStrong experience with data orchestration, scheduling, and automation tools.Familiarity with Python, Scala, or Java for data processing and scripting tasks.Knowledge of NoSQL databases (e.g., MongoDB, Cassandra) and streaming tools (e.g., Kafka) is a plus.Experience with data security, lineage, and cataloging tools (e.g., Unity Catalog, Collibra).Preferred Qualifications:Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.Microsoft Azure Data Engineer Associate or equivalent certifications preferred.Experience in data governance processes and supporting enterprise analytics platforms.Knowledge of containerization (e.g., Docker) and CI/CD pipelines in DevOps environments.