Senior Data Engineer

Job Title Senior Data Engineer (W2 Position) Experience 12+ Years Employment Type W2 Only Location San Antonio, TX, USA Duration Long-Term Contract Job Summary We are seeking a highly experienced Senior Data Engineer with 12+ years of IT experience in designing, building, and optimizing scalable data platforms and modern data pipelines. The ideal candidate will possess strong expertise in Python, SQL, Spark, Databricks, Snowflake, AWS, ETL/ELT, and cloud-native data engineering solutions. This role involves developing high-performance data architectures, implementing data governance standards, and enabling advanced analytics and business intelligence initiatives. Key Responsibilities Design, develop, and maintain scalable data pipelines for batch and real-time data processing. Build enterprise-grade ETL/ELT workflows using modern cloud technologies. Develop and optimize large-scale data solutions using Python, SQL, Spark, and Databricks. Design and implement data lakes, data warehouses, and Lakehouse architectures. Build robust data models to support analytics, reporting, and machine learning workloads. Integrate data from multiple structured, semi-structured, and unstructured sources. Optimize SQL queries, Spark jobs, and overall pipeline performance. Develop scalable data ingestion frameworks using Kafka or other streaming platforms. Implement Infrastructure as Code (IaC) for data platform deployment and automation. Collaborate with Data Scientists, BI Developers, Analysts, Architects, and business stakeholders. Ensure data quality, lineage, governance, security, and compliance across enterprise data platforms. Implement monitoring, logging, alerting, and troubleshooting for production data pipelines. Support CI/CD automation for data engineering deployments. Perform root cause analysis and resolve production issues. Mentor junior data engineers and establish engineering best practices. Required Qualifications Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or related field. 12+ years of overall IT experience. 8+ years of hands‑on Data Engineering experience. Strong programming expertise in Python and advanced SQL. Extensive experience with Apache Spark and PySpark. Hands‑on experience with Databricks. Strong expertise in Snowflake data warehouse. Experience with cloud platforms, preferably AWS. Strong experience building ETL/ELT pipelines. Experience with Airflow or similar workflow orchestration tools. Hands‑on experience with Git and CI/CD pipelines. Strong understanding of data modeling, data warehousing, and dimensional modeling. Experience working in Agile/Scrum environments. Excellent analytical, troubleshooting, and communication skills. Programming Languages Python SQL PySpark Scala (Preferred) Shell Scripting Big Data Technologies Apache Spark PySpark Hadoop Ecosystem Hive Kafka Delta Lake Cloud Platforms AWS (S3, EMR, Glue, Lambda, Redshift, Athena, EC2, IAM, CloudWatch) Azure (Preferred) Google Cloud Platform (Preferred) Data Platforms Databricks Snowflake Redshift Azure Synapse Analytics BigQuery (Preferred) ETL/ELT & Orchestration Apache Airflow AWS Glue Informatica Talend dbt Azure Data Factory (ADF) Databases SQL Server PostgreSQL Oracle MySQL MongoDB DynamoDB Streaming Technologies Apache Kafka Kafka Connect Amazon Kinesis Data Warehousing Star Schema Snowflake Schema Dimensional Modeling Data Vault Lakehouse Architecture DevOps & CI/CD Git GitHub GitLab Jenkins Azure DevOps Docker Kubernetes (Preferred) Terraform (Preferred) Monitoring & Logging CloudWatch Datadog Splunk Prometheus Grafana Data Governance & Security Data Lineage Data Quality Data Catalog IAM Encryption RBAC GDPR HIPAA CCPA Methodologies Agile Scrum DevOps DataOps Preferred Qualifications AWS Certified Data Engineer – Associate. AWS Certified Solutions Architect. Databricks Certified Data Engineer Professional. Snowflake SnowPro Certification. Experience with Lakehouse architecture and Delta Lake. Experience supporting AI/ML data pipelines. Experience with real-time event‑driven architectures. Knowledge of Terraform and Infrastructure as Code. Exposure to MLOps and feature engineering pipelines. Soft Skills Excellent verbal and written communication skills. Strong analytical and problem‑solving abilities. Ability to lead data engineering initiatives and mentor team members. Strong stakeholder management and cross‑functional collaboration skills. Ability to work independently in fast‑paced enterprise environments. Strong attention to detail and commitment to delivering high‑quality data solutions. #J-18808-Ljbffr