Airflow Engineer - Data Platform, Apache Airflow

Role: Senior Data Platform / Airflow Engineer

Location: Plano, TX

Duration: Fulltime

About Smart IT Frame:

At Smart IT Frame , we connect top talent with leading organizations across the USA. With over a decade of staffing excellence, we specialize in IT, healthcare, and professional roles, empowering both clients and candidates to grow together.

Job description

• Design, build, and operate large-scale data platforms powered by Apache Airflow
• Manage and optimize hundreds to thousands of DAGs with high task throughput
• Drive platform reliability, scalability, and performance improvements
• Own end-to-end lifecycle: deployment, monitoring, incident response, and roadmap execution
• Collaborate with cross-functional teams to enable robust, secure, and efficient data workflows
• Strong hands-on experience with DAG design, testing & idempotency
• Deep knowledge of sensors, deferrable operators, dynamic task mapping, SLAs, retries, backfills
• Experience with cross-DAG dependencies, task groups, datasets, pools, and queues
• Experience running Airflow on Kubernetes (Helm, autoscaling, tuning)
• Understanding of network policies, PDBs, and deployment strategies (blue-green / canary)
• Automation-first mindset with Terraform, Helm, GitOps (Argo CD / Flux), CI/CD
• Strong Python skills for DAGs, operators, and utilities
• Familiarity with Bash; exposure to Go or Java is a plus
• Hands-on with Prometheus, Grafana, StatsD, and logging systems
• Experience with capacity planning, performance tuning, and reliability practices
• Experience with Azure (preferred) or any major cloud
• Exposure to Snowflake, BigQuery, Redshift, Databricks, Spark, EMR/Dataproc
• Solid understanding of IAM, VPC networking, and storage (S3/GCS/ADLS)
• Knowledge of SSO/OIDC, RBAC, secrets management (Vault/Secrets Manager)
• Strong focus on auditing, least privilege, and governance

Mandatory Skills :

Airflow advanced DAG design dynamic task mapping deferrable operators SLAs backfills crossDAG dependencies Kubernetes Helm autoscaling podnode tuning network policies PDBs canarybluegreen deployments Python operatorshooks development Terraform GitOps Argo CDFlux CICD Observability Prometheus Grafana logging ing Cloud Data Platforms AWSAzureGCP SnowflakeBigQueryRedshift SparkDatabricks EMRDataproc Security IAM RBAC OIDCSSO VaultSecrets Manager SRE Incident Management Bash scripting

Apply today or share profiles at View email address on click.appcast.io