Lead Data Engineer
United IT
Data Architect
6-8 years of IT experience focusing on enterprise data architecture and management.
Experience in conceptual/logical/physical data modeling & expertise in relational and dimensional data modeling
Experience with Databricks & on-prem, structured streaming, delta lake concepts, and delta live tables required
Experience with Spark scala
Data lake concepts such as time travel and schema evolution and optimization
Structured streaming and delta live tables with Databricks a bonus
Experience leading and architecting enterprise-wide initiatives specifically system integration, data migration, transformation, data warehouse build, data mart build, and data lakes implementation/support
Advanced level understanding of streaming data pipelines and how they differ from batch systems
Formalize concepts of how to handle late data, defining windows, and data freshness
Advanced understanding of ETL and ELT and ETL/ELT tools such as Data Migration Service etc
Understanding of concepts and implementation strategies for different incremental data loads such as tumbling window, sliding window, high watermark, etc.
Familiarity and/or expertise with Great Expectations or other data quality/data validation frameworks a bonus
Familiarity with concepts such as late data, defining windows, and how window definitions impact data freshness
Advanced level SQL experience (joins, aggregation, windowing functions, common table expressions, RDBMS schema design performance optimization)
Indexing and partitioning strategy experience
Debug, troubleshoot, design and implement solutions to complex technical issues
Experience with large-scale, high-performance enterprise big data application deployment and solution
Architecture experience in AWS environment a bonus
Familiarity working with Lambda specifically with how to push and pull data, how to use AWS tools to view data for processing massive data at scale a bonus
Experience with Gitlabs and CloudWatch and ability to write and maintain gitlabs for supporting CI/CD pipelines
Experience working with AWS Lambdas for configuration and optimization and experience with S3
Familiarity with Schema Registry, message formats such as Avro, ORC, etc.
- ...divh2Lead Data Engineer-DE/h2pLocation: Scottsdale AZ (day 1 onsite)/ppDuration: Fulltime/ppJob Description:/ppMust have skill set: Java, Scala, Python, Spark, S3, Glue, Redshift/pulliYou have 6-8 years of relevant software development experience./liliYou have hands-on...SuggestedFull time
Zortech Solutions
Scottsdale, AZ4 days ago - ...Role Overview The Lead Data Engineer will serve as a technical leader responsible for defining data architecture, leading the development of scalable and secure data platforms, and driving enterprise-wide data engineering best practices. This role combines hands...Suggested
VBeyond
Phoenix, AZ1 day ago - ...Lead Data Engineer-DE Location: Scottsdale AZ (day 1 onsite) Duration: Fulltime Job Description: Must have skill set: Java, Scala, Python, Spark, S3, Glue, Redshift You have 6-8 years of relevant software development experience. You have hands-on...SuggestedFull time
Zortech Solutions
Scottsdale, AZ22 hours ago $117.6k - $161.7k
...Become a part of our caring community This role leads the architecture, engineering, and operationalization of the centralized Finance reporting data platform supporting CenterWell Finance reporting, analytics, and month-end close processes. We are looking for deep...SuggestedBi-weekly payFull timeTemporary workApprenticeshipWork at officeRemote workWork from homeHome officeHumana
Phoenix, AZ22 hours ago- ...Jersey Hired is scouting for a heavy-hitting Senior Data Engineer to join a global consulting powerhouse. This isn't just about moving data from Point A to Point B, it's about architecting a fortress. We need a pro who can design, build, and operate secure, audited...Suggested
Jersey Hired
Phoenix, AZ22 hours ago - ...Lead Data Engineer The CVS Health Rebates team is looking for a Lead Data Engineer to assist us in solutioning architecture as we transform the Rebate Processing System from its legacy platform. This person will assist our software engineers in rearchitecting, transforming...
Samprasoft
Scottsdale, AZ2 days ago - ...Principal Engineer I - Lead Data Engineer As a Principal Data Engineer, you will serve as a Subject Matter Expert in the Enterprise Data & Analytics domain, as well as in adjacent areas, ensuring that data engineering solutions are safe, secure, compliant, and reliable...
Western Alliance Bank
Phoenix, AZ4 days ago - ...Job Title: Principal Engineer I - Lead Data Engineer Location: Block 23 What you'll do: As a Principal Data Engineer, you will serve as a Subject Matter Expert in the Enterprise Data & Analytics domain, as well as in adjacent areas, ensuring that...
Western Alliance Bank
Phoenix, AZ3 days ago $150k - $190k
...Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology. Headquartered in South... ...0,000 per year MUST HAVE: 5+ years of experience in data engineering within fast-paced, cloud-native environments Deep...Remote work- Humana Inc in Phoenix, Arizona, is seeking a Lead Data Engineer to enhance data engineering solutions impacting organizational initiatives. This individual will play a key role in architecting and modernizing the Part D and PDP data platform to improve its reliability...Remote job
$142.3k - $195.7k
...Become a part of our caring community The Lead Data Engineer oversees data processing architectures and solutions ensuring the efficient conversion of structured and unstructured data to insights at enterprise scale. You will work on problems of diverse scope and complexity...Full timeTemporary workApprenticeshipRemote workCenterWell
Phoenix, AZ1 day ago- A leading engineering firm in Arizona is seeking an experienced Data Center Design Engineer to lead the low-voltage infrastructure design for new greenfield data center buildouts. The ideal candidate should have at least 5 years of data center low-voltage design experience...
- ...Senior Data Architect with strong experience in Google Cloud Platforms What are we looking for: The Data Architect... ...partners to define and implement defined roadmaps. Leads engineering teams on specific projects or across multiple programs to achieve...Flexible hours
3B Staffing LLC
Phoenix, AZ1 day ago $135k - $150k
...Resource Innovations is seeking a Lead Java Software and Data Integration Engineer to join our growing Software as a Service (SaaS) team. As a hands-on technical lead at Resource Innovations, you will be instrumental in the design, development and deployment of innovative...Work experience placementWork at officeLocal areaRemote workResource Innovations
Phoenix, AZ4 days ago$135k - $150k
A women-led energy transformation firm is seeking a Lead Java Software and Data Integration Engineer in the United States. The role involves designing and developing cloud-based solutions for the energy sector. Key responsibilities include leading data integration efforts...Remote workResource Innovations
Phoenix, AZ14 days ago- ...Lead Data Platform Engineer Virtuous is evolving its data platform into an AI-ready foundation that powers trusted decision-making and self-service analytics across the company. We're hiring a Lead Data Platform Engineer to design, build, and own the systems that...Immediate startFlexible hours
Virtuous
Phoenix, AZ8 days ago $150k - $209k
...leadership role in scheduling for a portfolio of Data Center projects of strategic importance to Meta. The Schedule lead will act as a technical Schedule subject... ...Finance, Accounting, Sourcing and Operations Engineering. Responsibilities Responsible for end to end...Contract workFor contractorsLocal areaHome office- Ascent360 is hiring a Lead Data Platform Engineer based in Phoenix, AZ, who will be responsible for designing and managing their data platform that supports AI readiness. This role requires at least 7 years of experience in data engineering, with strong skills in Snowflake...Remote job
- A company focused on global generosity is seeking a Lead Data Platform Engineer to design and build systems that power their data ecosystem. The ideal candidate will have extensive experience with Snowflake and dbt, ensuring data accuracy and governance while enabling AI...
- A leading staffing agency is seeking a Principal Data Engineer in Phoenix, AZ. This high-impact role involves leading the development of scalable data solutions focused on compliance for large financial institutions. You will work closely with cross-functional teams and...Full time
- A leading engineering firm in Phoenix seeks a Project Manager, Engineering - Mission Critical to lead multidisciplinary teams delivering technical design packages for data center projects. The ideal candidate will have extensive experience in engineering design, especially...
- ...Machine Learning Engineer We are looking to add a Machine Learning Engineer to the team that will execute on approved deliverables, while also working with business users across Commercial Services to identify and create new POCS for solutions down the line. This person...
Software Technology Inc
Phoenix, AZ16 days ago $100k - $120k
...Sr Data Engineer Location - Phoenix, AZ Fulltime/FTE $100000 to $120000/Annum Job Description Role - Sr Data Engineer Experience Required - 6+ Years Must Have Technical/Functional Skills Strong hands-on experience in Big Data technologies and distributed...Full timeDiverse Lynx
Phoenix, AZ8 days ago$110k - $145k
...Senior Data Engineer, Solutions Architecture Scottsdale, Arizona, United States What The Role Is We are seeking a talented Senior... ...SLA monitoring, and incident response CI/CD & DevOps Lead continuous integration and deployment initiatives for Dagster...H1bWork at officeVisa sponsorshipClearway Energy, Inc.
Scottsdale, AZ4 days ago- ...Gke Data Engineering Lead Our client is in need of a GKE Data Engineering Lead for a 2 year engagement to assist in solutioning the architecture as we transform the rebate processing system from legacy platform. The potential candidate will be helping the engineers...
Samprasoft
Scottsdale, AZ2 days ago - ...Big Data/Hadoop Developer Mandatory Technical Skills Skills: PIG, Mapreduce, HDFS, HBase, Hive, YARN, SPARK, Impala, Kafka, Oozie, Java and shell scripting Strong hands on experience with Java, web-services and APIs Strong understanding of hadoop fundamentals Strong...Full timeWork experience placementH1b
SonSoft
Phoenix, AZ22 hours ago - ...third-party agencies, recruiters, or staffing firms will be considered for this position. Role Summary We are seeking a senior Data Engineer to design and build enterprise-scale data platforms that enable AI/ML and agentic systems. This role focuses on engineering AI-ready...Full timeVisa sponsorship
LumiJobs
Phoenix, AZ2 days ago $165k - $190k
...Senior Data Engineer Location: Remote Salary: $165,000 - $190,000 depending on experience Reports To: CTO About Mashura Mashura is building modern software and connected device solutions that support medical dispensing workflows across veterinary and dental practices....Local areaRemote workCubex
Scottsdale, AZ4 days ago- ...Senior Data Engineer We are seeking a senior data engineer with 4-8 years of experience, specializing in Spark and CoreJava. The ideal candidate will leverage technologies to deliver innovative mapping solutions, contributing significantly to our project outcomes and...
Keylent Inc
Phoenix, AZ4 days ago - ...Data Center Engineer The engineer in this position should have a wealth of experience engineering/installing datacenter equipment. The engineer... ..." such as field engineers, installation supervisors, lead installers, and installation techs. Guides detail engineer through...For contractorsWork at officeLocal area
ADEX
Phoenix, AZ2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!
- data coordinator Scottsdale, AZ
- master data coordinator Scottsdale, AZ
- data officer Scottsdale, AZ
- data technician Scottsdale, AZ
- test data management Scottsdale, AZ
- data governance Scottsdale, AZ
- data cabling Scottsdale, AZ
- data internship Scottsdale, AZ
- clinical data Scottsdale, AZ
- data intern Scottsdale, AZ