Senior Data Engineer - Identity Resolution & Large-Scale Data Engineering
Sumeru Solutions
Role: Senior Data Engineer Identity Resolution & Large-Scale Data Engineering
Location: Bellevue, WA
Overview:
We are seeking an experienced Senior Data Engineer to design, build, and optimize production-grade data solutions supporting Identity Resolution (IDR) and large-scale customer data initiatives.
The ideal candidate will have proven experience delivering data solutions at billion-record scale, with strong expertise in Azure Databricks, PySpark, Python, Azure Data Factory, and distributed data processing.
The role requires strong production-scale engineering judgment-the ability to identify technical and scalability constraints early, evaluate alternatives quickly, and deliver tested, scalable, and production-ready solutions.
Experience with Identity Resolution or comparable high-throughput matching systems is highly preferred.
Key Responsibilities
Design and optimize "large-scale data processing solutions capable of operating at 1B+ record scale".
Develop scalable data pipelines using Azure Databricks, PySpark, Python, and ADF".
Optimize Spark workloads including large-scale joins, partitioning, shuffling, data skew, memory, and compute utilization.
Design and improve solutions for Identity Resolution, Entity Resolution, Record Linkage, Deduplication, or similar high-volume matching problems.
Evaluate technical approaches against production data volumes and infrastructure constraints.
Identify and communicate scalability and environment constraints early and recommend viable alternatives.
Perform performance and scalability testing using production-like workloads.
Build reliable data pipelines with appropriate validation, error handling, monitoring, and recovery mechanisms.
Collaborate with architects, engineers, data scientists, and stakeholders to deliver solutions from design through production.
Required Qualifications
7+ years of Data Engineering experience.
Proven experience working with very large-scale datasets, preferably 1B+ records.
Strong hands-on experience with Azure Databricks and PySpark.
Strong Python and SQL skills.
Experience with Azure Data Factory (ADF) and Azure data services.
Strong understanding of distributed processing and Spark performance optimization.
Experience delivering production-grade, scalable data solutions.
Ability to identify technical constraints early and make timely technical decisions.
Strong problem-solving and communication skills.
Preferred Qualifications
Experience with Identity Resolution / Entity Resolution / Record Linkage / Deduplication / Fuzzy Matching.
Experience with Cosmos DB and low-latency APIs.
Experience with Azure Functions or Azure Web Apps.
Experience with Event Hub, Kafka, or Spark Structured Streaming.
ML experience related to matching or customer data.
Exposure to Generative AI, LLMs, RAG, or Agentic AI.
Telecommunications or large-scale customer data experience.
- ...work gets done. We are hiring a Senior Software Engineer across our Identity, Data Security and Trust organization. These... ...the most secure platform at scale, and giving customers the tools to... ...designing, building, and operating large-scale distributed systems in production...SeniorFull time
Snowflake
Bellevue, WA1 day ago $130k - $196.5k
LiveRamp is the data collaboration platform of choice... ..., and foundational identity, LiveRamp is setting... ...core identity, entity resolution, and measurement capabilities... ...Analyze and transform large-scale, high-dimensional, and... ...actionable insights, engineer robust features, and...SeniorFull timeWork from homeFlexible hoursNight shift$153.5k - $207.7k
...environment, then come join us. As a Senior Data Engineer, you will share ownership of Redfin’s... ...purposesYou have experience building large scale Data lakesYou can add architectural insights... ..., sexual orientation, gender identity or expression, age, military or veteran...SeniorMinimum wageFull timeLocal area- ...talented Team. Job Title: Senior Data Engineer Location: Seattle, WA... ...solutions for enterprise scale data warehouse and data lake... ...EMR Experience operating large data warehouses Degree... ...sexual orientation, gender identity, national origin, age,...Senior
Ampcus
Seattle, WA5 days ago $200.4k - $260.5k
Senior Machine Learning Engineer, Data InfrastructureUnity Vector builds an Data platform that... ....Our systems operate at scale across batch and streaming... ...platform also supports large-scale model training, feature... ..., disability, sex, gender identity or expression, sexual...SeniorFull timeWork at officeWorldwideRelocation package$124k - $280k
...SummaryAt PwC, our people in data and analytics engineering focus on leveraging... ...future business needs. As a Senior Manager you lead large projects, innovate... ...strategies- Lead large-scale projects and innovate processes... ...orientation, and gender identity); age; disability;...SeniorFull timeH1b- ...the Role Build the data foundation that... ...systems that operate at scale. At the core of... ...system is Data Engineering. This role sets direction... ...You’ll Do As a Senior Data Engineer, you... ...and evolving large-scale production data... ..., led incident resolution, and driven long-term...Senior
$150k - $200k
...allocated. Our team includes researchers, engineers, and investment professionals who care... ...creation. Responsibilities Build and maintain data pipelines and infrastructures that power... ...ingesting, processing, and analyzing large-scale datasets Implement ETL workflows and...SeniorFull timeRemote work- ...DoorDash success. The Data Engineering team builds database solutions... ...is looking for a Senior Data Engineer to be a... ...powerhouse to help us scale our data... ...develop and implement large scale, high volume, high... ...sexual orientation, gender identity or expression, disability...SeniorHourly payWork at officeLocal areaRemote workFlexible hours
$182k - $242k
...enables innovators to build and scale AI with confidence. Trusted... ...at What You'll Do: The Data Engineering Team builds and operates the... ...the role: We're seeking a Senior Data Engineer to design and... ...sexual orientation, gender identity, national origin, veteran status...SeniorPermanent employmentFull timeTemporary workCasual workWork at officeFlexible hoursCoreWeave
Bellevue, WA8 days ago$190.8k - $286.8k
...and Amsterdam. Making data‑driven decisions is key... ...support that, we need to scale our data systems while... ...to teams across engineering, product, and business... ...data pipelines on top of large datasets (in the order... ...orientation, gender, gender identity, gender expression, transgender...SeniorLocal areaFlexible hours$198k - $297k
..., and happy. The Role As a Data Engineer you will handle everything from... ...the foundation of our large and unique datasets that span... ...critical domains including identity, account access, eligibility... ...positive reduction and automated resolution. Participate in on‑call...SeniorWork at officeLocal areaRemote workFlexible hours$128k - $161k
...the world. At DigitalOcean, Data Center Engineers play a critical role in... ...deploying, maintaining, and scaling the servers and networking equipment... ...opportunities to work on large‑scale deployments, new data... ...orientation, gender, gender identity or expression, age, disability...SeniorLocal areaRemote workWorldwideFlexible hours$124.9k - $228.9k
...businesses responsibly. The scale of our platform brings... ...Technology Operations group in Engineering at The Trade Desk and... ...well‑governed enterprise data solutions. As a Senior Software Engineer - Data Engineer... ...orientation, gender identity or expression, veteran status...SeniorFull timeTemporary workWork experience placementLocal areaWorldwide- ...member of PNNL’s Software Engineering and Architectures (SEA)... ...software, scalable data architecture & pipelines... ...entry [Link] Develop large data pipelines to thwart... ...grid to prevent large‑scale outages [Link] Optimize... ...people of all racial/ethnic identities, women, veterans, and...SeniorWork experience placementLocal areaRemote work
- A leading infrastructure consulting firm is seeking a Senior Drainage Lead in Seattle, WA. The role involves overseeing drainage design for large scale projects, managing teams, and ensuring compliance with relevant regulations. The ideal candidate will have extensive experience...Senior
$152.2k - $243.7k
...opportunity to create impact at scale — tackling meaningful... ...entrepreneurs and engineers in 2016, Pismo is a... ...technological challenges that large banks, marketplaces,... ...and evolution of the data platform, ensuring reliability... ...orientation, gender identity, disability or protect...SeniorFull timeWork experience placementWork at officeLocal area$128.7k - $214.6k
...team of builders inside our Data Analytics organization. Our goal... ...the role As a Lead Analytics Engineer, you'll design and build the... ...and unstructured data at scale (object storage, data lakes).... ...celebrate diverse experiences, identities, and perspectives, because lifting...SeniorFull timeImmediate startRemote workWork from homeFlexible hours- ...Corporation in Bothell, WA seeks an experienced Senior Preconstruction Project Manager to lead preconstruction activities for large-scale commercial construction projects ranging... ..., coordinating with owners, designers, engineers, and subcontractors to develop accurate...SeniorFor subcontractor
$120k - $160k
Consigli Construction is seeking a Senior Estimator in our Estimating division to lead complex, large-scale projects from concept to GMP. The role requires a BS or MS in Engineering or Construction Management with extensive construction estimating experience, including...SeniorFull time$153k - $204k
...enables innovators to build and scale AI with confidence. Trusted... .... About the role: As a Senior Data Engineer you will own and evolve the... ..., modeling, and querying large datasets. ~ Proficiency in... ...sexual orientation, gender identity, national origin, veteran status...SeniorPermanent employmentFull timeTemporary workCasual workWork at officeFlexible hours$112.8k - $153.7k
...listen to your ideas. This is a senior engineering role. You will architect,... ...continuously improve cloud data and analytics platforms (e.g... ...response: drive triage, own the resolution, run the post-incident... ...command of cloud fundamentals: identity and access, resource...SeniorFull timeContract workLocal areaFlexible hours- ...There is only one Data Cloud. Snowflake’s... ...stop there. They engineered Snowflake to power... ...with near-unlimited scale, concurrency, and... ...tomorrow. The identity & access management... ...engineers. AS A SENIOR SOFTWARE ENGINEER... ...Lead and deliver large projects,...SeniorFull time
Snowflake
Bellevue, WA1 day ago $147k - $202.4k
Secure Every Identity, from AI to HumanIdentity is the key to... ...seeking a highly skilled Senior Site Reliability Engineer to join our team. We are a... ...specializing in securing large-scale systems. This role is a blend... ...tooling for our Snowflake data systems.Automation:...SeniorPermanent employmentWork at officeLocal areaWorldwideFlexible hoursShift work$214k - $302k
Senior Staff Data Center Engineer, Mechanical Engineering $214,000.00/yr - $302,000.00/yr Preferred locations... ...in designing modular building scale assemblies, with an understanding of... ...marital status, disability, gender identity or veteran status. We also consider...SeniorFull timeWorldwide$134.5k - $265.1k
...Join our AI & Engineering team in transforming... ...in software, data, AI, network,... ...they can grow and scale effectively.... ...engagement, issue resolution, and delivery... ...to both large and small audiences... ...level employees to senior leaders, we... ...orientation, gender identity, national...Local areaVisa sponsorship$222.5k - $257.5k
...seamlessly and scale solutions smarter... ....Observability Engineering owns the... ...Observability & Telemetry Data), you will set... ..., and mentor senior and mid-level... ...complex issue resolution and driving root... ...and operating large-scale distributed... ..., gender identity, national origin...Full timeTemporary workWork at officeLocal areaRemote work$172.5k - $260.1k
Job Category Software Engineering Job Details About Salesforce Salesforce... .... We are looking for a Data Engineer who thrives at the intersection... ...partners on patterns that scale (windowing, partitioning,... ..., gender expression or identity, transgender status, age, disability...Senior$147k - $181k
...brighter way forward. About JLL and Data PlatformJLL is a leading... ....What this job involvesAs a Senior Data Engineer on the Data Consumption team,... ...has delivered at least one large or complex data platform initiative... ..., and sharing reliability at scale.Cross-Functional...SeniorFull timeContract workLocal areaRemote work- ...organizations modernize data ecosystems, build real-... ..., and responsibly scale AI. As part of Accenture... ...Enterprise.You AreA hands-on Engineer with foundational... ...with clients, supporting senior architects, and contributing... ...for security, identity management, and compliance...SeniorFull timeWork experience placementLive inWork at officeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer - Identity Resolution & Large-Scale Data Engineering. Be the first to apply!
- data platform engineer Bellevue, WA
- aws data engineer Bellevue, WA
- data science developer Bellevue, WA
- sr information security engineer Bellevue, WA
- senior data engineer Bellevue, WA
- data engineer machine learning Bellevue, WA
- junior data engineer remote Bellevue, WA
- data center engineer Bellevue, WA
- data developer Bellevue, WA
- senior data center engineer Bellevue, WA