Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Software Engineer II - Streaming and Durability

$176k - $196.1k

Compass

At Compass, our mission is to help everyone find their place in the world. Founded in 2012, we’re revolutionizing the real estate industry with our end-to-end platform that empowers residential real estate agents to deliver exceptional service to seller and buyer clients.

About the Role

 

Every transaction, every listing view, every agent interaction at Compass touches a streaming pipeline, a database, or a cache owned by the Streaming & Durability team. This team is responsible for the foundational data layer: the systems that have to be fast, correct, and isolated across thousands of tenants, every single time.

 

We don't just keep the lights on. We're solving hard problems: how do you run Kafka as a true platform, with clear authentication, managed schemas, and standard tooling, when it has run for years without a dedicated owner? How do you make Kafka upgrades, schema changes, and connector rollouts routine instead of risky? How do you enforce tenant-level data isolation across shared PostgreSQL clusters and Redis/Valkey caches without sacrificing latency? How do you build SDKs that make the safe path the easy path for hundreds of engineers who never think about infrastructure?

 

As a Senior Engineer, you'll own answers to those questions. You'll design systems that quietly protect millions of records, ship tooling that engineers across the company reach for by default, and operate infrastructure where the cost of getting it wrong is measured in trust, not just latency.

 

Responsibilities

 

- Own Kafka at Compass end to end, across our MSK and Confluent clusters: platform health, upgrades, capacity, and cost. Set the technical direction for how the rest of engineering uses streaming.

- Run the Kafka operational program: client authentication and access standards, Schema Registry ownership with PR-driven schema registration, Kafka Connect upgrades and standard management, producer compression defaults, and the runbooks and drills that let application teams recover quickly.

- Build and scale Kafka/MSK streaming infrastructure, the connective tissue between Compass services, including topic architecture, consumer patterns, partition rebalancing, upgrades, and dead-letter-queue and redrive guidance.

- Build multi-tenant data isolation into Compass's data layer, shipping SDKs in Go, Java, and Python that enforce tenant-safe access patterns so product engineers can move fast without risking cross-tenant data exposure.

- Support the database and caching fleet (RDS/Aurora PostgreSQL, ElastiCache Redis/ValKey, DynamoDB) alongside our database platform partners, including replication, failover, and capacity decisions.

- Drive production database upgrades at fleet scale, moving toward automation, with Terraform state changes across hundreds of clusters and zero customer-facing impact.

- Develop deep observability into the data layer and streaming platform, including SLI dashboards, tenant-dimensioned telemetry, and real-time performance insights that let the team (and the company) understand system behavior before customers do.

- Partner directly with product and platform teams to assess database readiness for scaling events, migrate workloads onto golden-path infrastructure, and resolve the hardest production incidents in the stack.

 

Basic Qualifications

 

- You've operated Kafka/MSK at scale and dealt with the hard parts: upgrades, consumer lag, partition rebalancing, and authentication (SASL/SCRAM or IAM).

- You know the surrounding Kafka ecosystem (Schema Registry, Kafka Connect) well enough to run it and to set standards for other teams.

- 7+ years of experience building and operating distributed systems at production scale in AWS.

- Working knowledge of a major managed database (PostgreSQL or similar), enough to partner with a dedicated database team and reason about the infrastructure around it.

- You've built shared tooling (SDKs, client libraries, platform APIs), and you understand what it takes to make infrastructure accessible and safe for engineers who aren't infrastructure specialists. Kafka client libraries are a plus.

- You think in trade-offs, not absolutes. You can articulate why you'd choose a soft application-enforced boundary over a hard infrastructure lock and when that calculus changes.

- You write Go (or can ramp quickly), with Java a plus, and you're comfortable in Terraform, Kubernetes, and IAM-heavy AWS environments.

- You've led cross-team operational programs, including change management, migration coordination, and rollback planning, where the blast radius of a mistake extends well beyond your own team.

- You can drive a technical decision end to end: frame the options, prove them out with benchmarks or a proof of concept, and lay out cost and risk for senior leaders.

 

Preferred Qualifications

 

- You've worked on multi-tenant isolation problems such as key prefixing, row-level security, and RBAC/ACL models and have opinions about where application-layer enforcement breaks down.

- You've built or contributed to observability platforms, specifically for database and cache monitoring.

- You've worked on Kafka cost and efficiency, such as compression, retention, and cross-AZ data transfer.

 

Compensation:  The base pay range for this position is $176,000-$196,100; however, base pay offered may vary depending on job-related knowledge, skills, and experience. Bonuses and restricted stock units may be provided as part of the compensation package, in addition to a full range of benefits. Base pay is based on market location. Minimum wage for the position will always be met.

 

Perks that You Need to Know About:

Participation in our incentive programs (which may include eligible cash, equity, or commissions). Plus paid vacation, holidays, sick time, parental leave, and recharge leave; medical, tele-health, dental and vision benefits; 401(k) plan; flexible spending accounts (FSAs); commuter program; life and disability insurance; Maven (a support system for new parents); Carrot (fertility benefits); UrbanSitter (caregiver referral network); Employee Assistance Program; and pet insurance.

 

Do your best work, be your authentic self.

At Compass, we believe that everyone deserves to find their place in the world — a place where they feel like they belong, where they can be their authentic selves, where they can thrive.  Our collaborative, energetic culture is grounded in our Compass Entrepreneurship Principles and our commitment to diversity, equity, inclusion, growth and mobility. As an equal opportunity employer, we offer competitive compensation packages, robust benefits and professional growth opportunities aimed at helping to improve our employees' lives and careers.

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer II - Streaming and Durability in Manhattan, KS vacancy
  • $500 per month

     ...investors and advisors About This Role We are hiring a Senior Software Engineer to own the infrastructure behind our product and...  ...infrastructure behind our real-time decision systems, including durable execution, workflow orchestration, reliability, and performance... 
    Senior
    Full time
    H1b
    Work at office
    Local area
    Visa sponsorship
    Night shift

    Cephas Consultancy Services Private Limited

    Manhattan, KS
    7 days ago
  •  ...About the Role: Title: Senior Salesforce Developer Location: Manhattan, NY - 5 Days onsite Required Skills: ~8+ years of Salesforce Developer experience . ~ Hands-on experience with Apex triggers. ~ Hands-on experience with Lightning components. ~ Strong... 
    Senior
    Full time

    PTR Global

    Manhattan, KS
    1 day ago
  • Job Type : Full-Time/Contract (Open to C2C) Requirements: ~9+ years of experience in C# .NET Framework and SQL Server Development. ~ Experience building, integrating, maintaining, and supporting Trading Systems. ~ Strong understanding of systems architecture and...
    Senior
    Full time
    Contract work

    INFOTRON

    Manhattan, KS
    3 days ago
  • $153k - $204k

     ...company (Nasdaq: CRWV) in March 2025. Learn more at  . What You’ll Do: Applications Engineering, part of CoreWeave’s Data Infrastructure organization, builds the software CoreWeave uses to plan and operate its own infrastructure business. Some of that is UNION,... 
    Senior
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    Manhattan, KS
    2 days ago
  • $126.99k - $141.1k

     ...end-to-end platform that empowers residential real estate agents to deliver exceptional service to seller and buyer clients. Software Engineer ll Compass envisions a world where the experience of selling or buying a home is simple and pleasant for everyone. Founded... 
    Suggested
    Minimum wage
    Flexible hours

    Compass

    Manhattan, KS
    2 days ago
  • $165k - $250k

     ...About the team Mbodi builds the software layer that lets industrial robots learn new skills...  ...inference. As one of our early engineers, you'll work on the core systems that translate...  ...About the role We're looking for a senior software engineer with deep intuitions... 
    Work at office
    3 days per week

    Mbodi AI

    Manhattan, KS
    1 day ago
  •  ...Software Engineer | Manhattan, NYC – In Office | Competitive Base + Equity | ASAP Start We’re partnering with a seed-stage fintech building the infrastructure that helps institutions access prediction markets with confidence. Following an oversubscribed seed round... 
    Full time
    Work at office
    Immediate start

    Albert Bow

    Manhattan, KS
    5 hours ago
  •  ...-growing startup building creator tools is hiring a Full Stack Engineer to design, build, and scale core product features on a modern TypeScript...  ...What We're Looking For ~2-5+ years of professional software engineering experience ~ A solid grasp of modern frontend architecture... 
    Full time

    arcadia

    Manhattan, KS
    3 days ago
  •  ...Senior Mechanical Engineer Full-time Employment Metric Geo have partnered with a very reputable consulting engineering firm in the New York City Metropolitan Area and are hiring for a wonderful position. I am looking for a candidate with the following qualifications... 
    Senior
    Full time

    Metric Geo

    Manhattan, KS
    3 days ago
  •  ...2+ contract Midtown Manhattan We’re looking for an M365 Engineer to help design, manage, secure, and automate enterprise Microsoft...  ...as code, and automated deployments. • Support the full software development lifecycle, from design and testing through deployment... 
    Senior
    Long term contract
    Full time
    Work at office
    Local area

    PRI Technology

    Manhattan, KS
    14 hours ago
  • $200k - $250k

     ...Principal Software Engineer | New York, NY (Hybrid) | $200,000 to $250,000 base plus equity I am recruiting a Principal Software Engineer for a venture-backed healthcare company in New York City. They are moving from an outsourced engineering team to a small in-house... 
    Full time
    H1b
    Work at office
    Remote work
    3 days per week

    OmniTek Recruitment

    Manhattan, KS
    2 days ago
  • $120k - $150k

     ...Electrical Engineer - Mission Critical / Data Centers Your new company Join a specialist engineering and architectural consultancy in New York with a strong focus on mission critical and technically complex projects. You’ll be joining a collaborative team delivering... 
    Senior
    Full time
    Work at office
    Work from home

    Hays

    Manhattan, KS
    3 days ago
  • $200k - $250k

     ...Staff Platform Engineer (AWS / Kubernetes / Terraform) New York City – 4 Days Onsite | $20...  ...financial services organizations. This is a senior, hands-on engineering role combining...  ...systems, event-driven services and durable workflows, while remaining close to implementation... 
    Full time
    Work at office

    IT Recruitment Solutions

    Manhattan, KS
    5 hours ago
  •  ...Company Description FMC Engineering, P.C., founded in 1980, is a consulting engineering firm...  ...cost parameters. Role Description The Senior Electrical Design Engineer is a full-...  ...BIM tools and related electrical design software; ability to read and coordinate multidisciplinary... 
    Senior
    Full time
    Local area

    FMC Engineering, P.C.

    Manhattan, KS
    1 day ago
  • $400k

     ...experience in financial markets, the organization combines innovation, engineering excellence, and data-driven insights to support complex trading operations worldwide. This opportunity is for a Senior Site Reliability Engineer to join a high-performance infrastructure... 
    Senior
    Permanent employment
    Worldwide
    Manhattan, KS
    more than 2 months ago
  •  ...We are seeking an experienced Senior Managed Services Engineer (Tier 3) to provide advanced technical support, troubleshoot complex IT issues, and...  ...Tier 2 support teams, troubleshooting complex hardware, software, network, and infrastructure issues. Provide onsite and... 
    Senior
    Full time
    Remote work
    3 days per week

    ProSearch

    Manhattan, KS
    3 days ago
  •  ...Description Job Description Company Description We are Olsson. We engineer and design solutions that improve the world around us. As a...  ...impact on the world around us. Job Description Are you a Senior-Level Professional Engineer looking to be part of an... 
    Senior
    Full time
    Relocation
    Flexible hours

    Olsson

    Manhattan, KS
    12 days ago
  • About The Job: Hey my name is Elken. I'm seeking a personal support worker working in Manhattan, New York. I plan to hire a dedicated, ethical employee with live-in availability. Details: My family needs personal care, assessments, and respite work. It is preferable if...
    Senior
    Live in

    ElderCare.com

    Manhattan, KS
    26 days ago
  •  ...WE’RE HIRING | TAX SENIOR Manhattan & Long Island, NY | Onsite Are you an experienced Tax Senior looking to take the next step in your public accounting career? We are working with a growing CPA firm seeking a motivated and detail-oriented Tax Senior to join... 
    Senior
    Full time

    Smart4 Energy

    Manhattan, KS
    4 days ago
  • $85.39k - $116.98k

     ...Description Syms Strategic Group (SSG)  is seeking a talented Senior Systems Engineer (Angular) Location: Remote Department: Veterans...  ...services in production on AWS Apply and promote best software engineering practices within an Agile framework Use Git... 
    Senior
    Full time
    Remote work

    Syms Strategic Group, LLC (SSG)

    Manhattan, KS
    5 days ago
  • About the RoleWe are currently seeking an experienced and ambitious Senior Audit Manager to lead attest engagements, manage audit staff and review financial statements. This role will oversee audit planning, execution, reporting, and team leadership while partnering closely... 
    Senior

    Joseph Michaels International

    Manhattan, KS
    1 day ago
  •  ...throughout the year. This firm knows the importance of taking a step back to recharge and connect. They're looking to hire a Senior Tax Associate with experience in real estate, high-net-worth, or manufacturing & distribution sectors to be a part of this dynamic... 
    Senior
    Full time
    Work at office

    Distinct | North America

    Manhattan, KS
    2 days ago
  • $110k - $120k

     ...Small CPA firm ( 25 professionals) in Midtown is looking for a staff/senior tax accountant to Prepare individual, closely held corporate and partnership tax returns with an emphasis on high net worth individual returns with many k-1's and multi state. You should have... 
    Senior
    Full time

    Career Professional Associates

    Manhattan, KS
    1 day ago
  • US Citizen or Green Card holders only/ No Corp to Corp or 3rd party agencies Java Microservices Developer – AWS Position Overview We are seeking an experienced Java Developer to support and enhance our existing desktop applications and backend microservices ...

    The Custom Group of Companies

    Manhattan, KS
    3 days ago
  • $240k

     ...Staff Full Stack Engineer | AI Healthtech | NYC | Up to $240k + Equity The Company...  ...technology company in New York that is building software to solve some of the most complex...  ...testing, observability and CI/CD. Mentor senior engineers, lead technical discussions... 
    Full time
    3 days per week

    Refactor.

    Manhattan, KS
    3 days ago
  •  ...JD- Must-Have Skills: ~8+ years of backend engineering experience. ~ Strong hands-on experience with Java, Spring Boot, and Microservices. ~ Reactive Java experience (highly preferred). ~ Strong Kafka experience and event-driven architecture knowledge. ~ Strong... 

    Samson Software Solutions, Inc.

    Manhattan, KS
    3 days ago
  •  ...CAS (Client Accounting Services) Senior Manhattan, NY Some accounting jobs are about keeping the books tidy. This isn't one of them. If you're an experienced CAS professional, Accounting Manager or Assistant Controller who enjoys getting close to clients... 
    Senior
    Full time
    Work at office
    Work from home

    Distinct | North America

    Manhattan, KS
    1 day ago
  • $160 per hour

     ...AMBITIOUS FOUNDING iOS ENGINEER — Start Up NYC Some products you build. Some you help scale into something huge. Every so often, you get in early enough to do both — and this is one of those. We're after an ambitious iOS engineer who wants real ownership from... 
    Full time
    Work at office
    Night shift

    Initi8 Recruitment

    Manhattan, KS
    2 days ago
  • $90k - $120k

     ...Forward Deployed Engineer (AI / TypeScript) New York City, NY, Manhattan (Minimum 4 Days per Week in Office) $90,000–$120,000 + Equity Are you a Software, Solutions or DevOps Engineer who enjoys working directly with customers and taking technical projects from... 
    Full time
    Work at office

    Rise Technical

    Manhattan, KS
    2 days ago
  • $170k - $190k

     ...Senior Manager, Accounting | Series B SaaS | Midtown, NYC Hybrid working (2-3 days Midtown, NYC) $170,000 - $190,000 (depending on experience) + equity + benefits Talentedge are delighted to be exclusively partnering with a hyper-growth VC backed SaaS scaleup... 
    Senior
    Full time
    Contract work

    Talentedge

    Manhattan, KS
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Software Engineer II - Streaming and Durability. Be the first to apply!