Senior Software Engineer - Observability & IRM

The Trade Desk is a global technology company and the world's leading independent platform for digital advertising, with nearly 4,000 employees across more than 30 offices. Our technology helps advertisers reach the right audiences across the open internet - from streaming TV and podcasts to mobile apps, news, and more.


Advertising powers the content people love. By making it more transparent, effective, and responsible, we help support trusted journalism, quality entertainment, and creators worldwide. The world's brands and agencies rely on us to reach their customers and grow their businesses responsibly.


The scale of our platform brings unique technical challenges - from processing massive datasets in real time to building systems that operate reliably on a global scale. When you work here, your impact is worldwide. We welcome diverse perspectives, encourage curiosity, and build teams that learn from one another. If you're driven to solve meaningful challenges, we'd love to meet you.

About the Team:

The Service Excellence (SE) team owns the tools and infrastructure that help engineers at The Trade Desk understand and operate production systems. The Incident Response Services (IRS) taskforce focuses on the on-call experience. The team is responsible for making incidents easier to detect, manage, and optimize using historical data points information.

What you will work on:

• Incident management tooling
• Build and maintain automation around the incident lifecycle: alerting, escalation, incident channels, retros, and SLA tracking
• Help evaluate and migrate our logging stack
• Participate in the re-evaluation of our logging vendor and collection architecture
• Backstage/Service catalog - Extend our internal developer portal with K8s integrations, maturity models, and SLO adoption tooling
• Alert quality tooling - Build the systems that give engineers better signal and less noise - smarter routing, better grouping, tighter feedback loops between alerts and the teams that own them

Who you are:

• Experience building and operating production infrastructure or internal developer tooling
• Comfort working across the stack - this role touches distributed systems, Kubernetes, observability pipelines, and web-based tooling
• Familiarity with observability concepts: logging, alerting, on-call workflows
• Strong debugging instincts: You will be expected to be called on when things break
• Clear communication: The team works closely with engineers across the company; you'll need to explain tradeoffs and advocate for solutions

Plus skills:

• Experience with Grafana, Prometheus, or similar observability tools
• Familiarity with Sumo Logic or other log management platforms
• Prior work on developer portals or service catalog tooling (Backstage, OpsLevel, etc.)
• Experience with Kubernetes at scale
• A deep understanding of HunnyPt

Variety of technical opportunity is one of the best things about working at The Trade Desk as a software engineer which is why we do not expect you to know every technology we use when you start. What we care about is that you can learn quickly and find solutions to complex problems using the optimum tools for the job.What you know is less important than how well you learn and innovate. We don't need engineers who know all the answers; we need engineers who can invent the answers no one has thought of yet, to the questions yet to be asked.

The Trade Desk does not accept unsolicited resumes from search firm recruiters. Fees will not be paid in the event a candidate submitted by a recruiter without an agreement in place is hired; such resumes will be deemed the sole property of The Trade Desk. The Trade Desk is an equal opportunity employer. All aspects of employment will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, veteran status, or any other status protected under federal, state, or local law.

#LI-AD1

In accordance with various US state laws, the range provided is the Trade Desk's reasonable estimate of the base compensation for this role. The actual amount may differ based on non-discriminatory factors such as experience, knowledge, skills, and location.All employees may be eligible to become The Trade Desk shareholders through eligibility for stock-based compensation grants, which are awarded to employees based on company and individual performance. The Trade Desk also offers other compensation depending on the role such as variable compensation-based incentives and commissions. Plus, expected benefits for this role include comprehensive healthcare (medical, dental, and vision) with premiums paid in full for employees and dependents, retirement benefits such as a 401k plan and company match, short and long-term disability coverage, basic life insurance, well-being benefits, reimbursement for certain tuition expenses, parental leave, sick time of 1 hour per 30 hours worked, vacation time for full-time employees up to 120 hours thru the first year and 160 hours thereafter, and around 13 paid holidays per year. Employees can also purchase The Trade Desk stock at a discount through The Trade Desk's Employee Stock Purchase Plan.

The Trade Desk also offers a competitive benefits package. Click here to learn more.

Note: Interns are not eligible for variable incentive awards such as stock-based compensation, retirement plan, vacation, tuition reimbursement or parental leave

At the Trade Desk, Base Salary is one part of our competitive total compensation and benefits package and is determined using a salary range. The base salary range for this role is $124,900—$228,900 USD

As an Equal Opportunity Employer, The Trade Desk is committed to creating an inclusive hiring experience where everyone has the opportunity to thrive.

Please reach out to us View email address on click.appcast.io request an accommodation or discuss any accessibility needs you may require to access our Company Website or navigate any part of the hiring process.

When you contact us, please include your preferred contact details and specify the nature of your accommodation request or questions. Any information you share will be handled confidentially and will not impact our hiring decisions.