Sr. Manager, Engineering - AI Gateway (LLM Inference)
$228.6k - $314.25kMenlo Ventures
At Databricks, we are passionate about enabling data teams to solve the world’s toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world’s best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers — and customer obsessed — we leap at every opportunity to tackle technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines. And we're only getting started. The impact you will have As the Sr. Manager of Engineering in our newly opened NYC engineering hub, you will be leading teams building multiple products that enable our customers to use innovative ML/AI techniques to add value to their business. You will work closely with product and research on cutting‑edge technologies and be responsible for launching and growing new products, playing a key role in defining the future of how AI and data applications are built. The Databricks AI Gateway is an enterprise control plane for governing, routing, and monitoring LLM endpoints, coding agents, and model serving endpoints on Databricks. AI Gateway lets you standardize, secure, and observe all LLM inference traffic on Databricks, while capturing detailed production data to manage cost, performance, and quality over time. What we look for 8+ years of industry experience building and supporting large-scale data or production applications Experience building classic ML or GenAI systems. A passion for building and scaling early‑stage products Ability to adapt in a fast paced environment. Motivated by delivering customer value and impact. Experience driving company initiatives towards customer satisfaction. BS/MS/PhD in Computer Science or related majors, or equivalent experience. Pay Range Transparency Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles. Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Based on the factors above, Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above. For more information regarding which range your location is in visit our page here. Local Pay Range
$228,600 — $314,250 USD
About Databricks Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortun 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region, please visit Our Commitment to Diversity and Inclusion At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics. Compliance If access to export‑controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone. #J-18808-Ljbffr Menlo Ventures- ...Senior Manager Of Software Engineering When you mentor and advise multiple technical teams and move financial... ...-level software engineers delivering AI-enabled capabilities across the... ...strategies for use-case delivery, including LLM integration approaches, evaluation...SeniorFlexible hoursShift work
$229.9k - $262.4k
...Senior Lead AI Engineer (LLM Gateway, FM Hosting) Overview: At Capital... ...scientists, technical program managers, and product managers to... ...training, large language model inference, similarity search,... ...MA: $229,900 - $262,400 for Sr. Lead AI Engineer...SeniorFull timePart timeLocal area- ...Senior Principal Software Engineer We're looking for a... ...to-end model lifecycle management, including training,... ...optimization using Model Inference servers such as Triton... ...production operations for AI workloads, including... ...architecting and deploying LLM & GNN solutions on AWS...Senior
- BaseTen Labs, Inc. seeks an Engineering Manager (Player & Coach) to lead a team focused on LLM inference workloads. You will mentor engineers, ensuring high-quality delivery and aligning project goals with company objectives. Applicants should have a strong background...SuggestedFlexible hours
$286.2k - $326.7k
...Sr Director, AI Engineering Overview: At Capital One, we are creating responsible... ..., technical program managers, and product managers to deliver... ..., large language model inference, similarity search,... ...introduce state-of-the-art LLM optimization techniques to...SeniorFull timePart timeLocal area$174k - $189k
...At TradeStation, we are building an AI-First culture. We expect team members... ...Are Looking For We are looking for a Sr. Manager, Software Engineering to lead the build‑out of TradeStation... ...WebSockets) Strong understanding of API gateway and management platforms (e.g., Kong,...SeniorWork experience placementLocal areaRemote workFlexible hours- ...Sr Manager of Software Engineering This is your chance to change the path of your career and guide a team to... ...Encourage responsible, effective use of AI developer tools (e.g., Copilot) to... ...on experience with AI (e.g., Co-Pilot, LLM). ~ Experience with Java. ~ Experience...Senior
$300k
...interpretable, and steerable AI systems. We want AI to... ...committed researchers, engineers, policy experts, and... .... About The Role Our Inference team builds and... ...production workloads. Managing multi‑region deployments... ...traffic management systems. LLM inference optimization,...SeniorWorldwide$176.17k - $251.67k
...Machine Learning Engineering Manager - LLM Serving (Remote - US) We are currently looking for a Machine... ..., and driving adoption of innovative AI‑powered features. This is a highly visible... ..., offline evaluation, and real‑time inference. Oversee the development and...Remote workFlexible hours- A leading data and AI platform company in New York is seeking a Sr. Manager of Engineering to lead teams focused on developing innovative products utilizing ML/AI techniques. The successful candidate will have a minimum of 8 years in building robust applications and will...Senior
$300k
...Staff + Sr. Software Engineer, Inference San Francisco, CA | New York City, NY | Seattle, WA About... ..., interpretable, and steerable AI systems. We want AI to be safe and... ...balancing, request routing, or traffic management systems LLM inference optimization, batching,...SeniorWork at officeWorldwideVisa sponsorshipFlexible hours$180k - $240k
A leading AI company seeks a Senior Software Engineer to join their LLM team. This person will build and maintain the LLM gateway service that connects customers to multiple LLM providers. Responsibilities include working closely with the customer success team to understand...SeniorRemote work- A leading financial services company is seeking a technical professional to develop AI and ML algorithms. The ideal candidate has at least 4 years of experience in programming and deploying AI solutions on cloud platforms. Responsibilities include building state-of-the...Senior
- THE ROLE As an Engineering Manager (Player & Coach), you will lead and mentor a team of Forward... ...on building, scaling, and optimizing LLM inference workloads for Baseten customers. Applying... ...high performance, low latency AI applications on Baseten’s platform. FDE...Temporary workFlexible hours
- A leading financial services company in New York seeks an experienced AI Engineer to develop and optimize machine learning algorithms. Candidates should hold a Bachelor's or Master's degree and have significant experience programming in Python and leveraging cloud platforms...Senior
$192.1k - $264.18k
...As the manager of the Digital Natives Solutions Architect (SA) team, you will focus on growing... ...with sales, services, product, and engineering to drive solutions and outcomes for these... ...across a wide range of data, analytical, and AI workloads. Adapt the SA team's skills and...SeniorLocal areaWorldwide$150k - $200k
...UPCLEAR UpClear delivers a SaaS revenue management platform that is used by some of the... ...POSITION OVERVIEW We are hiring a Sr. Manager Data Engineering & Analytics to define our data... ...Professional experience integrating in-house AI models and API calls to foundational...SeniorWork at office$286.2k - $326.7k
...Sr Director, Software Engineer - Card Tech AI We are seeking a Senior Director of Software Engineering to join our... ...I=E). In this role, you won't just manage; you will operate as a "Player-... ...engineering ~2+ years of experience with LLM coding assistance tools such as...SeniorFull timePart timeLocal area$171k - $260k
...seeking an experienced and highly skilled engineering leader to drive the architecture,... ...years of experience leading technologists to manage and solve complex technical items within... ...application development. Experience with AI coding tools like GitHub Copilot, Cline,...Senior$158.6k - $181k
...Associate, Data Scientist - LLM Customization TeamAt... ...the Open Banking future. AI is transforming every... ...applied researchers, software engineers, machine learning engineers and product managers to deliver AI powered... ...in training data and inference volumes. You have experience...Full timePart timeLocal areaFlexible hours- ...Senior Manager Of Cloud Engineering As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive... ...ve redefined modern security with the world's most advanced AI-native platform. Our customers span all industries, and they...SeniorWork at office2 days per week
- ...Senior Vice President, Engineering Leader We're seeking an experienced and highly skilled... ...years of experience leading technologists to manage and solve complex technical items within... ...development. Experience with AI coding tools like GitHub Copilot, Cline,...Senior
- ...Staff + Sr. Software Engineer, Inference Deployment San Francisco, CA | New York City, NY | Seattle, WA... ...reliable, interpretable, and steerable AI systems. We want AI to be safe and... ...intersection of automation and resource management, this team will give you an outsized...SeniorWork at officeVisa sponsorshipFlexible hoursShift work
$180k - $240k
...at the forefront of Speech AI, creating powerful models for... ...Senior Software Engineer to join our LLM team. This role is focused... ...building and maintaining our LLM gateway service—a unified API platform... ...load balancing, and traffic management Experience with circuit...SeniorFull timeRemote workEasy work$176.5k - $201.4k
...York City seeks a Principal Associate, Data Scientist to join its LLM Customization Team. The ideal candidate is skilled in Natural... ...has experience collaborating in cross-functional teams to develop AI-driven products. Responsibilities include leveraging advanced technologies...$195k - $275k
...securities, investment management, and wealth management... ...Machine learning and Gen AI Platform team(s),... ...overall cloud adoption and engineering roadmap and ensure scalable... ...Responsibilities: Sr Hand-On Engineer who... ...multi-agent workflows. LLM Implementation: Deep...SeniorTemporary workWorldwide$204k - $259k
...Senior Machine Learning Engineer – VLM/LLM Evaluation Waymo is an autonomous driving technology... ...S. states. The mission of the Waymo AI Foundations team is to develop machine... ...demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust...SeniorFull timeTemporary workRemote work$280k - $325k
...SoundCloud. As a Senior Principal Engineer, you will combine deep... ...architecting the agentic workflows and LLM-assisted systems that will... ...helping to lead the charge on AI-native engineering Stay... ...infrastructure and lifecycle management ~ Hands-on familiarity with...SeniorTemporary workWork at officeWork from homeFlexible hours$186.1k - $292.81k
...transforming the way companies plan, manage, and optimize sales... ...revenue performance. About the Role AI agents are becoming central to... .... We’re looking for a Senior Engineering Manager to own our AI agent... ...responsible for our internal agent SDK, LLM orchestration layer, and the...SeniorRemote workFlexible hoursShift work- ...Consultants is seeking an experienced Artificial Intelligence Engineer to design and deploy AI/ML and Generative AI solutions addressing real-world... ...to deliver AI-driven applications using production-grade LLM-powered systems. The ideal candidate has 5+ years in AI/ML...Senior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Sr. Manager, Engineering - AI Gateway (LLM Inference). Be the first to apply!
- senior development executive New York, NY
- senior technical manager New York, NY
- senior medical writer New York, NY
- senior procurement specialist New York, NY
- senior software development engineer in test New York, NY
- senior communications specialist New York, NY
- senior manager data science New York, NY
- senior platform engineer New York, NY
- senior procurement New York, NY
- senior director product management New York, NY

