Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Software Engineer, Infrastructure

Full-time

Fudo Partners

Infrastructure Software Engineer | AI Infrastructure | ML Systems | Inference

We're representing an ambitious, venture-backed AI technology company building the infrastructure that enables next-generation AI applications to operate at scale.

The company operates at the intersection of machine learning infrastructure, distributed systems, cloud computing and high-performance model inference. Its engineering platform enables AI teams to deploy, serve and scale increasingly sophisticated models in production environments.

This is an opportunity to work on the underlying systems powering modern AI, solving complex infrastructure challenges across compute, orchestration, performance and reliability.

The Role

We're looking for an Infrastructure Software Engineer to help design, develop and scale the core systems behind a high-performance ML inference platform.

You'll work across infrastructure engineering, distributed systems and machine learning operations, building software that enables production AI workloads to run efficiently across cloud and GPU environments.

This is a hands-on engineering position suited to someone with strong software engineering fundamentals, an interest in complex infrastructure problems and a desire to work closer to the systems powering AI.

Engineering Challenges

The team is working across several technically demanding areas, including:

  • Distributed infrastructure and multi-cloud resource management
  • GPU infrastructure and accelerated computing
  • High-performance LLM inference and model serving
  • Multi-node distributed inference architectures
  • Kubernetes-based orchestration and container scheduling
  • Compute allocation, capacity planning and resource optimisation
  • Infrastructure automation and deployment systems
  • Observability, monitoring and production reliability
  • Scaling machine learning workloads across cloud environments

Responsibilities

  • Design, build and maintain infrastructure software supporting production machine learning workloads.
  • Develop backend infrastructure services using Python and Go.
  • Build and improve Kubernetes-based deployment and orchestration systems.
  • Contribute to the infrastructure responsible for deploying, managing and scaling machine learning models.
  • Develop systems for resource allocation, workload scheduling and compute optimisation.
  • Improve the performance, scalability and reliability of distributed infrastructure.
  • Build monitoring, observability and operational tooling for production AI systems.
  • Support infrastructure automation and improve deployment workflows.
  • Collaborate with engineers across infrastructure, platform and applied machine learning teams.
  • Identify opportunities to improve system architecture, engineering efficiency and operational performance.

Technical Experience

We're interested in engineers with experience or exposure to several of the following:

  • Software Engineering: Python, Go, backend engineering and systems programming.
  • Cloud Infrastructure: AWS, GCP, Azure and cloud-native architecture.
  • Containerisation: Kubernetes, Docker and container orchestration.
  • Distributed Systems: Distributed computing, service architecture, concurrency and fault tolerance.
  • ML Infrastructure: Model deployment, model serving, ML platforms and production machine learning systems.
  • AI Inference: LLM inference, inference optimisation, model execution and serving infrastructure.
  • Compute Infrastructure: GPU workloads, resource management and accelerated computing.
  • Observability: Monitoring, logging, metrics and production infrastructure reliability.

What We're Looking For

  • Strong software engineering fundamentals and the ability to build production-quality systems.
  • Experience with at least one modern programming language, ideally Python or Go.
  • Working knowledge of Kubernetes, containers and cloud infrastructure.
  • An understanding of distributed systems and infrastructure architecture.
  • Exposure to machine learning infrastructure, model deployment or production AI systems.
  • An interest in GPU computing, inference infrastructure and the engineering challenges associated with scaling AI.
  • A practical, problem-solving mindset and an appetite for tackling technically complex challenges.
  • Strong communication skills and the ability to work effectively within a collaborative engineering environment.

Compensation & Benefits

  • Competitive salary and meaningful equity participation.
  • Comprehensive medical, dental and vision coverage for employees and eligible dependants (US).
  • Flexible paid time off and company-wide winter break.
  • Paid parental leave.
  • Family-building and fertility benefits.
  • Company-supported retirement savings plan (US).
  • Exposure to a technically ambitious engineering environment working at the forefront of AI infrastructure and machine learning systems.

Interested?

If you're an infrastructure engineer interested in distributed systems, GPU computing, ML infrastructure and the next generation of AI inference platforms, we'd be interested in speaking with you.

Apply directly or get in touch for a confidential conversation.

Vacancy posted 12 hours ago
Similar jobs that could be interesting for youBased on the Software Engineer, Infrastructure in New York, NY vacancy
  • $10,500 per month

     ...Company Palantir builds the world's leading software for data-driven decisions and...  ...The Role Are you passionate about engineering quality, performance, and increasing the...  ...ever imagined, as you build the shared infrastructure that underpins the Palantir Foundry, Palantir... 
    Suggested
    Work experience placement
    Internship
    Work at office
    Remote work
    Work from home

    Palantir

    New York, NY
    4 days ago
  •  ...Join DoppelDoppel is building the future of social engineering defense. Our AI-native platform uses agentic AI to protect executives...  ...world's most recognized brands detect and dismantle attacker infrastructure while strengthening employee resilience through threat-... 
    Suggested

    Doppel

    New York, NY
    2 days ago
  • $164k

     ...Software Engineer, InfrastructureChicago, IL, USA; New York, NY, USA; San Francisco, CA, USAAbout the RoleThe Infrastructure Engineering organization comprises three sub-teams: Core Infrastructure (which manages the infrastructure systems that power Chime, including AWS... 
    Suggested
    Full time
    Work at office
    Local area
    Remote work

    CHIME INC.

    New York, NY
    2 days ago
  • $160k - $300k

     ...Infrastructure EngineerWe are looking for an infrastructure engineer who treats cloud infrastructure as a software problem. You will own Hebbia's AWS footprint end-to-end — accounts, networking, container orchestration, and everything defined in code — and the developer... 
    Suggested

    Hebbia

    New York, NY
    5 days ago
  •  ...Epiq AI Labs is the innovation and engineering hub behind Epiq’s next-generation AI platform...  ...You will build and operate the cloud infrastructure and automation behind Epiq AI Labs’s AI...  ...and accountable for delivering working software. Required Qualifications ~3+... 
    Suggested

    Epiq

    New York, NY
    2 days ago
  • $180k - $230k

     ...modern world: everyone relies on the same stagnant cellular infrastructure and legacy systems that track our every movement, monitor...  ...end-to-end for FedRamp.Shape and influence what great software engineering practices look like.Balance short term critical business needs... 
    Odd job
    Temporary work
    Work at office
    Work from home
    Flexible hours

    Private Tech

    New York, NY
    2 days ago
  • $260k - $300k

     ...CognitionWe are an applied AI lab building end-to-end software agents. We're the makers of Devin, the first AI software engineer.Our team is extremely talent-dense. Among our...  .... None of that works without world-class infrastructure underneath it. You will own the compute,... 

    Cognition AI

    New York, NY
    2 days ago
  •  ...Software Engineer, InfrastructureScaled Cognition is the world's only model lab dedicated exclusively to customer experience and pioneering...  ...that enterprises can trust.As a Software Engineer, Infrastructure at Scaled Cognition you will:Design and improve inference infrastructure... 

    Scaled Cognition

    New York, NY
    2 days ago
  • $184k - $259.44k

    Scale AI is seeking a highly skilled and motivated Software Engineer, Frontier AI Infrastructure to join our dynamic Public Sector Engineering team. As a part of this team, you will own the model inference layer - enabling state of the art models, debugging the latest AI... 
    Full time
    Work at office
    3 days per week
    Early shift

    Scale AI

    New York, NY
    1 day ago
  • $159k - $275k

     ...including stablecoins. You’ll help design, deploy and operate the infrastructure that makes this possible.This is a hands-on devops role with...  ...role sits at the intersection of infrastructure, platform engineering and security. You’ll define how Modern Treasury scales and... 
    Local area
    Immediate start
    Remote work

    Modern Treasury

    New York, NY
    3 days ago
  • $160k - $240k

    Senior Software Engineer - Hadoop Infrastructure Location New York Business Area Engineering and CTO Ref # 10054847 Description & Requirements The Team Bloomberg runs on data! It's our business and our product. Financial institutions need timely,... 
    Temporary work
    For contractors
    Work experience placement

    Bloomberg

    New York, NY
    3 days ago
  • $202k

     ...Sunnyvale, CA, United States; New York City, NY, United StatesSchedule: Full time, Level 5Posted: 2026-11-12Senior Software Engineer - ML Infrastructure, Mobility Simulation and PlanningAbout the role and teamHow can Uber plan pricing and incentives to meet financial goals... 
    Full time
    Work at office
    Remote work

    Uber

    New York, NY
    4 days ago
  • $200k - $325k

     ...customers. Cohere is a team of researchers, engineers, designers, and more, who are all...  ...Seoul. Join us!Why this team?Internal Infrastructure builds the infrastructure and tools used...  ...or accelerator health is a plus.Written software in Python or Go. You would rather extend... 
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    New York, NY
    2 days ago
  • $160k - $240k

    Senior Software Engineer - RDF Infrastructure Location New York Business Area Engineering and CTO Ref # 10054258 Description & Requirements Our groupThe BQL and Semantic Platform group builds the core query and semantic infrastructure used to access... 
    Temporary work
    For contractors
    Work experience placement

    Bloomberg

    New York, NY
    3 days ago
  • $150k - $200k

     ...customers more time for their life’s work.About the Role:The Product Infrastructure team works on creating abstractions and data models that...  ...by handling entire classes of problems up-front for product engineers.Solve hard technical challenges such as designing... 
    Local area

    Notion Labs

    New York, NY
    4 days ago
  • $150k - $168k

     ...time for their life’s work.About the Role:Web Infrastructure builds the foundations behind Notion’s web clients so product engineers can ship fast, reliable, accessible...  ...fundamentals and some experience building software with JavaScript or TypeScript; familiarity... 
    Local area

    Notion Labs

    New York, NY
    2 days ago
  • $175k - $215k

     ...magic of frictionless experiences.We’re looking for a Senior Software Engineer to join our Network Engineering team to accelerate building...  ...platform. In this role, you will build the next-generation infrastructure that underpins all systems at CLEAR. The ideal candidate... 
    Casual work
    Work at office
    Flexible hours

    Secure Identity

    New York, NY
    3 days ago
  • $500 per month

     ...Office snacks ~ Direct access to world-class investors and advisors About This Role We are hiring a Senior Software Engineer to own the infrastructure behind our product and engineering team. You will build and run the systems that power real-time dispatch... 
    Full time
    H1b
    Work at office
    Local area
    Visa sponsorship
    Night shift

    Cephas Consultancy Services Private Limited

    New York, NY
    7 days ago
  • $250k

     ...Job Description Senior Software Engineer (Infrastructure) | New York (On-site) | Up to $250,000 Join a growing AI lab, who are building software that enables AI-driven scientific discovery. You’ll join a small engineering team, to take lead on building the... 

    Edison Smart®

    New York, NY
    13 days ago
  •  ...Palantir builds the world’s leading software for data-driven decisions and operations...  ...The Role   Are you passionate about engineering quality, performance, and increasing the...  ...ever imagined, as you build the shared infrastructure that underpins the Palantir Foundry, Palantir... 
    Internship

    palantir

    New York, NY
    5 days ago
  •  ...goals. Whoever deploys frontier compute infrastructure fastest will decide whether AI expands...  ...them - with teams spanning hardware and software. Speed and scale are our key...  ...that matters to the world.The Production Engineering TeamExamples of key exciting problems the... 
    Local area

    Fluidstack

    New York, NY
    4 days ago
  • $158.1k - $213.8k

     ...Ads Agent Fabric team delivers the AI infrastructure platform that accelerates agentic application...  ...will not merely go through the full software development cycle but more importantly...  ...- 1+ years of software development engineer or related occupational experience- 1+... 
    Permanent employment
    Temporary work
    Internship
    Flexible hours

    Amazon

    New York, NY
    1 day ago
  • $164k - $295k

     ...this pivotal role, you'll harness cutting-edge technologies to engineer robust backend services that seamlessly analyze massive data...  ...that integrate seamlessly with our platform ecosystem Infrastructure Design: Architect supporting systems including databases, real... 
    Permanent employment
    Work experience placement
    Work at office
    Local area
    Worldwide
    2 days per week
    3 days per week
    New York, NY
    a month ago
  • $182k - $242k

     ...startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate...  ...customers perceive as the platform's.About the Role:As a Senior Engineer on the Cloud Platform API team, you'll design and operate the... 
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    2 days ago
  • $175k - $315k

     ...cybersecurity starts with you. The CrowdStrike Cloud Detection Engine (CDE) team is building a powerful, next-generation platform...  ...skills to meet your commitments on time and produce high-quality software that is unit tested, code reviewed and checked in regularly for... 
    Permanent employment
    Work experience placement
    Work at office
    Local area
    2 days per week
    3 days per week
    New York, NY
    a month ago
  •  ...About the Team The Agent Infrastructure team at OpenAI is responsible for building systems that...  ...code, debug issues, and develop software just as human SWEs do. Our training environment...  ...world. About the Role As a Software Engineer on the Agent Infrastructure team, you... 
    Full time
    Work at office
    Worldwide
    Relocation package

    OpenAI

    New York, NY
    13 days ago
  • $125k - $225k

     ...(CSPM) offering provides visibility into your entire cloud infrastructure, continuous monitoring for misconfigurations, and proactive...  ...increasing complexity. Relentlessly pursue quality by following engineering best practices including: Appropriate testing paradigms,... 
    Permanent employment
    Work experience placement
    Work at office
    Local area
    3 days per week
    New York, NY
    a month ago
  • $160k - $240k

    Senior Software Engineer - Cloud Native Compute Infrastructure Core Location New York Business Area Engineering and CTO Ref # 10053399 Description & Requirements Who We AreThe CNCI Core & Lifecycle team - part of Bloomberg’s Cloud Native Compute... 
    Temporary work
    For contractors
    Work experience placement

    Bloomberg

    New York, NY
    4 days ago
  •  ...world's most reliable, on-demand logistics engine for last-mile delivery! We're bringing...  ...maintain a 24x7, no downtime, global infrastructure system that powers DoorDash’s three-...  ...because…You have 5+ years of experience as a software engineer in infrastructure, cloud, or... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    New York, NY
    4 days ago
  • $147k - $210k

     ...experience with developing large-scale infrastructure, distributed systems or networks, or experience...  ....2 years of experience with software development in C++, Python, or Java programming...  ...and resolution, and software test engineering. Experience with configuration management... 

    Google

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineer, Infrastructure. Be the first to apply!