Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

LLM Engineer

Ova Technologies

Job Title:


LLM Engineer (Large Language Model Engineer)


Job Summary:


We are seeking a highly skilled LLM Engineer to design, develop, fine-tune, and deploy Large Language Model (LLM) based applications and AI-powered solutions. The ideal candidate will have expertise in Generative AI, Natural Language Processing (NLP), Prompt Engineering, Retrieval-Augmented Generation (RAG), and AI system architecture. This role involves building intelligent assistants, AI copilots, enterprise search platforms, and automation solutions using cutting-edge LLM technologies.


Key Responsibilities:

  • Design, develop, and deploy applications powered by Large Language Models (LLMs).
  • Build AI assistants, chatbots, knowledge management systems, and AI copilots.
  • Develop and optimize Prompt Engineering strategies to improve model responses.
  • Implement Retrieval-Augmented Generation (RAG) pipelines using vector databases.
  • Fine-tune open-source and proprietary LLMs for domain-specific use cases.
  • Integrate AI models with enterprise systems, APIs, databases, and cloud services.
  • Develop evaluation frameworks to measure model performance, accuracy, and reliability.
  • Implement guardrails, safety mechanisms, and hallucination mitigation techniques.
  • Collaborate with Data Scientists, AI Engineers, Product Managers, and MLOps teams.
  • Monitor, optimize, and maintain AI systems in production environments.
  • Research and evaluate emerging AI models, frameworks, and architectures.
  • Ensure compliance with Responsible AI, security, and governance standards.
Required Skills:
  • Strong understanding of Large Language Models (LLMs) and Generative AI concepts.
  • Experience with NLP, Transformer architectures, and AI model deployment.
  • Expertise in Prompt Engineering and Retrieval-Augmented Generation (RAG).
  • Knowledge of AI evaluation, model optimization, and inference strategies.
  • Strong problem-solving and system design skills.
  • Excellent communication and collaboration abilities.
Technical Skills:
  • Programming Languages: Python, SQL, JavaScript
  • LLM Platforms: OpenAI GPT, Claude, Gemini, Llama, Mistral, Cohere
  • AI Frameworks: LangChain, LlamaIndex, LangGraph, Haystack
  • Machine Learning Frameworks: PyTorch, TensorFlow, Hugging Face Transformers
  • Vector Databases: Pinecone, ChromaDB, Weaviate, Milvus, FAISS
  • Cloud Platforms: AWS, Azure, Google Cloud Platform (GCP)
  • APIs: REST APIs, GraphQL
  • MLOps Tools: MLflow, Kubeflow, Airflow
  • Containerization: Docker, Kubernetes
  • Version Control: Git, GitHub, GitLab
Qualifications:
  • Bachelor's degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
  • Master's degree in AI, Data Science, or Machine Learning is a plus.
  • Relevant AI and cloud certifications are preferred.
Experience:
  • 3-8 years of experience in AI Engineering, Machine Learning Engineering, NLP, or Software Development.
  • Hands-on experience building and deploying LLM-powered applications.
  • Experience with RAG architectures, vector databases, and AI orchestration frameworks.
  • Experience integrating AI services into enterprise applications.
Preferred Qualifications:
  • Experience with fine-tuning techniques such as LoRA, PEFT, RLHF, and QLoRA.
  • Knowledge of Agentic AI, Multi-Agent Systems, and Autonomous AI Workflows.
  • Experience with LangGraph, CrewAI, AutoGen, or Semantic Kernel.
  • Understanding of AI Security, Responsible AI, and Model Governance.
  • Experience deploying AI applications in cloud-native environments.
Preferred Qualities:
  • Strong innovation and research mindset.
  • Passion for AI, NLP, and emerging technologies.
  • Excellent troubleshooting and analytical skills.
  • Ability to rapidly learn and adapt to new AI frameworks.
  • Strong ownership and accountability for AI solution quality.
Employment Type:


Full-Time


Location:


Remote / Hybrid / On-site


Nice to Have:
  • Experience building AI copilots, enterprise search systems, and knowledge assistants.
  • Knowledge of multimodal AI models (text, image, audio, video).
  • Experience with AI observability and evaluation platforms such as LangSmith, Arize AI, or Weights & Biases.
  • Familiarity with enterprise SaaS, HR Tech, Healthcare, FinTech, E-commerce, or Customer Support AI solutions.
  • Contributions to open-source AI projects, research papers, AI communities, or hackathons.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the LLM Engineer in New York, NY vacancy
  •  ...You will reach for whatever technique fits the problem, prompt engineering, agentic orchestration, retrieval, or post-training, and because...  ...What You Bring Experience building production LLM or ML powered features end to end, with the judgment to know which... 
    Suggested
    Full time

    Discernis

    New York, NY
    1 day ago
  • $170k - $230k

    DataJobs in New York, NY is seeking to build production-ready AI and LLM systems that directly support marketplace growth and operational efficiency. You will take ownership of AI initiatives from early problem framing and architecture through deployment, evaluation, and... 
    Suggested
    Relocation
    Relocation package

    DataJobs

    New York, NY
    5 days ago
  •  ...LLM Engineer Location: Hybrid - Chicago, IL, New York City, NY, Princeton, NJ Duration: Contract Pay Rate: $52.7/hour W2 Interview Type: Video Visa Restrictions: None Required Skills: Experience implementing RAG and have just used managed stack, and... 
    Suggested
    Contract work

    Syntricate Technologies

    New York, NY
    3 days ago
  • $152k - $241.5k

    We are looking for a passionate and energized engineer to accelerate the integration of APIs across the CUDA-X math library software stack into modern LLM and agentic AI workflows. Leading organizations globally are using GPU-powered data centers for AI, data analytics... 
    Suggested
    Full time
    Remote work

    Nvidia

    New York, NY
    1 day ago
  • Mercury is seeking a Senior Software Engineer for Mercury Command in a role that spans the full stack and LLM-powered workflows. You will design and ship new Command skills, shape multi-step agentic interactions, and own capabilities end-to-end from system prompts to UI... 
    Suggested

    Mercury Technologies, Inc.

    New York, NY
    1 day ago
  • $194k - $349k

     ..., file I/O, hashing, work queues, and memory management. Every engineering team that builds sensor capabilities depends on this layer, so...  ...production rollout, particularly for SLM and large language model (LLM) workloads where inference cost and behavioral correctness both... 
    Permanent employment
    Work experience placement
    Work at office
    Local area
    New York, NY
    1 day ago
  • $95k - $149k

     ...Pune, Dubai, Geneva, Singapore and São Paulo. The RoleAs a Field Engineer you will embed directly with clients to architect and build...  ...transformationsFamiliarity with AI/ML tools and experience implementing LLM-driven automation solutionsStrong understanding of financial... 
    H1b
    Remote work
    Worldwide
    Visa sponsorship
    Work visa
    Flexible hours

    Addepar

    New York, NY
    3 days ago
  • TALENDICA, New York, US, is seeking MS Dynamics 365 Functional Resources with a focus on large language model (LLM) development and optimization for a remote contract role lasting 2-3 months. This opportunity targets professionals who can adapt quickly to evolving NLP workloads... 
    Remote job
    Contract work

    AIToolboard

    New York, NY
    5 days ago
  •  ...faster and building it into the solutions themselves, like putting LLM-powered steps or small agents right inside a workflow. We...  ...so other teams can pick them up later. Building with AI and Engineering Practices Design and build AI into the solutions you ship,... 
    Live in
    Work at office
    Visa sponsorship
    Flexible hours
    Shift work

    IntegriChain

    New York, NY
    6 days ago
  • $134.7k - $202.2k

     ...initiatives, applying strong technical capability and hands-on engineering experience. Contributes to the design, development, delivery, and...  ..., and supportable implementation patterns.Build and integrate LLM, RAG, and agentic solution components into enterprise... 
    Summer holiday
    Immediate start
    Flexible hours

    EY (Ernst & Young)

    New York, NY
    2 days ago
  • $180k - $220k

     ...Business System which makes everything possible. The Staff DevOps Engineer owns the delivery and operational backbone of the platform...  ...delivery infrastructure looks like as workloads become increasingly LLM-driven and agentic.This position reports to the Senior Director,... 
    Full time
    Work at office
    Remote work
    Work from home
    Flexible hours

    Danaher Corporation

    New York, NY
    21 hours ago
  • Wonder seeks a Staff Machine Learning Engineer in New York to own the end-to-end ranking and recommendation stack, including candidate retrieval, multi-objective ranking, and calibration. You will drive cross-team architectural decisions, mentor engineers, and push for... 
    Work at office
    3 days per week

    Blue Apron

    New York, NY
    5 days ago
  • $166.9k - $278.1k

     ....The opportunityWe are seeking a hands-on Lead Forward Deployed Engineer (FDE) to own the end-to-end delivery of AI-native systems into a...  ...application stacks (e.g., agentic systems, orchestration frameworks, LLM-backed services).Proven ability to operate in forward-deployed,... 
    Summer holiday
    Immediate start
    Flexible hours

    EY (Ernst & Young)

    New York, NY
    2 days ago
  • WRITER is seeking an AI engineer to shape how enterprises harness superintelligence. You will build and deploy AI solutions powering the future of work across major clients, ensuring performance, scalability, and ethical alignment of LLMs and AI agents grounded in company... 

    WRITER

    New York, NY
    5 days ago
  •  ...Akamai Security Engineer (AI/LLM Security) NYC, NY or Atlanta, GA (3 Days Onsite) Long-Term Contract Interview: In-Person We are seeking an experienced Akamai Security Engineer with expertise in enterprise-scale Akamai implementations, web application security... 
    Long term contract

    Charter Global

    New York, NY
    3 days ago
  • $153k - $204k

     ...CRWV) in March 2025. Learn more at .What You'll Do:The Systems Engineering team owns the host software stack that turns a freshly provisioned...  ...red run into a root cause quickly.Explore AI-native testing — LLM-driven log triage, failure classification, and regression... 
    Permanent employment
    Full time
    Temporary work
    Casual work
    Live in
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    4 days ago
  • $115k - $135k

     ...Innovation) Team. This position is a hands-on cloud and platform engineering role with a strong focus on AI-enabled platforms, GitHub-based...  ...improvement of AI-enabled platforms and large language model (LLM) services using Azure-native capabilities.Participate in the engineering... 
    Full time
    Work at office
    Remote work
    Relocation package
    Monday to Friday

    AlixPartners

    New York, NY
    2 days ago
  • $500 per month

     ...Angelo , Larry Summers , and Jack Dorsey . Position: Agent Engineer Type: Contract Compensation: $100–$500 per task...  ...Location: Remote Role Responsibilities Build and operate LLM agents in production environments. Evaluate agent... 
    Remote job
    Contract work
    Summer work

    Mercor

    New York, NY
    14 days ago
  • $116k - $182k

     ...WHAT YOU'LL DO We're looking for a Senior Lead Systems Engineer who's passionate about building the next generation of enterprise...  ...Bonus: Experience implementing AI agents, MCP integrations, LLM-powered workflows, or enterprise AI platforms Experience with... 
    Work at office
    Local area
    Flexible hours

    Braze

    New York, NY
    3 days ago
  • $135k - $150k

     ...which makes everything possible.The Senior Data Platform Automation Engineer engineers the automation, developer experience, and reliability...  ...GitHub Actions or GitLab; strong Python skills. Hands-on with LLM/agent applications using Azure AI Foundry (Azure OpenAI, Prompt... 
    Full time
    Remote work
    Flexible hours

    Danaher Corporation

    New York, NY
    21 hours ago
  • $178k - $267k

    AlphaSense is looking for a passionate software engineer to join their AI & Search mission in New York. In this role, you'll architect and implement scalable services for AI and search products, improving reliability and cost efficiency. You should have solid back-end... 

    BetterCloud

    New York, NY
    5 days ago
  •  ...We're looking for a Software Engineer III to help build the next generation of Al-powered developer tools and automation platforms. You'...  ...with Al/ML or agentic frameworks (LangChain, AutoGen, CrewAl) and LLM APIs (OpenAl, Azure OpenAI, Anthropic) . Experience with... 

    Techvilla Solutions

    New York, NY
    8 days ago
  •  ...expense. LI-Hybrid DIRECTV is seeking a Principal, System Engineer 2 to design, build, and operationalize sophisticated agentic AI...  ...such as TypeScript, Java, Go, or C#.  Deep knowledge of LLM applications, multi-agent orchestration, tool use, structured outputs... 
    Work at office
    Local area
    Relocation
    Relocation package
    2 days per week

    DIRECTV

    New York, NY
    1 day ago
  •  ...VP Hardware Engineering Location: Remote - New York, NY or San Francisco, CA A stealth-mode AI hardware startup is turning breakthrough...  ...that maximizes tokens-per-second and minimizes latency for LLM workloads. Establish relationships with foundries, EDA vendors... 
    Full time
    Remote work
    Flexible hours

    Blue Signal Search

    New York, NY
    21 hours ago
  • $148.5k - $223.9k

     ...enterprise applications.Making computer use dependable raises hard engineering questions. How does an agent know it succeeded? When should it...  ....You have hands-on experience building and evaluating LLM-powered systems, and can explain how you assessed their quality... 
    Full time

    Salesforce

    New York, NY
    1 day ago
  •  ...cross-functional work involving Product, AI Infra, Platform, and Engineering teams across the AI Cloud orgBreak down ambiguous technical...  ...evolves as the technology evolvesStrong curiosity around AI agents, LLM infrastructure, agentic workflows, automation systems, or... 
    Work at office
    3 days per week

    Rippling

    New York, NY
    21 hours ago
  • $32 - $38 per hour

    DescriptionKforce has a client that is seeking a QA Engineer in New York, NY.Responsibilities:* Design, develop, and maintain automated test...  ...* CI/CD Pipelines* Git/GitHubAI & Data* Generative AI* LLM Evaluation* Prompt Testing* Golden Dataset Creation* Evaluation... 

    KForce

    New York, NY
    2 days ago
  •  ...spend it on something that matters to the world. The Production Engineering Team Examples of key exciting problems the team is working...  ...systemic cause, and move on. You're fluent with AI tooling. LLM APIs, MCP servers, and agentic frameworks, and you drive Claude... 
    Local area

    Fluidstack

    New York, NY
    3 days ago
  • $140k - $215k

     ...alerts, and preventions on the platform.About the Role:As a sensor engineer within the Windows Content Development team, you will focus on...  ...and big data systems.Experience leveraging frontier AI/LLM technologies to accelerate developer velocity, increase feature... 
    Full time
    Work experience placement
    Work at office
    Local area
    2 days per week

    CrowdStrike

    New York, NY
    4 days ago
  •  ...invite you to join us. The Role We're hiring a GTM Systems Engineer to build and own the systems layer behind Rogo's revenue engine....  ...building or extending MCP servers, agent frameworks, or LLM orchestration tooling. Production experience with iPaaS or automation... 
    Contract work
    Shift work

    Rogo

    New York, NY
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to LLM Engineer. Be the first to apply!