Senior Software Engineer for LLM Evaluation
$50 per hourSaidGig
Role Overview
As a Senior Software Engineer focused on LLM Evaluation, you will play a pivotal role in creating innovative datasets for training and benchmarking large language models. This position involves close collaboration with researchers to curate code examples, provide accurate solutions, and refine AI-generated code across multiple programming languages, including Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go.
Key Responsibilities
- Curate code examples, build solutions, and correct code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go for AI model training initiatives.
- Evaluate and refine AI-generated code to ensure efficiency, scalability, and reliability.
- Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks.
- Develop agents that can verify code quality and identify error patterns.
- Hypothesize on various steps in the software engineering cycle and evaluate model capabilities.
- Design verification mechanisms to automatically validate solutions to software engineering tasks.
Qualifications
- Minimum of 3 years of software engineering experience.
- Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools.
- Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
- Excellent oral and written communication skills for clear, structured evaluation rationales.
Work Terms
- Commitment : Flexible engagement, minimum of 10 hours per week, up to 40 hours per week.
- Type : Contractor (no medical/paid leave).
- Duration : 1 month, with potential extensions based on performance and fit.
- Location : Candidates must be based in the US, Canada, or Western European countries (e.g., Austria, Belgium, France, Germany).
Eligibility
- The application process takes 15-30 minutes.
- Completion of an AI video interview is required.
Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer for LLM Evaluation in Canada vacancy
- ...governments realize their greatest potential. Title and Summary Senior Software Engineer About Mastercard Mastercard is a global technology... ..., refreshing models, and serving low-latency inference. •LLM integration, prompt engineering, and agentic patterns. •...SeniorFull timeWorldwideShift work
$111k - $160k
...governments realize their greatest potential. Title and Summary Senior Software Engineer Overview: Ethoca, a Mastercard company, is seeking a... ..., ensuring timely delivery within budget. • Research and evaluate technical solution alternatives for evolving business needs...SeniorFull timeWorldwide$40 - $75 per hour
...Join a dynamic team as a Software Engineer, contributing to innovative backend software development and AI model evaluation. This role offers the opportunity to leverage your engineering skills to enhance next-generation AI systems by tackling real-world programming challenges...SuggestedRemote jobHourly payFor contractors$100 - $175 per hour
...Role Overview Join a dynamic team as a senior digital chip design and verification engineer, contributing to an AI evaluation program that focuses on cutting-edge silicon and... ...timing analysis ~ Familiarity with leveraging LLM-based tools to accelerate chip design, RTL...SuggestedRemote jobFull time$170k - $220k
...Senior Software Engineer Build financial systems Canadians trust with their money every day At Shakepay, we build products that help Canadians buy, earn, and save bitcoin simply and securely. Since launching in 2015, we’ve grown to more than one million customers across...SeniorRemote jobFull timeWork at officeImmediate start$111k - $160k
...products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Software Engineer Overview: Be part of the Operations & Technology Fraud Products team developing new capabilities for MasterCard's Decision...SeniorFull timeWorldwide$111k - $160k
...products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Software Engineer Who is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and...SeniorFull timeWork experience placementWorldwide$85k - $100k
.... At General Dynamics Mission SystemsCanada, were not just engineering technology were shaping the future of defence and security. Our... ...The LC4ISR ISTAR team is seeking a working level Software Engineer to support delivery of critical capabilities for the...SeniorFull timeWork at officeRemote workFlexible hours$181k
...protect the connected attack surface. We are committed to a remote first culture. The Role We’re looking for a Senior Full-Stack Software Engineer with deep expertise in Next.js, TypeScript, PostgreSQL/Supabase, and AI-assisted development to design, build, and...SeniorRemote jobFull time$166.6k - $263.45k
...ambitious startups. The Treasury team is a team of full-stack engineers that are building thoughtful products that manage billions in... ...and lay out tradeoffs Comfortable mentoring both junior and senior engineers Communicates and collaborates effectively with peers...SeniorFull time$185k - $210k
...identity fraud, payment fraud, account takeovers, and social engineering scams. We have raised $145M from world-class investors,... ...Canada: ~ Ontario About the Role: As a frontend-focused Senior Software Engineer, you will own the design and delivery of the user-facing...SeniorRemote jobFull timeCasual workWorldwideHome officeFlexible hours$132k - $236k
...platform. With AI at the core, Webflow helps teams move faster, create more performant digital experiences, and scale without heavy engineering support. From independent designers and creative agencies to global enterprises, hundreds of thousands of organizations use...SeniorRemote jobPermanent employmentFull timeTemporary workFixed term contractLocal areaFlexible hours$145k - $170k
...role The Backend Platform team's mission is to enable fast, safe, and delightful backend development at scale. As a Senior Software Engineer on Backend Platform, you will own impactful projects, accelerate developer velocity of all backend engineers at the company...SeniorRemote jobShift work$50 per hour
...This role focuses on evaluating AI models for visual document understanding and instruction-following within the Front-End Engineering domain. You will be involved in a benchmark dataset project that requires expertise in crafting complex tasks based on UI screenshots,...Hourly payRemote work$50 - $60 per hour
...per hour, with bonus rates available on some projects Responsibilities: Give AI chatbots diverse and complex problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications: Fluency in...SeniorHourly payContract workFor contractorsWork experience placementRemote workData Annotation
Canadamore than 2 months ago- ...role We're hiring our first Applied AI Engineer to work directly with our Advisory Intelligence... ...for audit Contribute to the AI output evaluation framework: structured test inputs,... ...handoffs Bonus points Hands-on production LLM work with AWS Bedrock, OpenAI, or the...Remote jobFull timeHome office
$25k
...the elegance of simplicity in engineering, transforming the demanding... ...and we’re looking to hire senior backend engineers . We’re a... ...quo for banking and fintech software, and many of our customers are... ...as an AEDT. As part of the evaluation process we provide Covey...SeniorRemote jobFull time- ...trusted brand. About the Role As a Senior Backend Engineer at Breakout, you’ll design and... ...problems and building robust, scalable software Unless a specific application deadline... ...process. These assessments are designed to evaluate competencies relevant to the role and...SeniorRemote jobFull timeLocal area
- ...fosters innovation, transparency, and team connectivity, we look forward to hearing from you! THE ROLE We are seeking a Senior DevOps Engineer with deep AWS expertise, multi-account architecture experience, and strong leadership capability. This role owns our cloud...SeniorFull timeWork at office2 days per week
$111k - $160k
...products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Software Engineer (Java) Who is Mastercard Mastercard is a global technology company in the payments industry. We work to connect and power...SeniorFull timeWorldwide$145k - $170k
...epicenter of this historic cultural and financial shift, keep reading. About the role We’re excited to invite talented and motivated Senior Android Developers to join our award-winning team to help expand Robinhood’s mobile experience for our customers. We’re looking...SeniorRemote jobWork experience placementWork at officeShift work$111k - $160k
...products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Software Engineer (Java) Our Purpose We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making...SeniorFull timeWork experience placementWork at officeWorldwide$147k - $245k
...thinking organization, apply now. We are currently seeking a Senior AI Engineer - Remote to join our team in Toronto, Ontario (CA-ON),... ...capabilities and systems. Person will work closely with our AI team, software engineers, and business stakeholders to create AI-driven...SeniorWork experience placementWork at officeRemote workFlexible hours$240k - $360k
...empowered to do your best work. Role Summary The Senior Principal Architect is a highly influential... ..., technical governance, and the evolution of software and network platforms within Network Platform Engineering (NPE). NPE designs, builds, and operates Equinix...SeniorFull timeWork at officeShift work$111k - $160k
...governments realize their greatest potential. Title and Summary Senior AI Engineer Overview The Security Solutions Data Science team is... ...’s degree in Computer Science, Data Analytics, Mathematics, Software Engineering, or a related field or equivalent practical...SeniorFull timeWorldwide$120k - $150k
...Sr. Embedded Firmware Engineer JOB-10046852 Anticipated... ...We are seeking an experienced Senior Embedded Firmware Engineer to... ...driving innovation, ensuring software reliability, and collaborating... ...seamless user experience Evaluate current applications to identify...SeniorFull timeLocal area$127k - $203k
...Title and Summary Lead AI Engineer Overview We are looking... ...Possess a strong grasp of modern software engineering principles,... ...Guardrails, Agent Validation, and Evaluation. • Advanced capability in... ...across RAG, few-shot prompting, LLM fine-tuning, and hybrid...Full timeWorldwide$136k - $204k
...because when you feel valued, you’re empowered to do your best work. Job Summary We are seeking a Senior Network Integration & Service Validation Engineer to lead end-to-end validation of network overlay services from customer portal request through orchestrator...SeniorFull timeWork at office- The exchange team develops and maintain a cutting-edge trading platform. With a global presence across APAC, EMEA and North America, the team has since grown with the goal to provide a seamless and secure trading experience for our clients. Responsibilities Responsible...SeniorFull timeRemote workFree visa
$298k
...do your best work. Job Summary We are seeking a Senior Director of AI Engineering to lead and scale a high-performing Machine Learning Engineering... ...AI technologies Qualifications ~12–15+ years in software engineering, data engineering, or ML engineering ~5+...SeniorFull timeWork at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Software Engineer for LLM Evaluation. Be the first to apply!
Related searches
- software engineer amazon Canada
- agile software developer Canada
- software developer positions Canada
- senior software design engineer Canada
- software developer Canada
- ngo software engineer Canada
- startup software engineer Canada
- senior software engineer Canada
- real time software engineer Canada
- senior software engineer ruby on rails Canada