Software Engineer for AI Model Evaluation
$50 - $75 per hourSaidGig
Contribute to the advancement of AI models by evaluating AI-generated solutions in a cutting-edge project focused on improving how language models understand, reason about, and generate production-quality software. This role offers a unique opportunity to leverage your software engineering expertise to shape the future of AI-assisted software development. Key Responsibilities
- Become familiar with assigned open-source codebases and their engineering practices.
- Review code changes, technical discussions, and related software artifacts.
- Evaluate AI-generated code for correctness, quality, maintainability, and engineering best practices.
- Provide structured evaluations with clear technical reasoning.
- Collaborate with fellow technical reviewers to ensure high-quality and consistent evaluations.
- Years of Experience: 3+ years of professional software engineering experience or equivalent open-source experience.
- Technical Expertise: Proficiency in JavaScript/TypeScript (Node.js, React, TypeScript), Go (backend services, distributed systems, REST/gRPC APIs, cloud-native development), and Python (backend development, open-source libraries, DevOps/infrastructure automation, testing frameworks).
- Software Engineering: Experience with large production codebases, ability to quickly understand unfamiliar repositories and software architectures, experience reviewing code for readability and maintainability, strong engineering judgment, and understanding of software engineering best practices.
- GitHub & Open Source: Strong understanding of Git and GitHub workflows (branching, Issues, Pull Requests, Code Reviews), experience navigating open-source repositories, familiarity with open-source development workflows, and familiarity with SWE-Bench or SWE-Bench Pro is a plus.
- Communication: Strong English communication skills.
- Employment Type: Hourly
- Expected Commitment: At least 10 hours
- Project Duration: Approximately 1 week
- Expected Timeline: July 8, 14
- Location: Candidates must be located in the US/Canada for real-time collaboration with the project team.
- Hourly Rate: $50 - $75
- This is not a software development role; the focus is on evaluating AI-generated solutions.
Vacancy posted 4 hours ago
Similar jobs that could be interesting for youBased on the Software Engineer for AI Model Evaluation in Canada vacancy
$40 - $75 per hour
...Join a dynamic team as a Software Engineer, contributing to innovative backend software development and AI model evaluation. This role offers the opportunity to leverage your engineering skills to enhance next-generation AI systems by tackling real-world programming challenges...SuggestedRemote jobHourly payFor contractorsSaidGig
Canada9 days ago- ...This role involves contributing to a benchmark dataset project that evaluates AI models focused on visual document understanding and instruction-following specifically within the Radiology domain. As an expert, you will author complex, grounded tasks that include clear...SuggestedHourly payRemote work
SaidGig
Canada10 days ago $100 - $175 per hour
...team as a senior digital chip design and verification engineer, contributing to an AI evaluation program that focuses on cutting-edge silicon and chip-design... ...Reusable verification IP, scoreboards, reference models, coverage-driven regression flows Work Terms Location...SuggestedRemote jobFull timeSaidGig
Canadaa month ago$50 per hour
...Role Overview As a Software Engineering evaluator, you will play a crucial role in creating advanced datasets... ..., and enhancing large language models. This position involves collaborating... ...provide precise solutions, and refine AI-generated code across various programming...SuggestedFor contractorsFlexible hoursSaidGig
Canada16 days ago$100 - $150 per hour
...in shaping the training of next-generation AI systems by providing high-quality, real-... ...legal reasoning and litigation experience to evaluate, refine, and enhance AI-generated legal... ...ambiguous or complex edge cases, informing model calibration for nuanced legal issues. Draft...SuggestedRemote jobHourly payFor contractorsSaidGig
Canada8 days ago$50 per hour
...This role involves contributing to a benchmark dataset project that evaluates AI models focused on visual document understanding and instruction-following within the Surveying & GIS domain. Experts will create complex, grounded tasks that include a clear ground-truth output...Hourly payRemote workSaidGig
Canada10 days ago$90 per hour
...Join a dynamic project focused on evaluating AI models for visual document understanding and instruction-following within the Telecom and Network domain. This role involves authoring complex, grounded tasks that include a clear ground-truth output and an objective rubric...Hourly payRemote workSaidGig
Canada11 days ago$75 per hour
...Join a dynamic project focused on evaluating AI models in the Energy and Utilities sector. This role involves authoring complex, grounded tasks that assess visual document understanding and instruction-following capabilities, ensuring clarity and precision in the output...Hourly payRemote workSaidGig
Canada11 days ago$70 per hour
...This role focuses on evaluating AI models in the Underwriting domain through a benchmark dataset project. Experts will create complex, grounded tasks that include a clear ground-truth output and an objective rubric, contributing to the advancement of visual document understanding...Hourly payRemote workSaidGig
Canada10 days ago$70 per hour
...This role focuses on evaluating AI models for visual document understanding and instruction-following specifically within the Pharma and Clinical Trials domain. You will contribute to a benchmark dataset project by authoring complex, grounded tasks that include a clear...Hourly payRemote workSaidGig
Canada11 days ago$50 per hour
...This role focuses on evaluating AI models specifically in the Construction Estimating domain through a benchmark dataset project. You will be responsible for authoring complex, grounded tasks that include clear ground-truth outputs and objective rubrics, contributing...Hourly payRemote workSaidGig
Canada10 days ago$170 - $190 per hour
...Join a dynamic healthcare AI partner dedicated to building advanced... ...data, and case notes for AI model training. Identify and... ...Collaborate with data scientists and engineers to enhance AI understanding of medical context. Model Evaluation & Feedback: Evaluate AI-...SaidGig
Canada19 days ago$60 per hour
...This role focuses on evaluating AI models through a benchmark dataset project specifically in the Real Estate Appraisal domain. Experts will be responsible for authoring complex tasks that are grounded in real-world scenarios, ensuring a clear ground-truth output and...Hourly payRemote workSaidGig
Canada10 days ago$30 - $50 per hour
...As an Audio Engineer, you will play a crucial role in shaping the future of AI systems by providing high-quality audio input that influences how models learn and perform. Your expertise in audio quality assurance and restoration will directly impact the development of...Remote jobHourly payContract workSaidGig
Canadaa month ago- ...highly skilled Senior Software Developer to design, develop... ...in computer science, Engineering, or a related field (... ...recruitment matching AI technology We use... ...in assessing and evaluating your job application.... ...This tool is a matching model based on factors such...Full timeLocal areaRemote workShift work
$166.6k - $218.7k
...deliberate, company-wide bet on AI. Frontier users are already... ...interface for agents and engineers. * Expand and operate our LLM... ...holding. * Build playgrounds and evaluation harnesses so internal AI... ...tradeoffs in AI deployments: cost modeling, observability, latency, and...Full timeShift work$181k
...We’re looking for a Senior Full-Stack Software Engineer with deep expertise in Next.js, TypeScript, PostgreSQL/Supabase, and AI-assisted development to design, build, and... ...researchers to ensure features align with threat models. Product Collaboration: Work hand-in-...Remote jobFull time$20 - $40 per hour
...expertise to contribute to a transformative project aimed at enhancing AI capabilities. Your role will involve creating high-quality legal... ...cycles to clarify content or enhance document utility for AI model training. Provide written and verbal updates, offering context...Remote jobHourly payFor contractorsSaidGig
Canada2 days ago- ...the role We're hiring our first Applied AI Engineer to work directly with our Advisory... ...and structured reports into typed data models for downstream AI processing Maintain... ...for audit Contribute to the AI output evaluation framework: structured test inputs, expected...Remote jobFull timeHome office
- ...greatest potential. Title and Summary Senior Software Engineer About Mastercard Mastercard is a... ...write code, run prototypes, and use AI coding tools every day to ship faster... ...features for batch and real-time use. •Data models for events, features, reference data,...Full timeWorldwideShift work
$111k - $160k
...greatest potential. Title and Summary Senior Software Engineer Who is Mastercard? Mastercard is a... ...cycle by leveraging scoring models, cohesive rule strategies and behavioural... ...Fraud Rules Engines as Rule Author • AI experience is focused on using AI-assisted...Full timeWork experience placementWorldwide$91k - $140k
...governments realize their greatest potential. Title and Summary Software Engineer II Overview • The Decision Management program enables... ...business agility. • This program leverages business rules & AI engines, a streaming big data cluster, an in memory data grids...Full timeWork at officeWorldwideFlexible hours$185k - $210k
...behavior biometrics, machine learning, and AI to stop fraud before it happens. Today,... ...payment fraud, account takeovers, and social engineering scams. We have raised $145M from world-... ...the Role: As a frontend-focused Senior Software Engineer, you will own the design and...Remote jobFull timeCasual workWorldwideHome officeFlexible hours$165k - $260k
...usable and powerful for all developers. As AI has advanced, Warp has evolved beyond its... ...to code, deploy, and debug production software. Today, we have two products: Warp, the... ...your work reaches hundreds of thousands of engineers every day, we'd love to have you. As a...Full timeWork at officeRemote workFlexible hours$170k - $220k
...Senior Software Engineer Build financial systems Canadians trust with their money every day At Shakepay, we build products that help Canadians... ..., pragmatism, and a bias for action. Active use of AI tools in your day-to-day workflow to accelerate development, debugging...Remote jobFull timeWork at officeImmediate start$132k - $236k
...combining visual development, powerful content management systems, AI-driven personalization, seamless hosting, and end-to-end... ...create more performant digital experiences, and scale without heavy engineering support. From independent designers and creative agencies to...Remote jobPermanent employmentFull timeTemporary workFixed term contractLocal areaFlexible hours$200.7k - $250.9k
...In 1965, an engineer in Scotland was given a mundane task with a tricky... ...sets that teach the model how to handle workflows like sending... ...candidate Has 7 or more years of software engineering experience, with... ...opinions about what makes an AI product trustworthy, not just...Full timeShift work- ...Just Appraised, we're building modern, AI-powered software for local government agencies — specifically... ...the Role We're hiring a Software Engineer II to help build and scale the systems... ...across services, infrastructure, and data models as we scale, and grow your scope of...Remote jobFull timeLocal areaWork from homeHome officeFlexible hours
$154k - $247k
...potential. Title and Summary Principal Software Architect, AI Overview: Mastercard is looking... .... This is a hands-on role for an engineer who is deeply engaged in using Generative... ...AI systems including observability, evaluation, and cost optimization • Establish...Full timeWorldwide$154k - $281k
...content management systems, AI-driven personalization... ...scale without heavy engineering support. From... ...are looking for a Staff Software Engineer to join our Applied... ...with large language models, diffusion models etc.... ...well as AI specific evaluations and quality measurements...Remote jobFull timeTemporary workFixed term contractLocal areaImmediate startFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer for AI Model Evaluation. Be the first to apply!
Related searches
- software engineer amazon Canada
- experienced software developer Canada
- senior software engineer Canada
- software developer fintech Canada
- part time software developer remote Canada
- software engineer Canada
- software engineer healthcare Canada
- network software engineer Canada
- software developer Canada
- software development engineer aws Canada