South Asian Software Engineers: Coding Tasks for AI Evaluation
$40 per hourTemporary
Terac
What You'll Do
Review proposed coding tasks and programming environments for realism Provide technical feedback on the structure and difficulty of evaluation harnesses Walk us through your approach to verifying complex code challenges Suggest improvements to make the test suites more robust
Who Should Apply
Professional software engineer based in South Asia Experience building or reviewing realistic programming tasks Familiarity with coding evaluation harnesses or test automation Comfortable discussing technical workflows in an interview setting
Contract & Payment Terms
You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will not involve access to confidential or proprietary information from any employer, client, or institution. Payments are processed weekly based on services rendered. We are unable to support H1-B or STEM OPT candidates at this time.
Pay: $40
Vacancy posted 20 days ago
Similar jobs that could be interesting for youBased on the South Asian Software Engineers: Coding Tasks for AI Evaluation in San Francisco, CA vacancy
$218.4k - $365.2k
...SalesforceSalesforce is the #1 AI CRM, where humans... ...- to change how software is designed, built... ..., and Context Engineering capabilities.This... ...human oversight, evaluation, and production... ...SDLC, including code understanding and... ...cost per successful task, token and context...SuggestedFull timeSalesforce
San Francisco, CA1 day ago- ...Evaluate and refine AI-generated code for efficiency and reliability. About Us Based in San Francisco... ...AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality... ...to a software engineering task. Required Skills Several...SuggestedPart timeFor contractorsFlexible hours
Turing
San Francisco, CA20 days ago - ...Evaluate and improve AI-generated code for accuracy and quality. About Turing Turing is one of the world... ...We are looking for experienced Software Engineers to help evaluate, benchmark, and improve... ...real-world software engineering tasks, identify correctness and quality...SuggestedContract workPart timeFor contractorsFreelanceRemote work
Turing
San Francisco, CA20 days ago - ...Full Stack Software Engineer Full-Time, Bay Area, Hybrid... ...exciting Build complex AI-native product... ...instead of a recurring task, and where human judgment... ...workflows, with appropriate evaluation, fallbacks, and human... ...you build, and use AI coding agents and automation...SuggestedFull timeRemote workHome office
Peer TechBio Inc.
San Francisco, CA1 day ago $65 per hour
...What You'll Do Review realistic programming tasks for accuracy and difficulty Verify the logic and test cases within provided evaluation harnesses Assess whether coding environments effectively measure software engineering skills Walk us through your thought process...SuggestedContract workFor contractorsRemote workTerac
San Francisco, CA20 days ago$100 - $150 per hour
...Evaluate and improve AI coding models by reviewing generated code. Freelance ·... ...training data, evaluations, and engineering talent. Engagement... ...for experienced, hands-on software engineers to help... ...reviewing: Can it understand the task? Did it choose the right approach...Hourly payPart timeFor contractorsFreelanceImmediate startRemote workTuring
San Francisco, CA20 days ago$117.2k - $176.7k
...SalesforceSalesforce is the #1 AI CRM, where humans... ...of Salesforce engineers use every day to... ..., maintainable code and raise the quality... ...years of professional software engineering... ...team beyond their own tasks.Unleash Your PotentialWhen... ...assess and evaluate candidates’ resumes...Full timeImmediate startSalesforce
San Francisco, CA4 days ago$164.7k - $266k
...and view infrastructure as code, all while we create an SW infrastructure... .... We are seeking a Senior Software Engineer to design, build, and scale... ...and Develop end-to-end evaluation pipelines for LLMs/SLMs,... ...Mississippi, North Dakota, South Dakota, Vermont, West Virginia...Contract workWork at officeLocal areaRemote work2 days per weekDocuSign
San Francisco, CA1 day ago$167.7k - $245.2k
...Our team blends deep expertise in AI, cybersecurity, platform and systems engineering. We are driven by a shared... ...measurable success criteria to evaluate iterations.Architect and build the... ...agent systems that will perform coding tasks for you.Preferred QualificationsSecurity...Full timeTemporary workLocal areaImmediate startFlexible hoursCISCO Systems
San Francisco, CA4 days ago$174k - $252k
...system development code.Participate in, or... ...feedback.Build the software systems, automated pipelines, evaluation harnesses, and knowledge... ...that accelerates AI agents and human... ...harnesses to measure agent task success, turn... ....Google's software engineers develop the next-...Google
San Francisco, CA1 day ago$75 - $90 per hour
...5 - 10 years Location: United States | Canada + 21 more Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands-on security experience to work involving vulnerabilities...Contract workRemote workBraintrust
San Francisco, CA20 days ago$300k
...learning environments that frontier AI labs use to train and evaluate their coding models. Learn more at mechanize.... ...have gotten good at narrow coding tasks but still fail at the complex, judgment-heavy parts of software engineering. We build the environments that expose...Mechanize, Inc.
San Francisco, CA1 day ago- ...Engineering at Qualified Engineering at Qualified... ...at the frontier of AI + product: developing agentic software and embedding AI... ...through collaboration, code review, and strong... ...outcome on large tasks. You know how to use... ...engineering around evaluation, observability, and...Remote work
Enboarder
San Francisco, CA1 day ago $400k
...learning environments that frontier AI labs use to train and evaluate their coding models. Learn more at mechanize.... ...have gotten good at narrow coding tasks but still fail at the complex, judgment-heavy parts of software engineering. We build the environments that expose...Mechanize, Inc.
San Francisco, CA1 day ago- Obsidian is seeking a candid evaluator to assess the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train frontier AI labs. You will review repository-level tasks, reference patches, test harnesses, and grading integrity, and provide...
$270k - $340k
...the collaborative AI workspace where teams... ...for an experienced engineer who has designed and built secure software systems to help... ...how engineers use coding agents, MCP-enabled... ...or pattern than a task or project for someone... ...teaming, model behavior evaluation, or abuse-resistant...Local areaNotion Labs
San Francisco, CA5 days ago$110k - $200k
...Senior Software Engineer Boston; Charlotte; Chicago; Dallas... ...data, technology, and AI, we help clients drive... ...and RAG pipelines to evaluation frameworks and production... ...into technical tasks, with support from your... ...0 days; shipping real code and learning the client...InternshipWork at officeLocal areaRemote work2 days per weekAccordion USA
San Francisco, CA1 day ago- ...product experience for an AI-driven software platform, from the... ...Design and maintain evaluation suites for everything... ...for AI-assisted coding workflows and share what... ...3 years of full-time engineering experience. Proven... ...depth how you scope tasks, supply context and judge...Full timeWork at office
Fast-growing software company
San Francisco, CA3 days ago - ...Corridor Software development is becoming... ...person wrote the code and a person reviewed... ...out how to make AI coding agents more... ...our architecture, engineering practices, and technical... ..., fine-tuning, or evaluation. Interested in... ...per week in our South Beach, San...Work at officeLocal area3 days per week
Corridor Security Inc
San Francisco, CA22 hours ago - ...learning work. Write code, review output... ...workflows for generating tasks, preparing... ...automation or applied AI work. A track... ...maintaining live software in production.... ...model serving or evaluation tooling. Familiarity... ...Degree from a top engineering university. Benefits...Full timeWork at officeRelocation package
Fast-growing technology company
San Francisco, CA4 days ago $350k
...reinforcement learning environments that frontier AI labs use to train and evaluate their coding models. AI models have gotten good at narrow coding tasks but still fail at the complex, judgment-heavy parts of software engineering. We build the environments that expose those...Mechanize
San Francisco, CA4 days ago- ...Clay AI is unleashing the biggest wave... ...is to be the engine those companies use... ...In particular, as coding agents continue to... ...next evolution of software engineering will come... ...development tasks end-to-end, and safely... ...pipelines, including evaluation and validation of...
clay.global
San Francisco, CA1 day ago $39 - $51 per hour
...What you'll do As a Software Engineering Intern on the DevX and... ..., and testing code for new features and improvements... ...-scoped engineering tasks Present your work... ...Copilot or similar AI-assisted coding tools... ...Mississippi, North Dakota, South Dakota, Vermont, West...Permanent employmentFull timeContract workInternshipWork at officeLocal areaRemote work2 days per weekDocuSign
San Francisco, CA22 hours ago$300k
...Uses AI-assisted coding tools (Claude Code, Codex, Cursor) and explicitly... ...Role OpenArt is hiring a Software Engineer to build the agent layer... ...complex, multi-step creative tasks (scene planning, storyboarding... ...loops). Contribute to evaluation frameworks and...Visa sponsorshipvibehackers
San Francisco, CA1 day ago$193.4k - $290k
...combining frontier agentic AI, an enterprise-grade platform... ...Role Overview As a Software Engineer, Agents, you'll build the systems... ...in using practical evaluations to drive task completion quality and customer... ...harness and libraries for coding agents. What You'll Do...Harvey
San Francisco, CA22 hours ago$120k - $250k
...America. By combining AI, advanced software, robotics, and full-... ...The Role Every engineer at Hadrian ships software... ...accessible to AI coding agents, and automating... ...recurring engineering tasks — dependency upgrades... ...decisions. All candidate evaluations and hiring decisions...Permanent employmentFull timeImmediate startRelocation packageFlexible hoursHadrian
San Francisco, CA2 days ago$102 per hour
...looking for an experienced chip design engineer to evaluate Mistral models on real-world... ...and technical review to improve how AI models perform tasks across RTL development, synthesis, physical... ...: Validate tasks involving RTL coding, logic synthesis, and synthesized netlists...Temporary workBraintrust
San Francisco, CA3 days ago- ...a Senior Android Engineer who is architecturally... ...-driven, and AI-native to join our... ...fluently with AI coding agents as a daily... ...tooling, context, and evaluation habits that make the... ...work - scoping tasks for agents, critically... ...the Senior Mobile Software Engineer, we will...Worldwide
Logoslabs
San Francisco, CA1 day ago - ...Analyze GitHub issues and evaluate LLM performance on real code. Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation... ..., is to build verifiable SWE tasks based on public repository... ...the world’s fastest-growing AI companies accelerating the advancement...Contract workPart timeFor contractorsFreelanceRemote work
Turing
San Francisco, CA8 days ago $50 - $100 per hour
...a data lab that helps AI labs & Enterprises train... .... We provide frontier evaluations and reinforcement... ...legal, medical, finance, coding, etc.), where expert-level... .... Role Title: Software Engineer (Open Source Contributor... ...–100/hour Time per task: ~15 hours per run...Part timeFor contractorsRemote workmicro1
San Francisco, CA8 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to South Asian Software Engineers: Coding Tasks for AI Evaluation. Be the first to apply!
Related searches
- software engineer amazon San Francisco, CA
- information technology software engineer San Francisco, CA
- federal - software developer San Francisco, CA
- graduate software developer San Francisco, CA
- rust software engineer San Francisco, CA
- software development engineer (robotics engineer) San Francisco, CA
- IT software developer San Francisco, CA
- software engineer remote San Francisco, CA
- junior software developer internship San Francisco, CA
- remote entry level software developer San Francisco, CA