Software Engineer for AI Model Evaluation

Use your software engineering expertise to shape next-generation AI systems by reviewing, refining, and evaluating AI-generated technical content. In this remote, part-time contractor role you will create high-quality prompts, documentation, and rubric-based assessments that directly influence how models learn, reason, and produce engineering deliverables. No prior AI experience is required, your domain knowledge is the primary asset.

• Analyze, review, and improve AI-generated technical content for clarity, correctness, and alignment with software engineering best practices.
• Create, refine, and evaluate prompts and instructions that guide AI models to produce high-quality technical artifacts.
• Perform rubric-based assessments of AI outputs, providing detailed feedback on quality, correctness, and completeness.
• Draft and edit technical documentation, architecture plans, RFCs, design specifications, engineering proposals, postmortems, and related artifacts for use in AI training.
• Conduct independent research and fact-checking to validate technical assertions and ensure accuracy.
• Interpret and annotate data, applying software engineering judgment to support evaluation and labeling tasks.
• Collaborate asynchronously with project leads and other experts to meet deliverable requirements and follow established guidelines.

• Required skills: critical thinking, analytical reasoning, attention to detail, quality assurance, written communication, technical documentation, prompt authoring and refinement, AI output evaluation, professional writing, report writing, business communication, problem solving, content review, data interpretation, logical reasoning, professional editing, independent research, data annotation, rubric-based evaluation, content evaluation, AI model evaluation, fact checking, RLHF familiarity, technical editing, prompt engineering.
• Preferred experience: 3+ years in software engineering roles such as Software Engineer, Senior Software Engineer, Staff Engineer, Technical Lead, Engineering Manager, or Solutions Architect.
• Proven history of authoring or reviewing technical artifacts, including documentation, architecture documents, design specifications, RFCs, engineering proposals, postmortems, technical blogs, or code reviews.
• Strong written communication and editing skills, with demonstrable attention to quality and detail.
• Familiarity with AI coding tools, such as code assistants or automated documentation generators, is a plus but not required.
• Advanced degrees or credentials from highly selective universities, or equivalent academic or professional achievements, are highly regarded.

• Role type: Contractor, part time.
• Location: Remote.
• Work is performed asynchronously and on a project basis; assignments and workload depend on customer project needs and matching to subject matter expertise.
• Independent contractor engagement, contributing to deliverables under established project guidelines.

• Pay range: $100 to $200 per hour.

• No prior AI experience required, subject matter expertise in software engineering is the deciding factor.
• Selection and ongoing engagement depend on successful vetting and project matching; applicants should be able to contribute as independent contractors.

Express interest through the platform. Candidates will be identified and vetted by the platform''s talent screening process, which includes an AI recruiter. Selected applicants may be asked to complete role-specific evaluations and onboarding steps before being invited to contribute to customer projects.