Agentic Judge (Real Estate License) [210483]

Our client, a tech real estate company, is looking for licensed real estate agents in San Francisco, CA, to provide part-time work as Agentic Judges

This is a fully remote role, but candidates must be located in the San Francisco, CA area.

This is a  part-time role with flexible hours . Work will be assigned on an as-needed basis and can be completed in the candidates’ free time. 

Must have an active Real Estate License , or previously been licensed with experience selling real estate in San Francisco. 

Our real estate tech client is looking for an Agentic Judge. The Judge plays a crucial role in fine-tuning and calibrating AI/ML processes that handle verbal interactions between home buyers and real estate agents for example. Works directly with experienced Operations Managers, ML Engineers, Data Scientists, Software Developers, and Natural Language experts.

• Responsible for ensuring the quality of a number of possible elements related to our projects.
• Projects such as reviewing the accuracy and humanlike traits of automated chatbot answers, classifying elements of a conversation annotated automatically, or reviewing search results based on an understanding of complex user queries.
• Plays an essential role in defining clear guidelines which may be used by non-native English speakers.
• Attention to detail and natural curiosity for language subtleties is essential for the long term scaling of our solutions.

Major responsibilities:

• Review and/or evaluate annotated data to ensure it meets quality standards and project requirements.
• Provide feedback and guidance to engineers to improve accuracy.
• Collaborate with engineers and data scientists to optimize model performance based on annotated data.
• Develop and maintain annotation guidelines and best practices.
• Provide training data to engineers to establish Ground Truth data.
• Provide input/examples on Training content.
• Identify gaps in training content and suggest improvements.

MINIMUM QUALIFICATIONS

• Minimum of 3+ years of active experience as a licensed real estate agent or broker, demonstrating a deep understanding of the real estate transaction process.
• Excellent attention to detail and analytical skills.
• Ability to work effectively in a collaborative team environment.

PREFERRED QUALIFICATIONS

• Demonstrated deep interest and curiosity in Artificial Intelligence, including experience reviewing or evaluating AI-generated content,
• Experience with copywriting, User Experience authoring, and content development around voice and brand guidelines.
• Experience with Machine-generated Speech evaluation, Search Results Evaluation, Search Query Categorization
• Experience following or managing complex processes, and working both independently and with a team to achieve a goal.
• Experience with tools and software used for Natural Language Processing annotation is a plus.
• Experience working on Natural Language Processing projects with a focus on transcriptions, localization, and translation.
• Bachelor’s degree in English, Linguistics or a related field.

#LI-ES1

#LI-Hybrid