AI Multimedia Generation Product Evaluator [Remote]

Job Responsibilities:

• Set up Halo tasks and run model inference
• Enqueue new queues to Halo
• Audit annotations from vendors
• Build necessary tooling for managing evaluation datasets
• Collaborate with model development teams to conduct offline evaluations and automate pipelines
• Facilitate, set up queues, manage image radio data, and audit results
• Ensure consistency and accuracy in measurements, involving programming and auditing tasks
• Collaborate with model development teams to conduct offline evaluations, connect offline metrics with online user behavior, and ensure offline metrics are predictive of online user engagement. Main tasks include facilitating, setting up queues, managing image radio data, and auditing results.

Skills:

• 1-2 years of basic Python knowledge, particularly in working with image/video data and JSON
• Experience in SQL and handling large data tables
• Experience in the image and video domain
• Ability to adapt to new code
• General know-how
• Flexibility in working hours
• Nice-to-Have Skills:
• Experience as a GenAI user in the image and video domain
• Basic shell script knowledge
• 1-2 years of basic Python knowledge, particularly in working with image/video data and JSON
• Experience in SQL and handling large data tables
• Experience in the image and video domain
• Experience as a GenAI user in the image and video domain
• Basic shell script knowledge

Education/Experience:

• Bachelor’s preferred, not required