Data Annotation Engineer

Job Description

Job Description

Data Annotation Engineer

Location - Louisville, Kentucky (Day 1 onsite)

Pay Rate: Market- based on experience

• Lead the day-to-day operation of the AI annotation program.
• Manage a team of annotators responsible for reviewing customer conversations and AI interactions.
• Develop annotation guidelines, procedures, and quality standards.
• Establish calibration programs and quality assurance processes to ensure consistency across reviewers.
• Partner with Data Scientists to create and maintain gold-standard datasets.
• Monitor annotation accuracy, throughput, and quality metrics.
• Coordinate reporting and deliverables for Legal, Compliance, Responsible AI, and executive stakeholders.
• Manage annotation workflows and tooling including setting up annotation jobs, running data processing scripts, and testing annotation UIs for quality before job launch.
• Plan capacity requirements to support seasonal increases in review volume.
• Support future expansion into multilingual evaluation programs.
• Identify process improvements that increase efficiency and annotation quality.

• Bachelor's degree or equivalent experience in Linguistics/Psychology.
• 5+ years of experience in program management, operations, quality assurance, data labeling, or related fields.
• Experience leading teams and managing operational workflows.
• Strong organizational and stakeholder management skills.
• Ability to analyze quality metrics and drive continuous improvement initiatives.
• Excellent written and verbal communication skills.
• Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.
• Strong organizational and stakeholder management skills.
• Ability to analyze quality metrics and drive continuous improvement initiatives.
• Excellent written and verbal communication skills.
• Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.

• Experience supporting AI, machine learning, or data annotation programs.
• Familiarity with Responsible AI, compliance, legal review, or governance processes.
• Experience developing operational standards and quality frameworks.
• Experience managing vendor or contractor resources.