AI Data Science Contractor

Job Description

Job Description

About the Job:
The AI Data Science Contractor will help monitor performance, risk, and controls across the Contact Center and Integrated Channel GenAI portfolio, and keeps the supporting evidence current for model risk review and regulatory examination. The AI Data Science Contractor extends the team's capacity under the direction of the team lead. The contractor builds and runs reusable testing harnesses, evaluation automation, monitoring, and documentation artifacts that help use cases move from intake to production and keep live models monitored.

Essential Job Functions

Build Reusable Testing & Evaluation Capabilities under supervision and day-to-day direction from the Contact Center AI model risk lead
• Build reusable SQL/BigQuery queries that pull real transcripts and call logs for testing.
• Develop a synthetic data generator to fill gaps, including edge cases, non-English callers, and PII scenarios.
• Build an audio testing harness with a simulated customer agent and noise, interruption, and crosstalk scenarios, run multiple times per scenario with latency and turn-taking metrics (Triage Agent, Agent Assist, Multimodal Conversational AI, Voice concierge - intelligent routing for contact center and etc.).
• Build a text/classification harness that scores precision, recall, accuracy, and slot-extraction accuracy as batch runs against a fixed, versioned test set that can be rerun after any change (Call & Chat Summarization, Unstructured to Structured Data Categorization, Voice concierge - intelligent routing for contact center, Triage Agent and etc.).
• Build LLM-as-a-judge evaluators with a scoring rubric per use case, benchmark them against human ratings, and support scheduled recalibration.
• Develop agentic AI automation workflows (e.g., Antigravity-based subagents) that streamline testing and documentation preparation.
• Help embed testing capabilities into the model development lifecycle.

Support Monitoring, Model Risk Documentation & Controls under supervision and day-to-day direction from the Contact Center AI model risk lead
• Apply the standard monitoring template (accuracy, refusals, PII leakage, escalation and override rates, with thresholds set by tier) to live models and support the semiannual live-model monitoring cycle.
• Populate the standard testing and monitoring sections of model documentation, with results generated automatically from the harnesses.
• Draft architecture sections covering data flow, redaction, logging, prompt and version control, and routing and handoffs.
• Prepare supporting evidence and statistical test results for Model Risk and regulatory reviews, under the direction of the team lead.

Enable Analytics, ML & Technical Scale (Request-Based) under supervision and day-to-day direction from the Contact Center AI model risk lead
• Conduct sample-based data analysis supporting post-implementation performance and QA reviews of live GenAI models, including validation of data visualization dashboard logic and data quality and etc..
• Support ML development on OPD, integrated channel data asset, such as customer-level scoring proofs of concept, and work with OPD data engineers on deployment patterns.
• Assist with virtual machine and environment setup, troubleshooting, and agentic AI tool configuration.

Required Qualifications
• Bachelor's degree in electrical engineering, computer science, data science, or another computer science field;
• 2 years of hands-on data science or ML/LLM/DL engineering experience.
• Strong proficiency in Python (Pandas, NumPy, Pytorch, Scikit-learn) and SQL/BigQuery.
• Proficiency with the Linux terminal and command line, including running scripts and working in remote or virtual machine environments.
• Experience building supervised/unsupervised ML/LLM/DL models, including classification/prediction metrics and statistical testing.
• Experience with LLM APIs (Gemini, GPT, Claude and etc.) and prompt engineering.
• Experience with Git-based version control.
• Strong technical writing
• Ability to work on-site in Brooklyn/Cleveland, OH on a hybrid schedule.

Preferred Qualifications
• Master's degree in electrical engineering, computer science, data science, or another computer science field;
• Strong proficiency in computer algorithms and data structures, with the ability to write efficient, well-tested code without assistance of AI code assist.
• Experience with cloud infrastructure such as GCP, Azure, AWS and etc..
• Experience with deep learning architecture such as the Convolutional Neural Network.
• Experience with full-stack software engineering development
• Experience with agentic AI frameworks or CLI-based AI tooling.