Senior Data Scientist II (Raleigh)

Role: Senior Data Scientist II

Location: Raleigh, NC (onsite arrangement) or Remote (EST only)

Duration: 1 year, potential for extension/FTE

Job Description:

Position Summary

*** Legal & Professional is seeking a hands-on Data Scientist to develop and evaluate AI solutions for legal content workflows. The role focuses on large language models, natural language processing, and reliable data pipelines. Experience with traditional machine learning is a strong plus.

The successful candidate will collaborate with engineers, product stakeholders, and legal subject matter experts to translate business requirements and editorial feedback into measurable improvements. This role requires strong Python skills, sound experimental judgment, and the ability to independently investigate issues and deliver working solutions.

What are the top 3 must-have skills?

Applied LLM development: demonstrated experience building and evaluating workflows involving prompts, retrieval, and structured generation.

Python and data engineering: ability to independently build, debug, and maintain reliable data-processing and model-integration code.

Experimental design and evaluation: ability to compare baselines, analyze errors, validate improvements, and communicate actionable findings.

Key Responsibilities

Develop and improve LLM-based workflows for information extraction, classification, legal summarization, and content generation.

Design and evaluate prompts, retrieval-augmented generation (RAG), and multi-step agent workflows.

Build reliable data-preparation pipelines, including parsing, transformation, validation, caching, and recovery from interrupted processing.

Create representative evaluation datasets and conduct controlled experiments, error analysis, and regression testing.

Work with subject matter experts to assess factual accuracy, completeness, source support, and consistency, and translate feedback into generalizable improvements.

Diagnose problems across data, retrieval, model behavior, and application logic.

Balance output quality, latency, and cost when comparing technical approaches.

Maintain tested, version-controlled code and clear documentation to support reproducibility and team handoffs.

Required Qualifications

Bachelor’s degree or higher in computer science, Data Science, Statistics, Mathematics, or a related quantitative field, or equivalent practical experience.

At least 3 years of relevant experience in data science, machine learning, or NLP, including hands-on experience developing LLM-based applications.

Strong Python programming skills and experience with SQL and structured or semi-structured data, such as JSON and XML.

Practical experience with prompt design, structured outputs, retrieval, and systematic LLM evaluation.

Understanding of statistical evaluation, train/validation/test separation, data leakage, and appropriate performance metrics.

Experience using Git, debugging existing code, and writing automated tests.

Ability to communicate findings clearly, incorporate feedback, and proactively raise delivery risks.

Preferred Qualifications

Experience with traditional machine learning, including feature engineering, classification, tree-based models, and imbalanced datasets.

Familiarity with scikit-learn, PyTorch, or comparable frameworks.

Experience fine-tuning transformer models for classification, information extraction, or sequence labeling.

Familiarity with AWS, particularly S3 and EC2, and Linux or GPU-based training environments.

Experience with legal documents, editorial workflows, or other accuracy-sensitive domains.