Senior Data Analyst
RightsLine
Reporting to the SVP Engineering, the Senior Data Analyst - LLMOps will work with the Application Development team to own the observability, evaluation, and quality measurement of Rightsline's LLM-powered features. This is a senior individual-contributor role in which you will instrument the LLM stack end to end - traces, prompts, tool calls, latency, token spend, and output quality - build the evaluation suites that tell us whether a change made the product better or worse, and turn that telemetry into decisions the engineering and product teams act on. You will work across a SaaS application, a wide range of third-party tools, an AWS-hosted infrastructure, and an increasingly agentic development workflow. The position is based in North America, has no direct reports, and partners closely with the wider engineering, product, and data teams. Mission & Impact The Senior Data Analyst - LLMOps makes the behavior of Rightsline's AI features measurable. Quality questions about prompts, retrieval, and agent workflows are too often answered with intuition and spot checks; you will replace that with tracing, offline and online evaluations, regression suites, and dashboards that show quality, cost, and latency trends over time. You will be working in a collaborative environment with a team who shares your passion for building something impactful, and success in the role looks like engineers and product managers reaching for your evaluation results and dashboards before they ship - and trusting what they see. What You Will Own This role's work spans five areas, each described below. Observability and Telemetry Instrument Rightsline's LLM and agent workflows end to end with tracing and structured logging, using tools such as LangSmith, Langfuse, or equivalent Define and maintain the metrics that describe AI system health - latency, token and cost per request, error and fallback rates, tool-call success, and retrieval hit rates Build and maintain dashboards and alerting so quality and cost regressions surface in hours rather than in customer tickets Evaluation and Quality Design and maintain offline evaluation suites - golden datasets, scoring rubrics, and LLM-as-judge grading - for prompts, retrieval pipelines, and agent flows Run online evaluations and A/B experiments on production traffic, and report results with enough rigor to support a ship / no-ship decision Track quality across model, prompt, and retrieval changes, and maintain the benchmark history that shows whether the product is actually improving Partner with engineers on human review and annotation workflows, including labeling guidelines and inter-rater agreement Data Analysis and Reporting Build and maintain the pipelines that move trace, evaluation, and usage telemetry into the analytics stack using SQL and Python Analyze how customers actually use AI features - adoption, drop-off, and failure patterns - and translate findings into prioritized recommendations Produce recurring reporting on AI quality, cost, and usage for engineering, product, and leadership audiences Cost, Reliability, and Troubleshooting Monitor token spend and model usage, and identify where prompt, model, or caching changes cut cost without hurting quality Investigate quality incidents - bad or unsafe outputs, hallucinations, tool failures, latency spikes - from trace to root cause Follow and help extend Rightsline's development best practices and standards in the pipelines, queries, and evaluation code you ship Team Contribution Attend and contribute to team meetings, and present telemetry and evaluation findings in a way non-specialists can act on Raise the team's LLMOps literacy - help engineers instrument their own features, write their own evals, and read the dashboards What You Will Bring 5+ years in data analytics, analytics engineering, or data science, including hands-on work with LLM-based systems in production Strong SQL and Python (including pandas and notebook-based analysis), with the discipline to build repeatable pipelines rather than one-off queries Hands-on experience with LLM observability and evaluation tooling such as LangSmith, Langfuse, LangChain/LangGraph, or equivalent Practical understanding of LLM application architecture - prompting, retrieval-augmented generation, tool calling, and agent workflows - and how each of them fails Experience designing evaluation methodology: golden datasets, scoring rubrics, LLM-as-judge grading, human annotation, and A/B testing on production traffic Working knowledge of experiment design and statistics, enough to say confidently whether a difference is real Ability to work independently, meet deadlines, and be accountable for your work, while operating within established standards Excellent written and verbal communication skills, and a strong sense of ownership, urgency, and initiative - you can explain a quality regression to an engineer and to an executive in the same afternoon BS in Computer Science, Statistics, Data Science, or equivalent knowledge Preferred Qualifications: Experience with AWS services such as Lambda, SQS/SNS, API Gateway, Step Functions, and Amazon Bedrock Familiarity with dbt, Airflow or a similar orchestrator, Git, and BI tools such as Looker, Power BI, or Tableau Working knowledge of MS SQL, Redis or another NoSQL store, and vector stores used for retrieval Exposure to model providers and platforms (Anthropic, OpenAI, Amazon Bedrock), guardrail and PII-redaction tooling, and prompt versioning and release workflows Experience with coding agents or AI developer tools (Claude Code, GitHub Copilot, Cursor, or similar) in day-to-day analysis work Our Core Values At Rightsline, we look for these qualities in every hire, and we hold ourselves to them every day. Be Curious and Smart Being smart isn't about having all the answers - it's about asking the right questions, seeking out new ideas, and never assuming you already know enough. Embrace Change This industry moves fast and so do we. We don't resist change - we lean into it. That means staying flexible, letting go of what isn't working, and treating uncertainty as opportunity rather than obstacle. Show Up Showing up means being present - not just physically, but mentally and emotionally. Your teammates count on you. Your customers count on you. We do our best work when everyone is fully engaged and accountable to each other. Get it Done Ideas are easy. Execution is the job. We hold ourselves to what we commit to, push through the hard parts, and measure success by our actual accomplishments. Be Humble Nobody here has a monopoly on good ideas. We listen more than we talk, share credit freely, and check our egos at the door. The best outcome matters more than whose idea it was. Equal Employment Opportunity Rightsline is an equal opportunity workplace. All candidates will be afforded equal opportunity through the recruiting process. We do not discriminate against any employee or applicant for employment because of race, color, sex, age, national origin, religion, sexual orientation, disability, gender identity and/or expression. We are dedicated to growing a diverse team of highly talented individuals and creating an inclusive environment where everyone feels empowered to bring their authentic selves to work. If you are contacted for an interview and require accommodation during the interviewing process, please let us know. Apply Today If you want to join a company that strives for a mission, purpose and making an impact, we encourage you to apply today. RightsLine
$90k
DescriptionData Analyst, User & Content InsightsRole Summary: The Data Analyst, User & Content Insights focuses on uncovering relationships between user behavior, content metadata, and engagement needs across Internet Brands’ platforms. This role analyzes patterns in how...SeniorTemporary workWork at officeFlexible hoursShift work$102k - $138k
...Strategic Research and Analytics team helps shape how we use data to inform strategies and investments across Marketing and... ...Corporate Responsibility functions at Intuit. We are looking for a Senior Staff Data Analyst to set technical direction and own the measurement strategy...SeniorFull timeWorldwide$32.6 - $59.5 per hour
...programs and national challenges. The Acquisition Planning and Decision Analytics (APDA) Department is seeking an Acquisition Data Analyst Specialist to join a diverse, innovative, and highly productive team of acquisition, systems, digital, and data engineering...SuggestedHourly payFull timeWork at officeImmediate startRemote workRelocation packageFlexible hours$28 - $38 per hour
...position onsite at Amca's headquarters with the opportunity for extensions. Responsibilities Obtain and organize Technical Data Packages from government and commercial sources. Generate CAD models and drawings based on source material and engineering requirements...SuggestedHourly payPermanent employmentContract workTemporary work- ...People Data AnalystEl Segundo, California, United StatesCHAOS Industries is redefining modern defense with a multi-product portfolio... ...looking for a highly analytical and mission-driven People Data Analyst to join our HR team. In this role, you will leverage data to inform...SuggestedWork experience placementCasual workRelocation package
$100k - $125k
...individuals who want to learn, play, and grow with us. Ready for a new challenge?Summary of PositionNexon America is looking for a Data Analyst to provide in-depth data mining and analyses to multiple departments, with a focus on understanding our players and identifying...Full timeTemporary workWork at officeFlexible hoursShift work- ...GeoProfessional to assist geotechnical and geological projects in California. You will support geoprofessionals in collecting and interpreting data, contribute to field investigations, laboratory tests, calculations, and reports, and help with project documentation and client...
- ...Description Job Description Description: On-site in either West Hollywood, CA or New York, NY Our client seeks a Senior Data Operations Analyst to maintain and improve a centralized platform for global content and campaign planning. You will embed with...SeniorHourly payContract workLocal area
- ...brave. ABOUT THE ROLE David&Goliath is looking for a Senior Analyst to join our Analytics team in a highly integrated, client-facing... ..., organic social, CRM, website behavior, and other available data sources work together across the full customer journey. The...SeniorFull time
$27 - $29 per hour
Remote Data Analyst CA Only Our client, a leading organization in the restructuring and legal support industry, is seeking a dedicated and detail-oriented Remote Data Analyst CA only to join their team. As a Remote Data Analyst CA only, you will be part of the Corporate...Remote jobWeekly payContract workTemporary workWork at officeMonday to FridayShift work$110k - $120k
Maintenance Data Analyst Join Entarian as a Maintenance Data Analyst! Entarian is seeking a highly skilled Maintenance Data Analyst to support our meshONE-T program by analyzing maintenance data, identifying trends, and providing actionable insights. This position plays...Full timeTemporary workWork experience placementWork at officeMonday to FridayFlexible hours- Job Code - Data Analyst Work Location - El Segundo CA 90245 (remote role but prefer local to be able to come in office) Total Hours - Contract initially up until 12/31/22 Description SQL and database digging (role would be focused on data support for upcoming product...Contract workWork at officeLocal areaRemote work
- ...journey. Be human. The Opportunity NRG is seeking a Research Analyst or Senior Analyst to join our Technology research practice. In this... ...quantitative and qualitative research; if you enjoy visualizing data and discovering insights with confidence; and are looking for...SeniorFull timeWork experience placementInternshipSummer holidayWork at officeLocal areaRemote workFlexible hours
$165.75k - $224.25k
Structural & Payload Design Engineer (Senior or Principal) Company: The Boeing Company Boeing Defense, Space & Security (BDS) is seeking a Senior or Principal Structural & Payload Design Engineer (Level 5 or 6) in El Segundo, CA. At Boeing, we are all innovators...SeniorPermanent employmentInterim roleRelocationVisa sponsorshipWork visaFlexible hoursShift work$55.35 - $87.39 per hour
Description Under the supervision of the Supervisor or Management, the Technical Specialist plans, directs and coordinates all technical activities performed in the division of technical expertise/practice. The Technical Specialist is responsible for all quality and...SeniorMinimum wageFull timeLocal areaShift work$176.8k - $239.2k
Satellite Payload Systems Engineer - Senior or Principal Company: The Boeing Company Boeing Defense, Space & Security (BDS) Space... ...(including Manufacturing Technology), Computer Science, Data Science, Mathematics, Physics, Chemistry or non-US equivalent qualifications...SeniorPermanent employmentWork experience placementRelocationVisa sponsorshipWork visaFlexible hoursShift work- ...Job Description Job Description Description: Job Title: Senior Technician Locations: Los Angeles (El Segundo) Status: Full-Time | Hourly What You'll Do As a Senior Technician, you’ll serve as the on-site leader, ensuring installations are completed on...SeniorHourly payFull timeFor contractorsApprenticeship
- ...Senior Sales Executive Job Duties: Enhances staff accomplishments and competence by planning delivery of solutions; answering technical and procedural questions for less experienced team members; teaching improved processes; mentoring team members. Identifies business...Senior
$132.5k - $302.4k
...Digital Core, Finance, Industry and Enterprise, Song, Supply Chain and Engineering, and Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We serve approximately 9,000 clients and generated approximately $74 billion in FY26 revenue....SeniorWork experience placementLive inWork at officeLocal area$100.3k - $175.9k
...The Aerospace Corporation in El Segundo is hiring a Surface Scientist (Senior Member of Technical Staff) to conduct vital research and technology development in surface science techniques, addressing challenges for spacecraft. The ideal candidate will have a PhD and extensive...Senior- National Research Group (NRG) seeks a motivated Research Analyst or Senior Analyst to join our Technology research practice. You will collaborate... ...the future of technology, working across survey design, data analysis, and storytelling to deliver actionable insights. You...Remote job
$95k - $115k
...entertainment, this is the opportunity for you! Job Description The Opportunity: Mattel Games Studios is seeking an experienced Data Analyst, LiveOps to take a key role in driving meaningful insights and providing recommendations to impact our business growth for a...Local area- CHAOS Industries is hiring three Recruiting Managers to lead a 4-6 person recruiting team, reporting to the Head of Recruiting. You will transform business priorities into hiring plans and coach recruiters to act as trusted advisors to hiring managers. This hands-on leadership...Senior
- Radiant seeks a Staff Loads & Dynamics Engineer in El Segundo, CA to own static, dynamic, and transient loads for reactor systems and transport configurations. You will perform modal, random vibration, shock, and fatigue analyses, and execute explicit dynamics simulations...Senior
- K1 Investment Management is seeking an Executive Assistant to support 3-5 Executives in Manhattan Beach, CA. The role demands high organization, initiative, and discretion to manage diverse responsibilities while aligning with the firm’s values of Passion, Persistence, ...Senior
- Core4ce, located in El Segundo, California, seeks an experienced Geodetic Scientist to support the measurement, analysis, and monitoring of Earth geophysical properties. You will apply advanced geodesy and geophysics to evaluate the Earth's size, shape, movement, and variations...Senior
- Analyzes customer and system requirements to develop basic architectural approaches and detailed specifications for various electronic products. Develops high-level and detailed designs consistent with requirements and specifications. Validates designs through various ...Senior
- Circadia Health in El Segundo, CA seeks a Senior Scientist, HEOR/RWE to own outcomes research, including study design, analysis, and dissemination... ...on ROI and cost-avoidance in senior care using large healthcare data. The role emphasizes publication, executive storytelling, and...Senior
- Restaurant Senior Kitchen ManagerAre you a hardworking, service-minded leader with a real passion for the hospitality industry? Are you looking to take a step towards building your restaurant manager career, instead of just working a job? We need extraordinary leaders...Senior
$71k - $88k
...Job Description The Opportunity: The Product Master Data Management Analyst will be part of the Global Supply Chain Master Data Management team and will report to the Manager, Master Data Management. This role is responsible for the creation, maintenance, validation...Full timeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Analyst. Be the first to apply!
- neuroscience data analyst El Segundo, CA
- data center analyst El Segundo, CA
- data protection analyst El Segundo, CA
- data analyst supply chain analytics El Segundo, CA
- data warehouse analyst El Segundo, CA
- data analyst bank El Segundo, CA
- remote data analyst part time El Segundo, CA
- data analyst El Segundo, CA
- data visualization analyst El Segundo, CA
- entry level data analyst no experience El Segundo, CA






