Senior Data Scientist

Full-time Description Position Summary The MFM IT team has a long history of delivering and maintaining core business systems in a mid-sized Property & Casualty insurance environment. As we expand our development team and adopt new AI development tools, we are making a deliberate investment in building this capability correctly from the start. As we begin to build agentic AI capabilities within our enterprise system, we are looking for an experienced engineer with strong knowledge of agentic AI systems to help guide this effort. The Senior Agentic AI Specialist will work closely with MFM's Principal Architect, who leads overall platform architecture, and partner with senior developers to design, implement, and improve agent-based solutions. This role will bring specialized expertise in areas where we are still developing depth, including agent orchestration, retrieval pipelines, LLM evaluation, and prompt design. This is an ongoing platform effort, not a one-time project. The work will evolve over time as we expand capabilities, integrate new tools, and improve system quality. Our core systems are built on a Microsoft/.NET stack. As we develop our agentic AI capabilities, we expect to leverage cloud-based tooling (likely within Azure) where appropriate. We are pragmatic about using the right tools for the problem, while maintaining alignment with our existing platform and operational requirements. At Midwest Family we value practical experience, clear thinking, and the ability to apply these skills in a real-world environment. Application Requirements Cover Letter and Resume: Please include the following in your cover letter A description of an AI or agent-based system you have worked on, including your role and how you evaluated quality Your approach to reducing incorrect or unsupported outputs An example of how you supported other developers or teams in adopting AI capabilities Your physical remote work location state Salary expectations Responsibilities: Platform Architecture & Evolution : Provide input into AI-related architecture decisions. Evaluate tools, models, and approaches, and make practical recommendations based on trade-offs and outcomes. Agent Lifecycle Management: Design, build, and maintain agents. Define prompts and configurations, support routing decisions, and monitor performance in production. RAG Pipeline Engineering: Improve retrieval quality by working with chunking strategies, embeddings, search methods, and metadata filtering. Support onboarding of new knowledge sources. Quality & Hallucination Reduction: Help reduce incorrect or unsupported outputs through system design, testing, and monitoring. Apply practical approaches across prompts, retrieval, and model behavior. Evaluation & Testing: Develop and maintain evaluation approaches to measure output quality, including relevance and accuracy. Support regression testing and identify failure patterns. Prompt Engineering: Create and maintain prompts for different agents. Diagnose unexpected behavior and develop reusable patterns where appropriate. Tool Integration: Support integration with internal systems and APIs. Help manage tool access and ensure reliable execution and error handling. Observability & Operations: Contribute to logging, tracing, and monitoring. Support analysis of system behavior, cost, and performance in production. Governance & Configuration: Work with configuration across agents, prompts, and models. Ensure changes are version-controlled and reviewed. Developer Support: Work with other developers to share knowledge, review designs, and establish consistent approaches. Research & Evaluation: Stay current with new models and tools. Evaluate them in a structured way and recommend adoption where appropriate. Requirements Education & Certification College degree or Programming Certification (preferred) 5+ years of equivalent work experience Required Experience Experience building or working with AI/LLM-based systems in a production or near-production environment Strong understanding of agent-based patterns (ReAct, tool usage, etc.) Experience with retrieval-based systems (RAG), vector stores, or search pipelines Familiarity with evaluating non-deterministic system outputs Experience improving system quality through testing and iteration Strong software engineering skills required. Experience with Python, TypeScript, or C# preferred Ability to work across technologies and integrate AI solutions into an existing enterprise environment Preferred Experience Experience working with multiple LLM providers Familiarity with emerging tools and frameworks in the agentic AI space Experience integrating with enterprise systems and data sources Experience mentoring or supporting other developers Personal Attributes Strategic thinker with strong analytical and problem-solving skills Technically proficient with broad systems knowledge Excellent interpersonal, written, and verbal communication skills Self-directed, collaborative, and highly organized Customer-oriented and able to navigate high-pressure environments Flexible and adaptable in regards to learning and understanding new technologies. Exceptional organizational skills and ability to keep track of multiple projects at a time. Ability to conduct research into software-related issues and products. Technically proficient. Exceptional attention to detail. Ability to work both independently and in a team-oriented, collaborative environment. Work Conditions Remote work environment following core hours 8AM-5PM Central Time Overtime may be required to meet project deadlines or monitor system processes. Sitting for extended periods of time. Dexterity of hands and fingers to operate a computer keyboard, mouse, and other devices. Minimal travel will be required for the purpose of off-site software training and team meetings. Physically able to participate in virtual and in-person training sessions, presentations, and meetings.