Senior Evaluator: Social Work Office of Research & Public Service - UTK

The Senior Evaluator provides day-to-day scientific and operational leadership for the three-year Supporting Families That Support Their Kin project, a statewide randomized controlled trial and implementation and cost evaluation of Tennessee's Relative Caregiver Program (RCP). Reporting to the Applied Research Team Lead and working with the Principal Investigator, Tennessee DCS, seven RCP providers, the Children's Bureau, and the federal Evaluation Technical Assistance provider, the Senior Evaluator leads the power analysis and final evaluation protocol; measure selection; recruitment, randomization, consent, retention, and analysis procedures; implementation, fidelity, qualitative, and cost studies; and integration of administrative, survey, provider, qualitative, and cost evidence. The position safeguards human subjects and study integrity, directs evaluation workflows, supports continuous quality improvement, and leads semiannual and final reporting suitable for federal review and potential Title IV-E Prevention Services Clearinghouse submission.

This is a grant-funded position and is contingent upon the continued funding of the grant.

Responsibilities

Evaluation Design, Governance, and Management:

• Lead development, federal technical-assistance review, revision, and implementation of the Children's Bureau-approved evaluation plan.
• Finalize the theory of change, research questions, power and minimum-detectable-effect analysis, intervention and comparison conditions, eligibility criteria, measures, family-level randomization, consent and retention procedures, urgent-risk safeguards, administrative-data linkage, data security, and prespecified analysis plan.
• Coordinate IRB materials, data-sharing agreements, protocol amendments, study registration and documentation, evaluation timelines, and compliance with PI-18-11 and Clearinghouse standards.
• Provide day-to-day work direction to the Data Analyst and other evaluation contributors.

Impact Study Leadership and Analysis:

• Direct statewide RCT enrollment and centralized randomization monitoring across all seven providers.
• Oversee common baseline and follow-up measures, survey retention, DCS administrative-data linkage, assignment balance, crossovers, emergency overrides, protocol deviations, attrition, and missing data.
• Lead reproducible intention-to-treat and approved secondary analyses of service connection, caregiver economic and housing stability, caregiver mental and emotional health, referrals, satisfaction, family functioning, child safety, permanency, and well-being.
• Interpret uncertainty, sensitivity analyses, and practical significance without releasing premature causal findings.

Implementation, Fidelity, and Qualitative Studies:

• Develop and apply the implementation framework, fidelity standards, dosage measures, provider-assessment procedures, interview/focus-group protocols, and adaptation documentation.
• Evaluate reach, timeliness, service initiation, referral and linkage processes, service duration and intensity, provider and regional variation, rural access, facilitators, barriers, and deviations from the approved model.
• Lead sampling, data collection, analysis, and integration for eight caregiver focus groups across immediate-service and delayed-access conditions and ensure qualitative procedures comply with the approved IRB protocol.

Cost Study, Performance Monitoring, and CQI:

• Specify cost-study methods and oversee collection and analysis of personnel, operating, stipend/service, provider, and implementation costs.
• Estimate total costs, cost per caregiver family and child, variation by family size and service intensity, and relevant comparisons with foster-care costs.
• Establish performance indicators and interpretation rules; review monthly dashboards, quarterly fidelity/data-quality findings, and exception reports; facilitate action-oriented CQI discussions; and ensure corrective actions have responsible owners, deadlines, and documented follow-through without compromising the experimental design.

Reporting, Dissemination, and Sustainability:

• Lead semiannual implementation and evaluation reports and the integrated final evaluation report; ensure conclusions are supported by approved analyses and clearly distinguish descriptive, qualitative, and causal findings.
• Produce or review executive and plain-language summaries, presentations, provider-facing CQI products, methods and analysis documentation, and sustainability-oriented products.
• Present findings to federal, state, provider, caregiver, policy, and professional audiences and assemble a submission-ready evidence package for potential Clearinghouse review after required Children's Bureau review.

Qualifications

Required Qualifications

• Education and/or Experience:
  • Master’s degree in program evaluation, social work, public health, psychology, sociology, education, public policy, statistics, economics, or a related field.
  • At least five years of progressively responsible professional experience conducting program evaluation or applied research beyond coursework, including evaluation design and management, quantitative and qualitative methods, statistical analysis, human-subjects or data-governance requirements, and preparation of reports for funders or decision-makers.
  • Experience must include responsibility for coordinating complex deliverables and clearly communicating methods and findings.
  • A relevant doctoral degree with substantial applied evaluation leadership may substitute for up to two years of required experience.
• Knowledge, Skills, Abilities Knowledge, Skills, Abilities:
  • Knowledge of rigorous program evaluation, experimental and quasi-experimental design, statistical power, causal inference, survey methodology, and mixed-method research.
  • Knowledge of implementation science, fidelity and dosage assessment, qualitative research, cost analysis, performance measurement, and continuous quality improvement.
  • Knowledge of human-subjects protections, IRB processes, data-use agreements, confidentiality, secure administrative-data linkage, disclosure review, and ethical research practice.
  • Advanced skill developing evaluation protocols, research questions, logic models, measures, sampling plans, analysis plans, and federal evaluation reports.
  • Advanced skill applying statistical methods in R, Python, SAS, Stata, or comparable software and interpreting effect estimates, uncertainty, attrition, missing data, and sensitivity analyses.
  • Skill designing and integrating quantitative, qualitative, implementation, fidelity, and cost evidence.
  • Skill managing complex, multi-partner evaluation workplans, risks, deliverables, and documentation within fixed deadlines.
  • Skill communicating technical findings accurately and accessibly in written, oral, and visual formats for technical and nontechnical audiences.
  • Ability to lead collaborative evaluation activities and provide work direction to analysts, researchers, students, consultants, and data-collection staff.
  • Ability to exercise independent scientific judgment, protect study integrity, resolve complex methodological issues, and work effectively with federal and state agencies, providers, caregivers, and community stakeholders.

Applicants must be legally authorized to work in the United States on a full-time basis without need now or in the future for sponsorship for employment-based visa status.

Preferred Qualifications

• Education:

  • Doctoral degree in program evaluation, social work, public health, psychology, sociology, education, public policy, statistics, economics, or a related field.

• Experience:

  • Experience leading randomized or strong quasi-experimental impact studies and complementary implementation, fidelity, mixed-method, or cost evaluations; evaluating child welfare, kinship, or other federally funded human-services programs; working with longitudinal administrative and survey data; supervising or directing analytic work; collaborating with government agencies and community providers; and preparing evidence products for federal review or peer-reviewed publication.

• Knowledge, Skills, Abilities:

  • Knowledge of child welfare, kinship care, Title IV-E, PI-18-11, Prevention Services Clearinghouse standards, or evaluation of federally funded human-services programs.

  • Knowledge of waitlist-control trials, longitudinal child welfare administrative data, service-access outcomes, family well-being measures, and community-engaged research.

  • Skill leading multi-site or statewide impact, implementation, mixed-method, and cost evaluations from planning through dissemination.

  • Skill preparing peer-reviewed publications, federal evidence submissions, preregistrations, reproducible code and documentation, and accessible stakeholder products.

  • Ability to facilitate participatory interpretation and CQI processes with caregivers, youth, providers, agency leaders, and policymakers.

Work Location

This is a remote position. The successful candidate must reside in Tennessee and be able to work remotely from a Tennessee residence. Occasional travel throughout Tennessee may be required.

Compensation and Benefits

• UT market range: MR11
• Anticipated hiring range: $80,000.00
• Find more information on the UT Market Range structure here
• Find more information on UT Benefits here

Application Instructions

Applications will be reviewed as received and interviews can begin as soon as 7 days from posting.

Interested individuals should provide the following:

• Resume
• Cover Letter

About The College/Department/Division

The Social Work Office of Research and Public Service (SWORPS) is an applied research center of the University of Tennessee College of Social Work (CSW). We are a dynamic and close-knit interdisciplinary team of individuals with a shared vision of thriving communities with equitable outcomes for all. We do that by building partnerships that leverage research, technology, and human connection to improve lives in Tennessee and beyond.

Over the past fifty years, SWORPS has worked with local nonprofits such as the United Way of Greater Knoxville, University of Tennessee departments and faculty researchers, state agencies such as the Tennessee Department of Human Services, and federal agencies like the Department of Housing and Urban Development. Our annual portfolio consists of 25-40 sponsored projects spanning a wealth of different content areas, but one common thread -each project contributes one more building block for thriving communities with equitable outcomes for all.

Learn more about SWORPS and some of our programs and resources by visiting our webpage:

The University of Tennessee, Knoxville, has shaped leaders, changemakers, and innovative thinkers since its founding in 1794. The university is home to more than 38,000 students and 10,000 statewide employees—the Volunteers—who uphold the university’s tradition of lighting the way for others through leadership and service.

UT Knoxville offers over 900 programs of study across 14 degree-granting colleges and schools. As Tennessee’s flagship land-grant university, its footprint spans the entire state. The university holds the highest Carnegie classification for research activity and has deep partnerships with industry leaders and the US Department of Energy’s largest multidisciplinary laboratory, Oak Ridge National Laboratory.

The Knoxville campus serves and recruits for UT Knoxville, including the Institute of Agriculture and the Space Institute, as well as the UT Institute of Public Service.

UT Knoxville considers its employees its number one asset. With values that focus on work-life balance, compensation, and innovation leadership, all Vols are supported to advance professionally. Employees have access to career development and coaching, continued education, and an extensive list of development and training possibilities. The Volunteer employee experience implements structures and practices to attract and retain top-tier talent, fostering a strong staff community and supporting a culture of involvement and engagement for everyone.

The university holds a strong commitment to its land-grant mission of learning and engagement, with a tradition of service and leadership that carries that Volunteer spirit throughout the state and around the world. It has been ranked nationally as “Best Employer for New Graduates,” “One of America’s Best Large Employers,” and “Best Workplace for Women,” and has been designated as “Best Place for Working Parents” by Forbes Magazine.

Apply today and join the Tennessee Volunteer community!