Applied AI Lead- Evaluation and Measurement (Hybrid: Dayton, OH or Frederick, MD or Fort Walton Beac
Leonardo DRS
Job Description
Job ID: 115632
Work Location: Dayton, Ohio OR Frederick, Maryland OR Fort Walton Beach, Florida.
Work Schedule: 4/10 schedule. Hybrid (Monday work from home, Tues-Thurs in office. Friday off)
The Leonardo DRS Airborne and Intelligence Systems business is a global leader and strategic partner committed to delivering world-class, full life-cycle defense and intelligence products that protect the security of our nation and our allies. From air combat training to state-of-the-art electronic warfare systems, our technology is deployed by virtually all U.S. military and government agencies around the world.
This is a hybrid position open to candidates who reside in or near Beavercreek, Ohio; Frederick, Maryland; or Fort Walton Beach, Florida.
Job Summary
Prove AI works. Prove what it's worth.
We've used AI before, now we treat it as a first-class citizen at Airborne & Intelligence Systems (AIS). We build and deploy AI across signals intelligence, electronic warfare, air combat training, and the mission systems that connect them. This work shows up on our floor, in the field, and on the company's P&L, while meeting the demanding standards of our defense and commercial customers. If you want your work and your talent to count — this is the place.
Two questions follow every AI system we build: does it actually work, reliably on real jobs and is it actually worth the money we say it is? You own both answers. You design the test every capability must pass before it can ship, and you build the measurement that turns results into honest numbers leadership can trust. You hold the line. Nothing ships until it clears your bar — no test, no launch — and you're deliberately independent from the person who builds and deploys, so the check is real. When there's pressure to over-claim a result, you're the one who holds the number honest.
At the heart of this function is a proprietary platform, the encoded way this business does AI. It takes any employee from a rough idea to a governed, well-formed use case, checks it against the rules, and prepares a recommendation a human decision-maker rules on. You don't just use it; you also build it. The quality bar and measurement logic you design get embedded into the platform itself, so the standard travels with the system to every team that touches it. You encode the acceptance standard and the value math once, and every team inherits both.
This is a high impact, founding individual-contributor seat within the technical core of AI Excellence function with meaningful end to end work ownership. You report directly to top leadership of the function giving you outsize visibility, opportunity for personal growth and direct line to impact. Your peers will be AI leads for Solutions & Delivery/ Infrastructure & Deployment
Job Responsibilities
Primary & Essential Accountabilities
- Define “good enough to ship.” Design the evaluation for each capability — baseline, acceptance criteria, error budget, human-review points — documented before launch. Gate the launch.
- Build the measurement. Stand up the pipelines and record keeping that track what each capability actually delivers.
- Establish the truth. Connect each measurement directly to the source system that holds the ground truth.
- Define what deployed systems must log. Set the trace and telemetry standard that keeps every shipped capability diagnosable and auditable.
- Produce the numbers. Deliver the monthly and quarterly results — forecast versus actual, clean and without double-counting — that the function defends to Finance.
- Keep the front door open. Run the record intake so every AI idea is captured, and nothing happens off the books.
- Scope boundaries. You are not the claim owner, governance or reporting. You build the instrument that proves or disproves the claims in an auditable manner.
Qualifications
- 5+ years in data, ML, or test engineering (regardless of total years exp), including 3+ years in evaluation or model validation, with recent hands-on large-language-model evaluation work.
- Demonstrated bi-directional evaluation rigor: an example where your evaluation stopped something from shipping, and one where it cleared something others doubted. Expect us to probe the methodology of each.
- Built a measurement practice where none existed. The first baselines, the first pipelines, the first dashboard leadership actually used, and it kept producing after you handed it off.
- Data and measurement instrumentation skills: pipelines, dashboards, record-keeping, baselines from source-of-record data.
- A demonstrated record of professional integrity under pressure: you have told someone, especially seniors “the number doesn’t support that claim” and held.
- Bachelor’s degree and 5 years experience or equivalent combination of both. Engineering, Computer Science, Data Science, or a related technical field is preferred
- Five years of relevant experience (or an equivalent combination of experience and training that provides the required knowledge, skills, and abilities
- Proficient technical expertise with demonstrated application
- Excellent interpersonal, leadership, negotiation, communication and writing skills
Preferred Qualifications
- Experience wiring metrics to operational systems of record (not survey- or estimate-based measurement).
- Statistical literacy for probabilistic acceptance: error budgets, confidence bounds, claims that survive challenge.
- Finance-adjacent exposure: cost accounting, benefits realization, or audit (a plus, not a requirement).
- Defense cost-accounting context.
- Active or prior security clearance.
U.S. Citizenship required. This position requires an active DOD security clearance or the ability to obtain such clearance within a reasonable time after commencement of employment.
Taking care of our people is a top priority at Leonardo DRS. We are proud to offer competitive salaries and comprehensive benefits, including medical, dental, and vision coverage, a company contribution to a health savings account, telemedicine, life and disability insurance, legal insurance, and a 401(k) savings plan. We champion wellness programs that focus on physical, emotional, and financial well-being. We develop our talent by offering programs and activities to support career-growth, professional development, and skill enhancement. And we understand there is more to life than work, and the importance of offering flexible work schedules with our 4/10 program, competitive vacation, health/emergency leave, paid parental leave, and community service hours.
*Some employees are eligible for limited benefits only
Leonardo DRS, Inc. and its subsidiaries provide equal opportunities to all employees and applicants for employment and prohibit discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, protected veteran status, or any other characteristic protected by federal, state, or local laws. #AIS
- ...Work Location: Dayton, Ohio OR Frederick, Maryland OR Fort Walton Beach, Florida.... ...Schedule: 4/10 schedule. Hybrid (Monday work from... ...engine room the whole AI function runs on.... ...peers will be AI leads for Solutions & Delivery/Evaluation & Measurement. Job...SuggestedWork at officeLocal areaWork from homeFlexible hours
$132.23k - $176.31k
...trusted network for the AI‑powered world,... ...Role The Senior Lead Marketing Data Platform... ...activation, measurement, and AI-driven decision... ...ecosystem. Evaluate emerging technologies... ...MO MS MT ND NE NM OH OK PA SC SD TN UT VT... ...CA CT DC DE IL MA MD NJ NY TX VA WA Lumen...SuggestedFull timeTemporary work$125.12k - $156.4k
...opportunities for cost-saving measures, quality improvement and incorporating... ...Strong analytical thinking to evaluate options and directions.Strong... ..., and trust.An opportunity to lead and build a business with the... ...ideas presented in writing.Apply general rules to specific...SuggestedTemporary workFor contractorsWork at officeLocal area$105.79k - $141.05k
...trusted network for the AI‑powered world,... ...GTM Growth Strategy Lead develops and... ...accountable for program measurement, optimization, and... ...MO MS MT ND NE NM OH OK PA SC SD TN UT VT... ...CA CT DC DE IL MA MD NJ NY TX VA WA Lumen... ...results will be evaluated on a case-by-case basis...SuggestedFull timeTemporary work$84.63k - $112.84k
...trusted network for the AI‑powered world, connecting... ...We are seeking a Lead Solutions Marketing Manager... ...LA ME MO MS MT ND NE NM OH OK PA SC SD TN UT VT WI... ...states: AK CA CT DC DE IL MA MD NJ NY TX VA WA Lumen... ...Background results will be evaluated on a case-by-case basis....SuggestedFull timeTemporary workRemote work$110 - $150 per hour
Blackbird Health in Frederick, Maryland, is seeking a Lead Neuropsychologist. The role involves leading neuropsychological... ...and experience in neuropsychological evaluations. This remote role offers the opportunity for hybrid work at clinic locations. The salary range...Remote jobHourly pay$104.24k - $130.3k
...Communicate information and ideas in speaking so others will understand.Read and understand information and ideas presented in writing.Apply general rules to specific problems to produce answers that make sense.Identify and understand the speech of another person.Pay...Contract workFor contractors$75k - $100k
...Support Coordinator Hybrid Remote•Frederick, MD ( Job Type Full-time... ...Research Application Evaluation and Support (BRAES)... ...at the client site in Fort Detrick, Maryland, may... ...Notice on Candidate AI Usage Tanaq is committed... ...the interview. To apply for this and other...Full timeContract workTemporary workFor contractorsWork experience placementFor subcontractorCasual workWork at officeLocal areaRemote workFlexible hours- ...Merchandising Lead Sam's Club #6652 5604 Buckeystown Pike Frederick, MD 21704-8312 Shifts may vary Position Summary.... ...scheduling promoting coaching and evaluating associatesReceives and stocks... ...business partners analyzing and applying information from multiple...Shift work
$103.71k - $138.28k
...trusted network for the AI‑powered world,... ...Marketing Manager to lead go-to-market... ...problems and drives measurable business outcomes.... ...with both technical evaluators and executive buyers... ...MS, MT, ND, NE, NM, OH, OK, PA, SC, SD, TN... ...CT, DC, DE, IL, MA, MD, NJ, NY, TX, VA, and...Full timeTemporary workLocal areaRemote work$105.79k - $141.05k
...trusted network for the AI‑powered world, connecting... ...Pricing Monetization Lead with deep expertise in consumption... ...LA ME MO MS MT ND NE NM OH OK PA SC SD TN UT VT WI... ...: AK CA CT DC DE IL MA MD NJ NY TX VA WA Lumen... ...results will be evaluated on a case-by-case basis....Full timeContract workTemporary workRemote workFlexible hours$35.83 - $60.92 per hour
...Location : Maintenance Shop - Frederick, MD Job Type: Full-time... ..., and be a voice for change, apply to Frederick County Government... ...assign, coordinate, review and evaluate work performance; complete performance... ...employ appropriate safety measures Ability to accurately...Hourly payFull timePart timeFor contractorsWork experience placementWork at officeLocal areaMonday to Friday$22 - $30 per hour
...Position Summary... As a Produce Department Lead you will drive sales in your area and... ...other business partners analyzing and applying information from multiple sources... ...Primary Location... 5604 BUCKEYSTOWN PIKE, FREDERICK, MD 21704-8312, United States of America...Hourly payMinimum wageJob sharingFull timeTemporary workPart timeImmediate start- ...supervise Transportation Security Officers and Lead Transportation Security Officers,... ...Position Details Location: Point Of Rocks, MD Employer: Transportation Security... ...section of the active announcement before applying. General Conditions of Employment Must...Permanent employmentFull timeTemporary workImmediate startTrial periodFlexible hoursShift workNight shiftWeekend workDay shiftAfternoon shiftEarly shift
$25 - $33 per hour
...Job Title: Lead Commercial Carpenter Job Description... ...facility at a major applied physics laboratory,... ...position based out of Frederick, MD. Pay and Benefits... ...Intelligence (AI): We may use Artificial... ...sourcing, screening, and evaluating candidates. AI helps assess...Weekly payPermanent employmentContract workTemporary workFor subcontractorLocal areaShift workDay shift$130k - $160k
...accommodation or an alternative application process. Lead Scientist Full Time Professional Frederick, MD, US 2 days ago Requisition ID: 1012 Salary... ...lignocellulosic or polymer chemistry. A solid grounding in applied statistics and design of experiments - with the...Full timeWork at officeAfternoon shift$125.1k - $225.2k
...Fieldable Analytics to lead bioinformatics,... ...genomic analysis, AI and machine-learning... ...Lead the testing and evaluation of bioinformatics... ...analytical workflows.Apply AI or machine-... ...primarily on-site at Fort Detrick and is contingent... ...: US - MD, FrederickType: Full...Full timeContract workFlexible hoursAfternoon shift$56.78k - $99.29k
...Rock Island, IL Aberdeen Proving Ground, MD Fort Detrick, MD Fort Meade, MD Prince George'... ...allegiance to the U.S. Duties Help Lead a patrol team of three or more officers,... ...accrual rate may be authorized. Who May Apply: US Citizens In order to qualify, you must...Permanent employmentFull timeWork experience placementLocal areaRemote workRelocationTrial periodRelocation packageShift workRotating shiftWeekend workAfternoon shift- ...Lead Transportation Security Officer Opportunities in Frederick Lead Transportation Security Officers—or LTSOs—help keep airport... ...Position Details Location: Frederick, MD Employer: Transportation... ...employees. Review the “Who May Apply” section of the official vacancy...Full timeFlexible hoursShift workNight shiftWeekend workDay shiftAfternoon shiftEarly shift
- ...NBACC) is a unique facility located at Fort Detrick in Frederick, MD, dedicated to defending the nation... ..., conducting regular performance evaluations, and developing growth plans to enhance... ...an interim secret clearance leading to a top-secret clearance and suitability...For contractorsInterim roleWork at office
$41.25k - $68.75k
...MD - Frederick (21702) Full time R238965 Position Overview At PNC, our people are our... ...contribute to the company's success. As a Teller Lead within PNC's Retail Branch organization,... ...page ( . If you're not ready to apply yet, or you'd like to learn more about...Full timeTemporary workPart timeWork experience placementWork at office$99.5k - $165.9k
...business including accountants, attorneys and other referral sources.Evaluate credit information and requests and recommend appropriate loan... ...inform their specific compensation.LocationFrederick, Maryland, United States of AmericaSummaryLocation: Frederick, MDType: Full timeFull time- ...team member for our store located at 2060 Yellow Springs Road in Frederick, MD. If you are passionate about providing excellent customer... ...looking for an opportunity to grow your career with a company that values its team members, we encourage you to apply today. Food Lion
$110 - $150 per hour
...pediatric mental health care. Lead Neuropsychologist... ...with opportunity to be hybrid at any of our clinic... ...comprehensive neuropsychological evaluations for children with... ...on interpreting and applying neuropsychological findings... ...about the use cases of AI and remote models to...Remote work$75k - $80k
...Overview Lead Registered Dietitian for Frederick, MD and Surrounding Areas Join Healthcare Services Group (HCSG) as a Lead Registered Dietitian, overseeing... ...care and be part of a supportive and dedicated team, apply now and start making a difference at HCSG! Additional...Full timeLocal areaShift workDay shift$73.82k - $92.27k
...On-Site Frederick, MD Full time JR65930 At Saputo, we bring... ...production run assignments. Evaluates needs in production for daily... ...code of ethics and actively apply the Saputo values. Performs... ...Manufacturing Practices. Experience leading and following Safety/Security...Full timeContract workWork at officeFlexible hoursShift workNight shift- ...on, customer-focused Store Manager Trainee based out of our Frederick, MD location. As a premier Parker Hannifin distributor since 195... ...multiple customers and tasks smoothly Strong math and measurement skills, including comfort with standard and metric conversions...Permanent employmentTraineeshipMonday to FridayFlexible hours
- ...CoordinatorLocation: Williamsport, MD, United States; Martinsburg, WV, United States; Frederick, MD, United StatesWorkplace: On... ....Participating in contractor evaluation, project review meetings, and... ...ethics standards. Consistently applies ethic guidelines in working with...Contract workFor contractorsH1bWork at office
- ...Filling Team Supervisor WM Supercenter #2233 7400 Guilford Dr Frederick, MD 21704-5183 CP-2233-9039 $20.00 - $33.00/hr* Full time Shifts... ...are completely paid for by Walmart. Eligibility requirements apply to some benefits and may depend on your job classification and...Hourly payMinimum wageFull timeTemporary workPart timeShift workWeekend workAfternoon shift
$88.86k - $118.48k
...is the trusted network for the AI‑powered world, connecting... ...us today. The Role The Lead Logistics Manager will manage... ...video interview immediately after applying. This will include a set of questions... .... Background results will be evaluated on a case-by-case basis....Full timeContract workTemporary workFor contractorsImmediate start
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Applied AI Lead- Evaluation and Measurement (Hybrid: Dayton, OH or Frederick, MD or Fort Walton Beac. Be the first to apply!






