AI Evaluation Analyst
VirtualVocationsLowell, Massachusetts, United Statesjob_description.job_card.variable_hours_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
serp_jobs.job_alerts.create_a_job
Ai Analyst • Lowell, Massachusetts, United States
Job_description.internal_linking.related_jobs
At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
We are currently seeking an experienced AML professional to join an investment banking client based in Central Boston.This is an exciting opportunity to support anti-money laundering efforts across...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Be among the first 25 applicants.This range is provided by Mercor.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Mercor connects elite creative...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days 
Intelligence Analyst Job Overview : We are currently seeking a highly motivated and detail-oriented individuals to join our team.
You will play a crucial role in supporting our decision-making proces...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Children’s Services of Roxbury (CSR) seeks one to two paid graduate student interns for the.Interns will support one of two focus areas—.
Program Data Integrity & Systems Development...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
At Spindrift, we’re making every beverage a positive force of nature.Founded in 2010, we believe the best flavors come directly from nature.
That’s why every Spindrift beverage is made the hard way—...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Endless Opportunities for Personal Growth in a Growing Company.Buyers Edge Platform team members work on projects with brands throughout our Digital Procurement Network that keeps life interesting...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
SharkNinja is a global product design and technology company, with a diversified portfolio of 5-star rated lifestyle solutions that positively impact people’s lives in homes around the world.Powere...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Primary Skills : Excel (Expert), Cost Modeling (Experienced), Forecasting (proficient), Project Management (Intermediate), SQL (Knowledge).
Duration : 5 months with possible extension.Location : North ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Principal, Data Strategy, Analytics & AI Practice.Principals are responsible for independently driving client and business development at the company.
They must effectively identify potential client...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Primary Skills : Machine Learning (Expert), Generative AI( Proficient), Statistical Modeling, Python (Expert), SQL (Expert), (Intermediate), PyTorch ((Intermediate).
Duration : 9 months with possible ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 
We are seeking a detail-oriented and analytical Research Analyst to join our team.The ideal candidate will possess strong quantitative and qualitative research skills, exceptional critical thinking...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
When you join the growing BILH team, you're not just taking a job, you're making a difference in people's lives.Supervisor, Epic / Vended Applications.
The Supervisor, Epic / Vended Applications, is a m...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
AI-Enabled Global Study Data Leader.Are you ready to shape the future of medicine? The race is on to speed up drug discovery and development to find answers for patients and their families.Your ski...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
At Evolv, we’re transforming public safety through bleeding edge security technology and AI, and we’re growing fast.As a high growth company with an inspiring mission to create safer experiences wh...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Applied Researcher I (AI Foundations).At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.
For years, Capital One has been leading the industry in using ma...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
Job Title : AI-Enabled Global Study Manager Location : Morristown, NJ Cambridge, MA About the Job Are you ready to shape the future of medicine? The race is on to speed up drug discovery and d...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Primary Skills : Machine Learning (Expert), Generative AI( Proficient), Statistical Modeling, Python (Expert), SQL (Expert), (Intermediate), PyTorch ((Intermediate).
Duration : 9 months with possible ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
- serp_jobs.job_card.promoted
Applied Researcher II
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
AML Analyst
G MASSBoston, MA, USserp_jobs.job_card.permanent
Hireio, Inc. is hiring : AI Communication Quality & Culture Evaluator in Boston
MediabistroBoston, MA, United Statesserp_jobs.job_card.full_time
Mercor is hiring : Audio Evaluation Expert - Fully Remote in Boston
MediabistroBoston, MA, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Intelligence Analyst
US ArmyBoston, MA, United Statesserp_jobs.job_card.part_time +1
- serp_jobs.job_card.promoted
Research and Evaluation Intern
Childrens Services Of RoxburyRoxbury, MA, USserp_jobs.job_card.full_time
Sensory Analyst
SpindriftNewton, MA, USserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
VP of AI Strategy and ImplementationLake Worth, FL; Remotely based; Waltham, MA
Buyers Edge PlatformWaltham, MA, USserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Lead Consumer Insights Analyst
SharkNinjaNeedham, MA, United Statesserp_jobs.job_card.full_time
Data Analyst : 25-05026
Akraya IncNorth Reading, Massachusetts, United Statesserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
Principal, Data Strategy, Analytics & AI PracticeBoston, Massachusetts
PutnamBoston, MA, USserp_jobs.job_card.full_time
Applied Scientist : 25-04992
Akraya IncBoston, Massachusetts, United Statesserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
Applied Researcher I
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
Research Analyst
Stella Contracting, IncBoston, Massachusetts, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
Supervisor, Epic Analyst (Research Certified)
Beth Israel Lahey HealthBoston, MA, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI-Enabled Global Study Data Leader
SanofiCambridge, MA, USserp_jobs.job_card.full_time
Business Intelligence Data Analyst
Evolv Technologies Holdings, Inc.Waltham, MA, USserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Applied Researcher I (AI Foundations)
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
AI-Enabled Global Study Manager
SanofiCambridge, MA, United Statesserp_jobs.job_card.full_time
Applied Scientist : 25-04992 (No C2C)
Akraya IncBoston, Massachusetts, United Statesserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply