AI Evaluation Analyst
VirtualVocationsLowell, Massachusetts, United Statesjob_description.job_card.variable_hours_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
serp_jobs.job_alerts.create_a_job
Ai Analyst • Lowell, Massachusetts, United States
Job_description.internal_linking.related_jobs
At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Intelligence Analyst Job Overview : We are currently seeking a highly motivated and detail-oriented individuals to join our team.
You will play a crucial role in supporting our decision-making proces...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Be among the first 25 applicants.This range is provided by Mercor.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Mercor connects elite creative...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 
At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
Children’s Services of Roxbury (CSR) seeks one to two paid graduate student interns for the.Interns will support one of two focus areas—.
Program Data Integrity & Systems Development...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
At Spindrift, we’re making every beverage a positive force of nature.Founded in 2010, we believe the best flavors come directly from nature.
That’s why every Spindrift beverage is made the hard way—...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
SharkNinja is a global product design and technology company, with a diversified portfolio of 5-star rated lifestyle solutions that positively impact people’s lives in homes around the world.Powere...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Primary Skills : Excel (Expert), Cost Modeling (Experienced), Forecasting (proficient), Project Management (Intermediate), SQL (Knowledge).
Duration : 5 months with possible extension.Location : North ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Intelligence Analyst Job Overview : We are currently seeking a highly motivated and detail-oriented individuals to join our team.
Is this your next job Read the full description below to find out, an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
We are seeking a detail-oriented and analytical Research Analyst to join our team.The ideal candidate will possess strong quantitative and qualitative research skills, exceptional critical thinking...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
Job Title : AI-Enabled Global Study Manager Location : Morristown, NJ Cambridge, MA About the Job Are you ready to shape the future of medicine? The race is on to speed up drug discovery and devel...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
When you join the growing BILH team, you're not just taking a job, you're making a difference in people's lives.Supervisor, Epic / Vended Applications.
The Supervisor, Epic / Vended Applications, is a m...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.For years, Capital One has been leading the industry in using machine learning to create real-time, in...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days 
AI-Enabled Global Study Data Leader.Are you ready to shape the future of medicine? The race is on to speed up drug discovery and development to find answers for patients and their families.Your ski...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Applied Researcher I (AI Foundations).At Capital One, we are creating trustworthy and reliable AI systems, changing banking for good.
For years, Capital One has been leading the industry in using ma...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
- serp_jobs.job_card.promoted
Applied Researcher II
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Remote Senior Financial Analyst - AI Trainer
Data AnnotationNewton, Massachusettsserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
Hireio, Inc. is hiring : AI Communication Quality & Culture Evaluator in Boston
MediabistroBoston, MA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Intelligence Analyst
US ArmyBoston, MA, United Statesserp_jobs.job_card.part_time +1
Mercor is hiring : Audio Evaluation Expert - Fully Remote in Boston
MediabistroBoston, MA, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Financial Analyst - AI Trainer
Data AnnotationFitchburg, Massachusettsserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Applied Researcher 1
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Research and Evaluation Intern
Childrens Services Of RoxburyRoxbury, MA, USserp_jobs.job_card.full_time
Sensory Analyst
SpindriftNewton, MA, USserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
Lead Consumer Insights Analyst
SharkNinjaNeedham, MA, United Statesserp_jobs.job_card.full_time
Data Analyst : 25-05026
Akraya IncNorth Reading, Massachusetts, United Statesserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
Remote Product Research Evaluator (Hiring Immediately)
Earn HausWeare, New Hampshire, USserp_jobs.filters.remote
serp_jobs.job_card.full_time +2
- serp_jobs.job_card.promoted
Intelligence Analyst
United States ArmyBoston, Massachusetts, USserp_jobs.job_card.part_time +1
- serp_jobs.job_card.new
Research Analyst
Stella Contracting, IncBoston, Massachusetts, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
AI-Enabled Global Study Manager
SanofiCambridge, Massachusetts, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Supervisor, Epic Analyst (Research Certified)
Beth Israel Lahey HealthBoston, MA, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied Researcher I
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
AI-Enabled Global Study Data Leader
SanofiCambridge, MA, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Applied Researcher I (AI Foundations)
Capital OneCambridge, MA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Remote Commercial Banking Analyst - AI Trainer
Data AnnotationBeverly, Massachusettsserp_jobs.filters.remote
serp_jobs.job_card.full_time +1