AI Evaluation Analyst
VirtualVocationsHayward, California, United Statesjob_description.job_card.variable_hours_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
serp_jobs.job_alerts.create_a_job
Ai Analyst • Hayward, California, United States
Job_description.internal_linking.related_jobs
Press Tab to Move to Skip to Content Link.Select how often (in days) to receive an alert : .We are seeking a talented and detail-oriented Visual Data Evaluator to join our team.The ideal candidate wi...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
USA Remote (Prefer PST Time Zone).The Technical Research Associate is a non-traditional role in which you will work on our AI data collection team.
You have a background in QA or data analysis and a...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days A company is looking for an AI Agent Evaluation Analyst.Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumpti...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location.
The Multimedia Generation team focuses on fine tuning an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
Applied AI company on a mission to automate sales, marketing, and customer success for B2B companies.We build the most complete and accurate picture of the B2B buyer.
We use this data to power appli...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
The application window is expected to close on 10 / 25 / 2025.Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.This is a hybrid role w...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
We believe in using space to help life on Earth.Planet designs, builds, and operates the largest constellation of imaging satellites in history.
This constellation delivers an unprecedented dataset ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Demonstrates good judgment in selecting methods and techniques for obtaining solutions.The Bodansky and DeRisi laboratories at the University of California at San Francisco are seeking a Research D...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 A company is looking for an AI Agent Evaluation Analyst.Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumpti...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location.
The Multimedia Generation team focuses on fine tuning an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
Our client, a world leader in diagnostics and life sciences, is looking for a.Data Analyst - VIsualisation”.Job Duration : Long term Contract (Possibility Of Further Extension).Company Benefit...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Be among the first 25 applicants.This range is provided by Mercor.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Mercor connects elite creative...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days 
Join Tether and Shape the Future of Digital Finance.At Tether, we’re not just building products, we’re pioneering a global financial revolution.
Our cutting-edge solutions empower businesses—from ex...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
At Parafin, we’re on a mission to grow small businesses.Small businesses are the backbone of our economy, but traditional banks often don’t have their backs.
We build tech that makes it simple for s...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 About the team Our TikTok Search Operation team plays a key role in improving TikTok users search experience by investigating user feedback, evaluating search result, analyzing data, etc.In order t...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Visual Data Evaluator Job at Firstsource Solutions Ltd in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
Technical Research Associate - AI
Keywords StudiosSan Jose, CA, USserp_jobs.job_card.full_time
serp_jobs.filters_job_card.quick_apply
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
AI Evaluation Analyst
VirtualVocationsSanta Clara, California, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.new
ICONMA is hiring : AI Multimedia Generation Product Evaluator in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Data Analyst
HockeyStack, Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Commercial Banking Analyst - AI Trainer
Data AnnotationBerkeley, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
AI Researcher
Cisco Systems, Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI Engineer, Digital Employee Experience
Planet Labs PBCSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Research Data Analyst
University of California - San Francisco Campus and HealthSan Francisco, CA, United Statesserp_jobs.job_card.full_time
AI Communication Quality & Culture Evaluator Job at Hireio, Inc. in San Francisc
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Financial Analyst - AI Trainer
Data AnnotationOakley, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
AI Agent Evaluation Analyst
VirtualVocationsSanta Clara, California, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.new
ICONMA is hiring : AI Multimedia Generation Product Evaluator in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Data Analyst - Visualisation
Dawar Consulting, Inc.South San Francisco, CA, USserp_jobs.job_card.temporary
Mercor is hiring : Audio Evaluation Expert - Fully Remote in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
Senior AI Research Engineer, Model Inference (100% Remote)
Tether Operations LimitedSan Francisco, CA, USserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI Engineer
Parafin Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Senior Financial Analyst - AI Trainer
Data AnnotationLivermore, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
Search Content Evaluation POC - USDS
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time
Hireio, Inc. is hiring : AI Communication Quality & Culture Evaluator in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time