AI Evaluation Analyst
VirtualVocationsHayward, California, United Statesjob_description.job_card.1_day_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
serp_jobs.job_alerts.create_a_job
Ai Analyst • Hayward, California, United States
Job_description.internal_linking.related_jobs
Press Tab to Move to Skip to Content Link.Select how often (in days) to receive an alert : .We are seeking a talented and detail-oriented Visual Data Evaluator to join our team.The ideal candidate wi...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Be among the first 25 applicants.This range is provided by Mercor.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Mercor connects elite creative...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location.
The Multimedia Generation team focuses on fine tuning an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
The Technical Lead will drive AI research in one or more of the following areas : structure prediction, protein design, and lead optimization.
PhD in AI, Machine Learning, Bioinformatics, or related ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
The application window is expected to close on 10 / 25 / 2025.Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.This is a hybrid role w...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
We believe in using space to help life on Earth.Planet designs, builds, and operates the largest constellation of imaging satellites in history.
This constellation delivers an unprecedented dataset ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Demonstrates good judgment in selecting methods and techniques for obtaining solutions.The Bodansky and DeRisi laboratories at the University of California at San Francisco are seeking a Research D...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days 
At Schwab, you will build a rewarding career while making a difference in the lives of our millions of clients.Here, innovative thinking meets creative problem solving as we work together to challe...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location.
The Multimedia Generation team focuses on fine tuning an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
Polychain is looking for a macro-thinking Cryptocurrency Research Analyst who has the ability to proactively identify strategic, early stage investment opportunities in companies and protocols with...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Competitive salary + meaningful equity (founding tier).Backed by 8VC, we\'re building a world-class team to tackle one of the industry’s most critical infrastructure problems.We’re designing the fu...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Analyst, Research — San Francisco, CA.The Research team at the Center for Effective Philanthropy (CEP) seeks a highly motivated Analyst with a desire to create positive social impact by improving t...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
At the University of California, Berkeley, we are dedicated to fostering a community where everyone feels welcome and can thrive.
Our culture of openness, freedom and belonging make it a special pla...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
At Parafin, we’re on a mission to grow small businesses.Small businesses are the backbone of our economy, but traditional banks often don’t have their backs.
We build tech that makes it simple for s...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
As a member of our Frontier Tech Consultant team, you will play a critical role in advancing cutting-edge AI innovations by conducting high-impact experiments and ensuring seamless execution at the...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 At Schwab, you will build a rewarding career while making a difference in the lives of our millions of clients.Here, innovative thinking meets creative problem solving as we work together to challe...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Visual Data Evaluator Job at Firstsource Solutions Ltd in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
Audio Evaluation Expert - Fully Remote Job at Mercor in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.new
ICONMA is hiring : AI Multimedia Generation Product Evaluator in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Research Engineer, Lead
Menlo VenturesSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Financial Analyst - AI Trainer
Data AnnotationMartinez, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
AI Researcher
Cisco Systems, Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI Engineer, Digital Employee Experience
Planet Labs PBCSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Research Data Analyst
University of California - San Francisco Campus and HealthSan Francisco, CA, United Statesserp_jobs.job_card.full_time
AI Communication Quality & Culture Evaluator Job at Hireio, Inc. in San Francisc
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Researcher
Charles SchwabSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.new
ICONMA is hiring : AI Multimedia Generation Product Evaluator in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Venture Research Analyst
P2PSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
ML / AI Research Engineer — Agentic AI Lab (Founding Team)
FabrionSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Analyst - Research
Center for Effective PhilanthropySan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Research Data Analyst (6256U), Haas School of Business - #81170
University of California-BerkeleyBerkeley, CA, United Statesserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Applied AI Engineer
Parafin Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Strategy Consultant, Frontier Tech
Scale AI, Inc.San Francisco, CA, United Statesserp_jobs.job_card.part_time
- serp_jobs.job_card.promoted
Applied AI Engineer - AI Transformation
Charles SchwabSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Senior Financial Analyst - AI Trainer
Data AnnotationConcord, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
Hireio, Inc. is hiring : AI Communication Quality & Culture Evaluator in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time