AI Agent Evaluation Analyst
VirtualVocationsSan Francisco, California, United Statesjob_description.job_card.1_day_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats like JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
serp_jobs.job_alerts.create_a_job
Ai Analyst • San Francisco, California, United States
Job_description.internal_linking.related_jobs
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location.
The Multimedia Generation team focuses on fine tuning an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
The company has developed an artificial intelligence (AI) powered technology stack purpose-built for commercial trucking and the public sector.
The company delivers freight daily for its customers a...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Applied AI / ML Engineer to help design, implement, and scale advanced models that power Replica’s urban simulation and analytics products.
This role is scoped at the Senior level or above : we expect ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
The Technical Lead will drive AI research in one or more of the following areas : structure prediction, protein design, and lead optimization.
PhD in AI, Machine Learning, Bioinformatics, or related ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
The application window is expected to close on 10 / 15 / 2025.Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.This is a hybrid role w...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
We believe in using space to help life on Earth.Planet designs, builds, and operates the largest constellation of imaging satellites in history.
This constellation delivers an unprecedented dataset ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Google — Mountain View, CA, USA.Fast-paced, dynamic, and proactive, YouTube’s Trust & Safety team is dedicated to making YouTube a safe place for users, viewers, and content creators around the wor...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Docusign brings agreements to life.Docusign solutions to accelerate the process of doing business and simplify people’s lives.
With intelligent agreement management, Docusign unleashes business-crit...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Our Client, a Internet Content & Information company, is looking for an AI Multimedia Generation Product Evaluator for their Remote location.
The Multimedia Generation team focuses on fine tuning an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Be among the first 25 applicants.This range is provided by Mercor.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Mercor connects elite creative...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the q...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 
Competitive salary + meaningful equity (founding tier).Backed by 8VC, we\'re building a world-class team to tackle one of the industry’s most critical infrastructure problems.We’re designing the fu...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
If you are a current and active PHI employee, do not use this site to apply for positions.The Public Health Institute (PHI) is an independent, nonprofit organization dedicated to promoting health, ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
At Parafin, we’re on a mission to grow small businesses.Small businesses are the backbone of our economy, but traditional banks often don’t have their backs.
We build tech that makes it simple for s...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Baseten provides the infrastructure, tooling, and expertise needed to bring great AI products to market - fast.Backed by top investors including IVP, Spark Capital, Greylock, and Conviction, we’re ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Atlas is the concierge and credit card built for those who expect more — unlocking coveted access across dining, travel, and lifestyle while making spending seamless and effortless.Our members are ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Join a collaborative remote team working to make AI communication emotionally intelligent, culturally aware, and engaging.
Score AI messages for tone, empathy, and contextual awareness.Identify subt...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
- serp_jobs.job_card.promoted
Remote Financial Analyst - AI Trainer
Data AnnotationRichmond, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.new
ICONMA is hiring : AI Multimedia Generation Product Evaluator in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI Engineer – Generative AI
KodiakSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI / ML Engineer
Replica Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Research Engineer, Lead
Menlo VenturesSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Researcher
Cisco Systems, Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI Engineer, Digital Employee Experience
Planet Labs PBCSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Staff Engineering Analyst, Generative AI
Google Inc.Mountain View, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Agentic Engineer
DocuSign, Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
AI Communication Quality & Culture Evaluator Job at Hireio, Inc. in San Francisc
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.new
ICONMA is hiring : AI Multimedia Generation Product Evaluator in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.job_card.full_time
Mercor is hiring : Audio Evaluation Expert - Fully Remote in San Francisco
MediabistroSan Francisco, CA, United Statesserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Remote Commercial Banking Analyst - AI Trainer
Data AnnotationNovato, Californiaserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
ML / AI Research Engineer — Agentic AI Lab (Founding Team)
FabrionSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Research Associate IV
Public Health InstituteRichmond, CA, United Statesserp_jobs.job_card.full_time +2
- serp_jobs.job_card.promoted
Applied AI Engineer
Parafin Inc.San Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Applied AI Inference Engineer
BasetenSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Founding Applied AI Engineer
AtlasSan Francisco, CA, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Software Engineer, Perception Evaluation
WaymoMountain View, CA, United Statesserp_jobs.job_card.full_time
Hireio, Inc. is hiring : AI Communication Quality & Culture Evaluator in San Jose
MediabistroSan Jose, CA, United Statesserp_jobs.job_card.full_time