AI Evaluation Analyst
VirtualVocationsArlington, Virginia, United Statesjob_description.job_card.1_day_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_description
A company is looking for an AI Agent Evaluation Analyst.
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Some understanding of scoring or evaluation in agent testing
serp_jobs.job_alerts.create_a_job
Ai Analyst • Arlington, Virginia, United States
Job_description.internal_linking.related_jobs
Distinguished AI Engineer (Agentic AI Platform Infrastructure) At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an indu...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
Seattle, Washington, United States Software and Services.Master’s degree or foreign equivalent in Data Science, Computer Science or related field.
Education and / or Experience must include : .Utilizing...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
The Johns Hopkins Data Science and AI Institute (DSAI) is focused on revolutionizing discovery by advancing artificial intelligence that evolves collaboratively with human intelligence, combining t...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Lead Machine Learning Engineer - ML / AI At Capital One, we are changing banking for good by creating responsible and reliable AI-powered systems.
Our investments in technology infrastructure and worl...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Senior Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery en...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Senior Lead AI Engineer (AI Foundations, LLM Core) Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an indust...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
Agentic AI Machine Learning Engineer.As an experienced machine learning engineer, you understand good.To compete in today's technical landscape, mission-oriented machine learning solutions must be ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Lead Machine Learning Engineer (Intelligent Foundations & Experiences) As a Capital One Machine Learning Engineer (MLE), you'll be part of an Agile team dedicated to productionizing machine learnin...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Senior Lead AI Engineer (Gen AI Platform Services) Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an indust...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Do you enjoy creating innovative solutions to complex challenges while working within a community of dynamic technology professionals? As an experienced data professional, you know that machine lea...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 
We use our platform to connect you with exciting contract opportunities and discuss projects on behalf of the companies we partner with.
As a member of the Technology Team, you’ll bring practi...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Senior Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delive...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Distinguished AI Engineer Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an industry leader in using machin...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Senior Lead AI Engineer (LLM Customization and Finetuning) Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been a...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
LIFELENZ is a rapidly growing team of 80+ mathematicians, engineers, designers, computer scientists, strategists and client success experts based in the U.
Canada, Australia, and UK building a machi...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Senior Machine Learning Engineer As a Capital One Machine Learning Engineer (MLE), you'll be part of an Agile team dedicated to productionizing machine learning applications and systems at scale.Yo...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours 
Data Scientist / Data Engineer (Python).We are seeking a highly skilled.Data Scientist and Data Engineer.This role combines advanced technical capabilities in Python development and data engineerin...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days Senior AI Engineer (LLM Core) Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an industry leader in using ma...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Senior AI Engineer (AI Foundations) Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an industry leader in us...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours Senior Lead AI Engineer (AI Foundations) Overview : At Capital One, we are creating responsible and reliable AI systems, changing banking for good.
For years, Capital One has been an industry leader ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Distinguished AI Engineer (Agentic AI Platform Infrastructure)
Capital OneArnold, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Machine Learning Engineer
Apple Inc.Washington, DC, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Scientific Software Engineer (Data Science & AI Institute)
InsideHigherEdBaltimore, Maryland, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Lead Machine Learning Engineer - ML / AI
Capital OneWashington, D.C., USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Data Engineer
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Lead AI Engineer (AI Foundations, LLM Core)
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Agentic AI Machine Learning Engineer
Phase2 TechnologyWashington, DC, United Statesserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Lead Machine Learning Engineer (Intelligent Foundations & Experiences)
Capital OneBaltimore, MD, United Statesserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Lead AI Engineer (Gen AI Platform Services)
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Machine Learning Engineer
Phase2 TechnologyWashington, DC, United Statesserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Data Modeler
hackajobMcLean, VA, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Lead Data Engineer
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Distinguished AI Engineer
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Lead AI Engineer (LLM Customization and Finetuning)
Capital OneArnold, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Machine Learning Engineer
LIFELENZWashington, DC, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Machine Learning Engineer
Capital OnePimmit, VA, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Data Scientist / Data Engineer
AOC Solutions, Inc.Fort Belvoir, VA, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior AI Engineer (LLM Core)
Capital OneArnold, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior AI Engineer (AI Foundations)
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Senior Lead AI Engineer (AI Foundations)
Capital OneEast Case, MD, USserp_jobs.job_card.full_time +1