AI Agent Evaluation Analyst - AI Trainer
MindriftSan Antonio, TX, USjob_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.part_time
- serp_jobs.job_card.permanent
- serp_jobs.filters.remote
- serp_jobs.filters_job_card.quick_apply
job_description.job_card.job_descriptionReviewing evaluation tasks and scenarios for logic, completeness, and realism. Identifying inconsistencies, missing assumptions, or unclear decision points. Helping define clear expected behaviors (gold standards) for AI agents. Annotating cause-effect relationships, reasoning paths, and plausible alternatives. Thinking through complex systems and policies as a human would to ensure agents are tested properly. Working closely with QA, writers, or developers to suggest refinements or edge case coverage. Excellent analytical thinking : Can reason about complex systems, scenarios, and logical implications. Strong attention to detail : Can spot contradictions, ambiguities, and vague requirements. Familiarity with structured data formats : Can read, not necessarily write JSON / YAML. Can assess scenarios holistically : What's missing, what’s unrealistic, what might break? Good communication and clear writing (in English) to document your findings. Experience with policy evaluation, logic puzzles, case studies, or structured scenario design. Background in consulting, academia, olympiads (e.g. logic / math / informatics), or research. Exposure to LLMs, prompt engineering, or AI-generated content. Familiarity with QA or test-case thinking (edge cases, failure modes, “what could go wrong”). Some understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.). Get paid for your expertise, with rates that can go up to $55 / hour depending on your skills, experience, and project needs. Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments. Participate in an advanced AI project and gain valuable experience to enhance your portfolio. Influence how future AI models understand and communicate in your field of expertise.
At Mindrift , innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.
The Mindrift platform, launched and powered by Toloka , connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.
Who we're looking for :
We’re looking for curious and intellectually proactive contributors, the kind of person who double-checks assumptions and plays devil’s advocate.
Are you comfortable with ambiguity and complexity? Does an async, remote, flexible opportunity sound exciting? Would you like to learn how modern AI systems are tested and evaluated?
This is a flexible, project-based opportunity well-suited for :
- Analysts, researchers, or consultants with strong critical thinking skills.
- Students (senior undergrads / grad students) looking for an intellectually interesting gig.
- People open to a part-time and non-permanent opportunity.
About the project :
We’re on the hunt for QAs for autonomous AI agents for a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks. Throughout the project, you’ll have to balance quality assurance, research, and logical problem-solving. This project opportunity is ideal for people who enjoy looking at systems holistically and thinking through scenarios, implications, and edge cases.
You do not need a coding background, but you must be curious, intellectually rigorous, and capable of evaluating the soundness and consistency of complex setups. If you’ve ever excelled in things like consulting, CHGK, Olympiads, case solving, or systems thinking — you might be a great fit.
What you’ll be doing :
How to get started :
Apply to this post, qualify, and get the chance to contribute to a project aligned with your skills, on your own schedule. Shape the future of AI while building tools that benefit everyone.
Requirements
We also value applicants who have :
Benefits
serp_jobs.job_alerts.create_a_job
Ai Analyst • San Antonio, TX, US
Job_description.internal_linking.related_jobs
APPLICATIONS ARE REVIEWED DAILY.YOU CAN EXPECT A RESPONSE WITHIN 24 HOURS.Our Patients : An intentional focus on small caseloads with a compassionate care approach.
Our Culture : ABC Core Values are m...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Become a part of our caring community and help us put health first.As a Medical Assistant, you will be a part of the Clinical Care Team engaging with our patients to develop lifelong well-being and...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
ABOUT Effective, secure communication in the cyber domain is essential to the everyday operations of military intelligence in America’s Navy.
Information Professionals who oversee the seamless opera...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
CS / Sales Agent - Entry Level & Remote, Work by Appointments.With consistent growth year over year, we're looking to add more talented individuals to our rapidly growing company.This career allows y...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
We are urgently looking for people interested in taking online surveys for Fortune 500 brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Earn up...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
Financial Representative Trainee (Sales) - Austin, TX.Sales, Early Career Programs, Financial Representative Trainee.Are you driven, self-motivated, and eager to jumpstart your career in the financ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
At PwC, our people in business application consulting specialise in consulting services for a variety of business applications, helping clients optimise operational efficiency.These individuals ana...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days ABOUT Effective, secure communication in the cyber domain is essential to the everyday operations of military intelligence in America’s Navy.
Information Professionals who oversee the seamless opera...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days 
Augusta, GA, San Antonio, TX, Oahu, HI, Various Locations.Skills and Experience You'll Bring : .Provide information and documentation that enables malware detection, signature creation, and mitigatio...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
Student success coach and mentor (entry level,.City Year, an AmeriCorps program, helps students across schools succeed.City Year AmeriCorps members provide support to students, classrooms and the.A...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_hour At PwC, our people in business application consulting specialise in consulting services for a variety of business applications, helping clients optimise operational efficiency.These individuals ana...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days ABOUT Effective, secure communication in the cyber domain is essential to the everyday operations of military intelligence in America’s Navy.
Information Professionals who oversee the seamless opera...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days 
The Credentialed Trainer I is responsible for training delivery, support, and all related tasks associated with their assigned application(s).
The Credentialed Trainer works with the Principal Train...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
We want you to join our team as an Assistant Manager.If you have the desire to be challenged, work in a fast-paced, fun environment and create a career in management look no further.As an assistan...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30 Student success coach and mentor (entry level,.City Year, an AmeriCorps program, helps students across schools succeed.City Year AmeriCorps members provide support to students, classrooms and the.A...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_hour 
Through our dedicated associates, Conduent delivers mission-critical services and solutions on behalf of Fortune 100 companies and over 500 governments - creating exceptional outcomes for our clien...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
Responsibilities The primary responsibility of this position is to read meters and record consumption of the water used, cleaning of meter boxes, and removal of vegetation impeding access to meters...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days At PwC, our people in business application consulting specialise in consulting services for a variety of business applications, helping clients optimise operational efficiency.These individuals ana...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
- serp_jobs.job_card.promoted
Board Certified Behavioral Analyst
Action Behavior CentersVilla Nueva, TX, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Medical Assistant
CenterWell Primary CareJourdanton, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Intelligence Officer
NavyCharlotte, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
CS / Sales Agent - Entry Level & REMOTE, work by Appointments
Global EliteNew Braunfels, TX, USserp_jobs.filters.remote
serp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Online Survey Taker. Earn up to $25 per survey. - Remote
Earn HausPoteet, Texas, USserp_jobs.filters.remote
serp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Financial Representative Trainee (Sales) - Austin, TX
Mutual of OmahaStaples, TX, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Data Engineer - Senior Associate
PwCSan Antonio, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Information Systems Technician
NavyCharlotte, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Intrusion Analyst (Levels I-III)
BTS Software SolutionsSan Antonio, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Commit to Serve Your City Year! (Full Pay / Benefits - Academic Mentor)
City YearLa Vernia, TX, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
AI Data Engineer - Senior Manager
PwCSan Antonio, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Military Intelligence
NavyFloresville, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Credentialed Trainer - Cupid / Radiant, Optime / Anesthesia, Beaker
Christus HealthSan Antonio, TX, United Statesserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
Store Assistant, Full Time
Circle KPleasanton, TX, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
- serp_jobs.job_card.new
Academic Tutor & Mentor (Entry Level, Paid, Full-time) 
City YearLa Vernia, TX, USserp_jobs.job_card.full_time +1
- serp_jobs.job_card.promoted
Customer Experience Associate II
Conduent State & Local Solutions, IncLa Vernia, TX, USserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
County Meter Reader
Meter ReaderFloresville, TXserp_jobs.job_card.full_time
- serp_jobs.job_card.promoted
AI Data Engineer - Manager
PwCSan Antonio, TX, United Statesserp_jobs.job_card.full_time