Talent.com
Evaluation Scenario Writer - QA

Evaluation Scenario Writer - QA

MindriftSC, US
job_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.part_time
  • serp_jobs.job_card.permanent
  • serp_jobs.filters.remote
  • serp_jobs.filters_job_card.quick_apply
job_description.job_card.job_description

At Mindrift , innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.

The Mindrift platform, launched and powered by Toloka , connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

Who we're looking for :

We’re looking for curious and intellectually proactive contributors who never miss an error and can think outside of the box when brainstorming solutions.

Are you comfortable with ambiguity and complexity? Does an async, remote, flexible opportunity sound exciting? Would you like to learn how modern AI systems are tested and evaluated?

This is a flexible, project-based opportunity well-suited for :

  • Analysts, researchers, or consultants with strong critical thinking skills.
  • Students (senior undergrads / grad students) looking for an intellectually interesting gig.
  • People open to a part-time and non-permanent opportunity.

About the project :

We’re on the hunt for an Evaluation Scenario Writer - QA for a new project focused on ensuring the quality and correctness of evaluation scenarios created for LLM agents. This project opportunity blends manual scenario validation, automated test thinking, and collaboration with writers and engineers. You will verify test logic, flag inconsistencies, and help maintain a high bar for evaluation coverage and clarity.

What you’ll be doing :

  • Reviewing and validating test scenarios from Evaluation Writers.
  • Spotting logical inconsistencies, ambiguities, or missing checks.
  • Suggesting improvements to structure, edge cases, or scoring logic.
  • Collaborating with infrastructure and tool developers to automate parts of the review.
  • Creating clean and testable examples for others to follow.

    • Although we’re only looking for experts for this current project, contributors with consistent high-quality submissions may receive an invitation for ongoing collaboration across future projects.

    How to get started

    Apply to this post, qualify, and get the chance to contribute to a project aligned with your skills, on your own schedule. Shape the future of AI while building tools that benefit everyone.

    Requirements

    The ideal contributor will have :

  • Strong QA background (manual or automation), preferably in complex testing environments.
  • Understanding of test design, regression testing, and edge case detection.
  • Ability to evaluate logic and structure of test scenarios (even if written by others).
  • Experience reviewing and debugging structured test case formats (JSON, YAML).
  • Familiarity with Python and JS scripting for test automation or validation.
  • Clear communication and documentation skills.
  • Willingness to occasionally write or refactor test scenarios.

    • We also value applicants who have :

  • Experience testing AI-based systems or NLP applications.
  • Familiarity with scoring systems and behavioral evaluation.
  • Git / GitHub workflow familiarity (PR review, versioning of test cases).
  • Experience using test management systems or tracking tools.

    • Benefits

    Contribute on your own schedule, from anywhere in the world. This opportunity allows you to :

  • Get paid for your expertise, with  rates that can go up to $55 / hour  depending on your skills, experience, and project needs.
  • Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments.
  • Participate in an advanced AI project and gain valuable experience to enhance your portfolio.
  • Influence how future AI models understand and communicate in your field of expertise.
    • serp_jobs.job_alerts.create_a_job

    Qa Evaluation Scenario Writer • SC, US

    Job_description.internal_linking.related_jobs
    • serp_jobs.job_card.promoted
    Non-Destructive Evaluation Technician

    Non-Destructive Evaluation Technician

    Alpek Polyester USAGaston, SC, US
    serp_jobs.job_card.full_time
    We are seeking a motivated individual to join and support our.CA - Columbia Quality Control team.Non-Destructive Evaluation (NDE) Technician. F;motivated by technical challenges and able to u...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Entry Level Pest Control Technician - Columbia, SC

    Entry Level Pest Control Technician - Columbia, SC

    Anticimex CarolinasEastover, SC, US
    serp_jobs.job_card.full_time
    Competitive Compensation – Impressive Benefits – Potential for Growth – Work / Life Balance.Now hiring in Columbia, Lexington, Blythewood, Irmo, Batesburg-Leesville, and Eastover ar...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Travel Speech Language Pathologist (SLP) - $789 per week in Camden, SC

    Travel Speech Language Pathologist (SLP) - $789 per week in Camden, SC

    AlliedTravelCareersCamden, SC, US
    serp_jobs.job_card.full_time
    AlliedTravelCareers is working with Skyline Med Staff to find a qualified Speech Language Pathologist (SLP) in Camden, South Carolina, 29020!. Join the Top- Rated Travel Healthcare Team!\n\nSkyline ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Tax Expert - Onsite - 2+ Yrs Paid Tax Experience Required

    Tax Expert - Onsite - 2+ Yrs Paid Tax Experience Required

    IntuitSwansea, SC, United States
    serp_jobs.job_card.full_time
    Intuit is seeking highly motivated individuals to join our dynamic team as dedicated TurboTax Live Seasonal Local Service Experts in one of our new TurboTax locations across the United States on a ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Travel Speech Language Pathologist (SLP) - $1,816 per week in Camden, SC

    Travel Speech Language Pathologist (SLP) - $1,816 per week in Camden, SC

    AlliedTravelCareersCamden, SC, US
    serp_jobs.job_card.full_time +1
    AlliedTravelCareers is working with Infojini Healthcare to find a qualified Speech Language Pathologist (SLP) in Camden, South Carolina, 29020!. Speech Language Pathologist (SLP),06 : 00 : 00-14 : 00 : 00....serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Travel Speech Language Pathologist (SLP) - $1,684 to $1,834 per week in Camden, SC

    Travel Speech Language Pathologist (SLP) - $1,684 to $1,834 per week in Camden, SC

    AlliedTravelCareersCamden, SC, US
    serp_jobs.job_card.full_time
    AlliedTravelCareers is working with Host Healthcare to find a qualified Speech Language Pathologist (SLP) in Camden, South Carolina, 29020!. Host Healthcare is an award-winning travel healthcare com...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Travel Speech Language Pathologist (SLP) - $1,994 per week in Camden, SC

    Travel Speech Language Pathologist (SLP) - $1,994 per week in Camden, SC

    AlliedTravelCareersCamden, SC, US
    serp_jobs.job_card.full_time
    AlliedTravelCareers is working with Medical Solutions to find a qualified Speech Language Pathologist (SLP) in Camden, South Carolina, 29020!. A facility in Camden, SC is seeking its next amazing SL...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Freelance Writer

    Freelance Writer

    OutlierRemote, SC, United States
    serp_jobs.filters.remote
    serp_jobs.job_card.full_time
    Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    Travel Speech Language Pathologist (SLP) - $1,806 to $2,006 per week in Camden, SC

    Travel Speech Language Pathologist (SLP) - $1,806 to $2,006 per week in Camden, SC

    AlliedTravelCareersCamden, SC, US
    serp_jobs.job_card.full_time +1
    AlliedTravelCareers is working with Jackson Therapy Partners to find a qualified Speech Language Pathologist (SLP) in Camden, South Carolina, 29020!. Join Jackson Therapy Partners as a travel SLP an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Travel Speech Language Pathologist

    Travel Speech Language Pathologist

    Medical Solutions AlliedCamden, SC, US
    serp_jobs.job_card.full_time
    Medical Solutions Allied is seeking a travel Speech Language Pathologist for a travel job in Camden, South Carolina.Job Description & Requirements. We’re seeking talented healthcare profes...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Licensed Clinical Therapist (LISW-CP / LPC / LMFT) •New office opens in December! •

    Licensed Clinical Therapist (LISW-CP / LPC / LMFT) •New office opens in December! •

    LifeStance HealthGaston, SC, US
    serp_jobs.job_card.full_time
    At LifeStance Health, we believe in a truly healthy society where mental and physical healthcare are unified to make lives better. Our mission is to help people lead healthier, more fulfilling lives...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Clinical Specialist

    Clinical Specialist

    SouthernMEDLexington, SC, US
    serp_jobs.job_card.full_time
    What it means to be a Clinical Specialist with Us.We value our families we serve, our communities and our team members.As such we need people like yourself who appreciate time spent with our famili...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    Entry Level Tax Preparer

    Entry Level Tax Preparer

    Jackson Hewitt - 1906Lexington, SC, US
    serp_jobs.job_card.full_time
    Seeking a job that fits around your life? Look no further! We provide.Embrace the possibilities by adding this side gig or growing a new career at Jackson Hewitt and apply today!.You will also have...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Remote AI Writing Evaluator

    Remote AI Writing Evaluator

    OutlierColumbia, SC, United States
    serp_jobs.filters.remote
    serp_jobs.job_card.full_time
    Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    Travel Speech Language Pathologist (SLP) - $1,420 per week in Camden, SC

    Travel Speech Language Pathologist (SLP) - $1,420 per week in Camden, SC

    AlliedTravelCareersCamden, SC, US
    serp_jobs.job_card.full_time
    AlliedTravelCareers is working with FlexCare to find a qualified Speech Language Pathologist (SLP) in Camden, South Carolina, 29020!. FlexCare is a nationwide leader in the staffing of travel nurses...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    English Writing and Content Reviewing Expertise Sought for AI Training

    English Writing and Content Reviewing Expertise Sought for AI Training

    OutlierColumbia, SC, United States
    serp_jobs.job_card.full_time
    Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Travel Speech Language Pathologist

    Travel Speech Language Pathologist

    Skyline Med Staff AlliedCamden, SC, US
    serp_jobs.job_card.full_time
    Skyline Med Staff Allied is seeking a travel Speech Language Pathologist for a travel job in Camden, South Carolina.Job Description & Requirements. Join the Top- Rated Travel Healthcare Team!\n\...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    MO-7 / 17 - 7513 - Technical Writer w / Medicaid Exp - Columbia, SC (Local to SC Candidates Only)

    MO-7 / 17 - 7513 - Technical Writer w / Medicaid Exp - Columbia, SC (Local to SC Candidates Only)

    FHRColumbia, SC, US
    serp_jobs.job_card.full_time
    Hybrid (4 days in office in Columbia, SC, 1 day remote).Candidate must be a CURRENT South Carolina resident.Our direct client has an opening for a Technical Writer w / Medicaid Exp # 7513.This posit...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30