Talent.com
Test Engineer-AI/LLM
Test Engineer-AI/LLMOPPO US Research Center • Palo Alto, CA, US
Test Engineer-AI / LLM

Test Engineer-AI / LLM

OPPO US Research Center • Palo Alto, CA, US
job_description.job_card.30_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.full_time
  • serp_jobs.filters_job_card.quick_apply
job_description.job_card.job_description

OPPO US Research Center is seeking a full-time meticulous and innovative AI / LLM Test Engineer to join our cutting-edge AI team. In this critical role, you will evaluate the performance, reliability, and safety of Large Language Models (LLMs) in real-world product scenarios and test end-to-end generative AI solutions. Your work will directly shape how users experience AI-powered features by ensuring robustness, accuracy, and alignment with product goals. This is a unique opportunity to pioneer testing methodologies for next-generation AI systems at the forefront of technology.

We are also seeking a Contractor based LLM Evaluation & QA Engineer to support the testing and validation of large language model (LLM)-powered applications. You will help implement test strategies, execute evaluation workflows, and assist in model performance validation across diverse generative AI use cases.

This contract role is ideal for someone with hands-on experience in AI / ML evaluation, QA engineering, or data analysis who wants to deepen their exposure to generative AI systems.

Requirements

Full-time position requirement :

Core Testing & Evaluation

  • Design and execute performance tests for LLMs across diverse product use cases (e.g., chatbots, content generation etc.).
  • Develop automated test frameworks to evaluate LLM outputs for accuracy, bias, safety, and coherence.
  • Conduct end-to-end testing of integrated generative AI solutions, including APIs, data pipelines, and user interfaces.

Optimization & Validation

  • Collaborate with ML engineers to validate fine-tuned models and optimize prompts for target scenarios.
  • Analyze model failures, edge cases, and adversarial inputs to identify risks and improvement areas.
  • Benchmark LLM performance against industry standards and product-specific KPIs.
  • Collaboration & Quality Assurance

  • Partner with product, engineering, and research teams to define test requirements and acceptance criteria.
  • Document defects, performance metrics, and test results to drive data-driven improvements.
  • Advocate for AI ethics and safety through rigorous testing of fairness, bias mitigation, and content moderation.
  • Innovation & Tooling

  • Build scalable tools for synthetic test data generation, prompt variation testing, and automated evaluation workflows.
  • Stay current with advancements in generative AI testing, including red-teaming techniques and evaluation frameworks (e.g., HELM, Dynabench).
  • Propose novel testing strategies for emerging challenges (e.g., hallucinations, context drift).
  • Basic Qualifications :

  • Bachelor’s degree in Computer Science, Data Science, Engineering, or a related technical field, or equivalent practical experience.
  • 1+ years of experience in software testing, data science, or ML validation, with exposure to AI / ML systems.
  • Proficiency in Python and testing frameworks (e.g., PyTest, Selenium).
  • Hands-on experience evaluating LLMs in production environments (e.g., GPT, Claude, Llama, Gemini).
  • Strong analytical skills for dissecting model behavior, statistical performance, and failure modes.
  • Familiarity with cloud platforms (GCP, Azure, or AWS) and MLOps tooling (e.g., MLflow, Weights & Biases).
  • Experience with version control (Git) and agile development methodologies.
  • Preferred Qualifications :

  • Master’s degree in AI, Machine Learning, or a related field.
  • Expertise in prompt engineering, LLM fine-tuning (e.g., LoRA, RLHF), or optimization techniques.
  • Experience with automated evaluation tools (e.g., LangChain, TruLens) or LLM-specific test suites.
  • Knowledge of data pipelines, SQL / NoSQL databases, and API testing (e.g., Postman).
  • Background in statistics, quantitative analysis, or data visualization for test insights.
  • Contributions to AI safety / ethics initiatives or open-source LLM evaluation projects.
  • Experience testing mobile-integrated AI solutions (Android / iOS).
  • Contractor position requirements :

    Testing & Evaluation Support :

  • Execute pre-defined performance tests for LLMs across various tasks (e.g., summarization, Q&A, chatbot flows).
  • Run scripted evaluations to assess outputs for factuality, coherence, and safety.
  • Perform manual and automated test execution on APIs and LLM-integrated user interfaces.
  • Prompt & model validation :

  • Assist ML engineers in evaluating prompt variations and prompt-tuning outcomes.
  • Log and analyze failure cases, anomalies, and edge cases based on provided guidelines.
  • Collabration & Documentation

  • Work with QA leads, product managers, and ML engineers to understand test goals and criteria.
  • Report defects, compile evaluation summaries, and maintain testing logs.
  • Tooling & Antomation :

  • Use existing internal tools or frameworks to automate test runs and result collection.
  • Contribute to prompt generation, input templating, or result tagging processes.
  • Basic Qualifications :

  • Bachelor's degree or equivalent work experience in a technical field (e.g., Computer Science, Engineering, Data Science).
  • 6+ months experience in software QA, data labeling, LLM evaluation, or ML testing projects.
  • Basic Python proficiency, especially for data processing and automation tasks.
  • Familiarity with LLMs (e.g., GPT, Claude, Gemini) and prompt-based outputs.
  • Comfortable working with tools like Jupyter, Postman, or testing dashboards.
  • Detail-oriented with good documentation habits.
  • Contractor Details :

  • Duration : Long term
  • Rate : Commensurate with experience
  • Conversion Opportunity : High-performing contractors may be considered for full-time roles
  • Benefits

    OPPO is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.

    The US base salary range for this full-time position is $100,000-$200,000 + bonus + long term incentives benefits. Our salary ranges are determined by role, level, and location.

    serp_jobs.job_alerts.create_a_job

    Test • Palo Alto, CA, US

    Job_description.internal_linking.related_jobs
    Test Engineer, Hardware-in-the-Loop (HIL)

    Test Engineer, Hardware-in-the-Loop (HIL)

    Nimble • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    You will be taking on a critical responsibility for ensuring the reliability and functional safety of our core robotics control software and firmware. This role sits at the intersection of developme...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Hardware Test Automation and Reliability Engineer

    Hardware Test Automation and Reliability Engineer

    Science • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Hardware Test Automation and Reliability Engineer.Hardware Test Automation and Reliability Engineer.Science is a clinical‑stage, vertically integrated technology company focused on solving neurosci...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.promoted • serp_jobs.job_card.new
    Senior Test Engineer, End-of-Line (EOL)

    Senior Test Engineer, End-of-Line (EOL)

    Nimble • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Senior Test Engineer, End-of-Line (EOL).Nimble is a robotics and AI company inventing and scaling autonomous logistics with intelligent robots to enable fast, efficient, and sustainable commerce.We...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Software Engineer, Tools & Test

    Software Engineer, Tools & Test

    JUUL Labs, Inc. • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Juul Labs' mission is to transition the world’s billion adult smokers away from combustible cigarettes, eliminate their use, and combat underage usage of our products. We have the opportunity to add...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Software Test Manager, AI Systems

    Software Test Manager, AI Systems

    quadric, Inc • Burlingame, CA, US
    serp_jobs.job_card.full_time
    serp_jobs.filters_job_card.quick_apply
    A Great Opportunity to Join a Rapidly Growing Company.Quadric is a young, rapidly growing company that is creating truly innovative solutions for the next generation of artificial intelligence prod...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days
    Product Test Engineer

    Product Test Engineer

    Cisco Systems, Inc. • San Jose, CA, United States
    serp_jobs.job_card.full_time
    The application window is expected to close on : 10 / 25 / 2025.Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.As part of the team yo...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted
    Senior Hardware Test Automation Engineer

    Senior Hardware Test Automation Engineer

    NightDragon Acquisition Corp. • San Francisco, CA, United States
    serp_jobs.job_card.permanent
    Capella Space is a pioneer in Synthetic Aperture Radar (SAR) satellite technology and space-based signal intelligence.We empower government, commercial, and research organizations around the world ...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    ATE Test Engineer

    ATE Test Engineer

    Jobot • Mountain View, CA, United States
    serp_jobs.job_card.full_time
    Sr Mechanical Energy Engineer - (Engineering Consulting / Design-Build) Hybrid.This Jobot Job is hosted by : Tony Barhoum. Are you a fit? Easy Apply now by clicking the "Apply" button and sending u...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Hardware Test Engineer

    Hardware Test Engineer

    Base Power Company • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    About Base : Base is building the foundation of American power.The grid is the largest, most complex machine in the world. Yet it is aging, struggling to keep up with today’s demand, and is unprepare...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Senior Software Engineer – AI / ML

    Senior Software Engineer – AI / ML

    Glue • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Glue is a well-funded startup working on the next generation of work communication tools.We believe that today’s work chat is noisy, unstructured, and not designed for productivity.We’re drawing fr...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Simulation Test Engineer

    Simulation Test Engineer

    Wealth Recruitment, LLC • Palo Alto, CA, US
    serp_jobs.job_card.full_time
    serp_jobs.filters_job_card.quick_apply
    In this role, you’ll play a key part in supporting simulation workflows by curating data, troubleshooting pipeline issues, and driving process improvements that enhance both quality and efficiency....serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days
    GNC Test Engineer - Model Based Design

    GNC Test Engineer - Model Based Design

    Pivotal • Palo Alto, California, United States, 94301
    serp_jobs.job_card.full_time
    GNC Test Engineer - Model Based Design.Pivotal is the leader in the emerging market of electric Vertical Takeoff and Landing (eVTOL) aircraft. We design, develop, and manufacture light eVTOL aircraf...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days
    Senior Software Engineer in Test

    Senior Software Engineer in Test

    Clockwork Systems, Inc. • Palo Alto, CA, United States
    serp_jobs.job_card.full_time
    Senior Software Engineer in Test.A Software-Driven Revolution in AI Networking.Clockwork Systems was founded by Stanford researchers and veteran systems engineers who share a vision for redefining ...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Senior Automation Test Engineer

    Senior Automation Test Engineer

    Compunnel, Inc. • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    We are seeking a highly skilled Senior Test Automation & Data Engineer to join a high-impact Agile delivery squad working on complex data engineering and quality assurance projects within a large-s...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Hardware Test Engineer

    Hardware Test Engineer

    Echo • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Echo Neurotechnologies is an exciting new startup in the Brain-Computer Interface (BCI) space, driving innovation through advanced hardware engineering and AI solutions. Our mission is to deliver cu...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Test Engineer, Hardware-in-the-Loop (HIL)

    Test Engineer, Hardware-in-the-Loop (HIL)

    Futureshaper.com • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Role SummaryYou will be taking on a critical responsibility for ensuring the reliability and functional safety of our core robotics control software and firmware. This role sits at the intersection ...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Hardware Test Automation and Reliability Engineer

    Hardware Test Automation and Reliability Engineer

    Science, Inc. • San Francisco, CA, United States
    serp_jobs.job_card.full_time
    Hardware Test Automation and Reliability Engineer Join to apply for the Hardware Test Automation and Reliability Engineer role at Science Overview Science is a clinical‑stage, vertically integrated...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.promoted • serp_jobs.job_card.new
    Manufacturing Test Software Engineer

    Manufacturing Test Software Engineer

    Ouster • San Francisco, CA, US
    serp_jobs.job_card.full_time
    serp_jobs.filters_job_card.quick_apply
    At Ouster, we build sensors and tools for engineers, roboticists, and researchers, so they can make the world safer and more efficient. We've transformed LIDAR from an analog device with thousands o...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days