About the Team
OpenAI's Human Data Team creates custom data solutions driving groundbreaking research. Our work enhances and evaluates our flagship models and products like ChatGPT, GPT-4, and Sora, and contributes to safety initiatives through collaboration with our Preparedness and Safety Systems teams.
We work with AI trainers to gather specialized data for training and evaluating our models across modalities such as video, audio, text, and tool actions.
Our goal is to develop scalable methods, tools, and platforms to generate and evaluate high-quality data from both synthetic sources and human experts in various fields, including mathematics, sciences, creative writing, programming, art, and safety.
We leverage OpenAI models to improve and streamline our data collection and quality processes.
About the Role
In this role, you will work side by side with research teams to iteratively design and evaluate new methods for collecting high quality human data.
You’ll use your knowledge of human cognition and data quality to co-design experiments, write instructions and assess the efficacy of our data collections.
In parallel, you’ll partner with engineering and operations to implement changes and iteratively improve how we run human data campaigns.
This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.
In this role that’s the first of its kind, you will :
Collaborate closely with researchers to understand, predict, and design tasks and quality rubrics to collect great human data.
Engage with data labelers and vendors to evolve training and feedback mechanisms
Create hiring assessments and brainstorm acquisition strategies for high skill data labelers
Design and conduct experiments to assess the impact of interventions on human data quality
Come up with creative strategies for collecting and assessing high-quality data.
Generate golden datasets and evaluation techniques
Work on multiple projects with tight timelines.
Do whatever needs to be done to make our models better.
You might thrive in this role if you :
Have a background in cognitive science, computational linguistics, human-computer interaction
Enjoy tackling big questions in an ambiguous and fast paced environment
Love being at the intersection of research and operational execution