Senior/Staff Machine Learning Engineer - Foundation Models

The Prediction & Behavior ML team is responsible for developing machine-learned models that understand the full scene around our vehicle and forecast the behavior for other agents, our own vehicle’s actions, and for offline applications.

To solve these problems we develop deep learning algorithms that can learn behaviors from data and apply them on-vehicle to influence our vehicle’s driving behavior and offline to provide learned models to autonomy simulation and validation.

Given the tight integration of behavior forecasting and motion planning, our team necessarily works very closely with the Planner team in the advancement of our overall vehicle behavior.

The Prediction & Behavior ML team also works closely with our Perception, Simulation, and Systems Engineering teams on many cross-team initiatives.

The Behavior Foundation Model team is responsible for building GPT (Generative Pre-Trained) for autonomous driving and ecosystems for prediction and planning for autonomous driving and offline applications.

We're looking for experts in this field to come work with us in building these models and applying them to downstream tasks across the company.

All candidates should make sure to read the following job description and information carefully before applying.

Responsibilities

• You will build the foundation models for on-vehicle and offline applications.
• You will push the boundaries of machine learning in the autonomous driving industry by developing state-of-the-art techniques in self-supervised learning, semi-supervised learning, multi-task learning, and generative models.
• You will leverage our large-scale machine learning infrastructure to discover new solutions.
• You will develop new algorithms to model the future behavior of all other agents in the world.
• You will develop new algorithms to model the future behavior of our own vehicle’s future actions, both in predicting our driving trajectories and estimating their quality in relation to our goals of safety, progress, and comfort.
• You will develop new algorithms to apply generative deep learning to simulation to improve the realism of our offline validation systems.
• You will engineer software that runs on-vehicle to efficiently execute our algorithms in real time.
• You will develop metrics and tools to analyze errors and understand improvements of our systems.
• You will collaborate with engineers on Perception, Prediction, Planning, Simulation, and Systems Engineering to solve the overall Autonomous Driving problem in complex urban environments.

Qualifications

• BS, MS, or PhD degree in Computer Science or a related field.
• Fluency with PyTorch, TensorFlow, or JAX.
• Experience with training and deploying Deep Learning models.
• Experience with production Machine Learning pipelines : dataset creation, training frameworks, metrics pipelines.
• Fluency in Python and a basic understanding of C++.
• Extensive experience with programming and algorithm design.
• Strong mathematics skills.

Bonus Qualifications

• Experience in training of large and complex models.
• Experience with LLMs.
• Conference or Journal publications in Machine Learning or Robotics.
• Prior experience with Prediction and / or Autonomous Vehicles or Robotics in general.

Compensation

There are three major components to compensation for this position : salary, Amazon Restricted Stock Units (RSUs), and Zoox Stock Appreciation Rights.

The salary will range from $221,000 to $319,000. A sign-on bonus may be part of a compensation package. Compensation will vary based on geographic location, job-related knowledge, skills, and experience.

Zoox also offers a comprehensive package of benefits including paid time off (e.g. sick leave, vacation, bereavement), unpaid time off, Zoox Stock Appreciation Rights, Amazon RSUs, health insurance, long-term care insurance, long-term and short-term disability insurance, and life insurance.

About Zoox

Zoox is developing the first ground-up, fully autonomous vehicle fleet and the supporting ecosystem required to bring this technology to market.

Sitting at the intersection of robotics, machine learning, and design, Zoox aims to provide the next generation of mobility-as-a-service in urban environments.

We’re looking for top talent that shares our passion and wants to be part of a fast-moving and highly execution-oriented team.

Accommodations

If you need an accommodation to participate in the application or interview process please reach out to accommodations@zoox.

com or your assigned recruiter.

A Final Note :

You do not need to match every listed expectation to apply for this position. Here at Zoox, we know that diverse perspectives foster the innovation we need to be successful, and we are committed to building a team that encompasses a variety of backgrounds, experiences, and skills.

J-18808-Ljbffr