Applied Scientist, Conversational Assistant Services

Applied Scientist, Conversational Assistant Services

Job ID : 2742001 Amazon.com Services LLC

As part of Alexa CAS team, our mission is to provide scalable and reliable evaluation of the state-of-the-art Conversational AI.

We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), Natural Language Processing (NLP), to invent and build end-to-end evaluation of how customers perceive state-of-the-art context-aware conversational AI assistants.

Check out the role overview below If you are confident you have got the right skills and experience, apply today.

A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas.

The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc.

As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel methods for evaluating conversational assistants.

You will analyze and understand user experiences by leveraging Amazon’s heterogeneous data sources and build evaluation models using machine learning methods.

Key job responsibilities

• Design, build, test and release predictive ML models using LLMs
• Ensure data quality throughout all stages of acquisition and processing, including such areas as data sourcing / collection, ground truth generation, normalization, and transformation.
• Collaborate with colleagues from science, engineering and business backgrounds.
• Present proposals and results to partner teams in a clear manner backed by data and coupled with actionable conclusions
• Work with engineers to develop efficient data querying and inference infrastructure for both offline and online use cases

About the team

Central Analytics and Research Science (CARS) is an analytics, software, and science team within Amazon's Conversational Assistant Services (CAS) organization.

Our mission is to provide an end-to-end understanding of how customers perceive the assistants they interact with from the metrics themselves to software applications to deep dive on those metrics allowing assistant developers to improve their services.

Learn more about Amazon’s approach to customer-obsessed science on the Amazon Science website, which features the latest news and research from scientists across the company.

BASIC QUALIFICATIONS

• PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
• Experience building machine learning models or developing algorithms for business application
• Experience in patents or publications at top-tier peer-reviewed conferences or journals
• Experience with programming languages such as Python, Java, C++
• Hands-on experience and deep understanding of the Large Language Models architectures

PREFERRED QUALIFICATIONS

• Research experience in conversational assistant or LLM evaluation
• Publications at peer-reviewed NLP / ML conferences (e.g. ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, AAAI, etc.)
• Hands-on experience with using RLHF models

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.

J-18808-Ljbffr