Member of Technical Staff, AGI Autonomy Human Feedback

Our team’s mission is to build a general-purpose agent that can do anything a human can do on a computer. We’re looking for an applied scientist to innovate new methods of collecting human data at scale. This will involve measuring human-agent interactions and building new evaluations of model capabilities. You’ll work alongside world class AI researchers, engineers, and program managers to identify and implement the best processes for scaling data collection and improving our tech for our early customers. This role is highly cross-functional, leveraging skills across machine learning engineering, machine learning research, and social/cognitive science research to advance our foundational agent in a rapidly-evolving technological environment.

Key job responsibilities
- Own the analysis, documentation, and visualization of our human data
- Build and maintain dashboards for cross-functional audiences
- Work closely with data scientists, engineers and researchers to create robust data pipelines and data collection tools
- Work closely with program managers to optimize data collection processes


About the team
Amazon has launched a new research lab in San Francisco to develop foundational capabilities for useful AI agents. We’re enabling practical AI to make our customers more productive, empowered, and fulfilled. In particular, our work combines large language models (LLMs) with reinforcement learning (RL) to solve reasoning, planning, and world modeling in both virtual and physical environments. Our research builds on that of Amazon’s broader AGI organization, which recently introduced Amazon Nova, a new generation of state-of-the-art foundation models (FMs).

Our lab is a small, talent-dense team with the resources and scale of Amazon. Each team in the lab has the autonomy to move fast and the long-term commitment to pursue high-risk, high-payoff research. We’re entering an exciting new era where agents can redefine what AI makes possible. We’d love for you to join our lab and build it from the ground up!

Basic Qualifications

- Bachelors Degree in applicable field of study , or equivalent work experience
- Demonstrated expertise in relevant business/data analysis experience
- Expert proficiency with SQL, Excel, ETLM process and data visualization tools like QuickSight, Power BI or similar platforms
- Experience defining requirements and using data and metrics to draw business insights
- Experience making business recommendations and influencing stakeholders
- Demonstrated ability to consistently maintain a high level of integrity and discretion to handle confidential information.
- Proven superb customer service to the leadership team and present complex information clearly and concisely
- Demonstrated written and verbal communication skills.
- Experience linking and handling large datasets, and conducting statistical analyses (e.g., regressions, t-tests, etc.)

Preferred Qualifications

- Experience working with human annotation teams for training AI models
- Proven experience in project management such as Lean Six Sigma
- Experience with Python, R and other data science tools
- Experience with business intelligent product development and/or automations
- Demonstrated program ownership and consistent delivery on commitments - great organizational skills with exceptional follow through and attention to detail
- Highly flexible, with the ability to identify, analyze and solve problems systematically in a fast-paced, rapidly-evolving organization
- Proven ability to manage multiple projects and competing priorities simultaneously
- Proactive, organized, and detail-oriented; able to ensure rigor and accuracy while meeting deadlines
- Excellent written and verbal communications skills - ability to interface with all levels of the organization, specific strength in influencing decision makers and managing expectations

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Our compensation reflects the cost of labor across several U.S. geographic markets. The base pay for this position ranges from $120,000/year in our lowest geographic market up to $350,000/year in our highest geographic market. Pay is based on a number of factors, including market location, and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. Applicants should apply via our internal or external career site.