Anthropic AI Safety Fellow, London

About Anthropic

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.

Responsibilities:

The Anthropic Fellows Program is a 6-month external collaboration program focused on accelerating progress in AI safety research by providing promising talent with an opportunity to gain research experience. Our goal is to bridge the gap between industry engineering expertise and the research skills needed for impactful work in AI safety.

Fellows will use external infrastructure (e.g. open-source models, public APIs) to work on an empirical project aligned with our research priorities, with the goal of producing a public output (e.g. a paper submission). Fellows will receive substantial support - including mentorship from Anthropic researchers, funding, compute resources, and access to a shared workspace - enabling them to develop the skills to contribute meaningfully to critical AI safety research.

We are piloting this program with a cohort of 10-15 new collaborators. We aim to onboard our first cohort of Fellows in March 2025, with the possibility of more cohorts depending on applicant interest and logistical needs.

What To Expect

• Direct mentorship from Anthropic researchers
• Connection to the broader AI safety research community
• Weekly stipend of 1300 GBP & access to benefits
• Funding for compute and other research expenses
• Shared workspaces in Berkeley, California and London, UK

This role will be employed by our third-party talent partner, and may be eligible for benefits through the employer of record.

Mentors & Research Areas

Fellows will undergo a project selection & mentor matching process in March 2025. Potential mentors include

• Ethan Perez
• Jan Leike
• Andi Peng
• Samuel Marks
• Joe Benton
• Akbir Khan
• Fabien Roger
• Alex Tamkin
• Kyle Fish
• Nina Panickssery
• Mrinank Sharma
• Evan Hubinger

Our mentors will lead projects in select AI safety research areas, such as:

• Scalable Oversight: Developing techniques to keep highly capable models helpful and honest, even as they surpass human-level intelligence in various domains. 
• Adversarial Robustness and AI Control: Creating methods to ensure advanced AI systems remain safe and harmless in unfamiliar or adversarial scenarios.
• Model Organisms: Creating model organisms of misalignment to improve our empirical understanding of how alignment failures might arise.
• Model Internals / Interpretability: Advancing our understanding of the internal workings of large language models to enable more targeted interventions and safety measures.
• AI Welfare: Improving our understanding of potential AI welfare and developing related evaluations and mitigations.

For a full list of representative projects for each area, please see our blog post.

You may be a good fit if you:

• Are motivated by reducing catastrophic risks from advanced AI systems
• Are excited to transition into full-time empirical AI safety research and would be interested in a full-time role at Anthropic
  • Please note: We do not guarantee that we will make any full-time offers to Fellows. However, strong performance during the program may indicate that a Fellow would be a good fit here at Anthropic, and external collaborations have historically provided our teams with substantial evidence that someone might be a good hire.
• Have a strong technical background in computer science, mathematics, physics, or related fields
• Have strong programming skills, particularly in Python and machine learning frameworks
• Can work full-time on the fellowship for 6 months
• Have US or UK work authorisation, and are able to work full-time out of Berkeley or London.
  • We may be able to support Fellows based in other locations on a case-by-case basis.
• Are comfortable programming in Python
• Thrive in fast-paced, collaborative environments
• Can execute projects independently while incorporating feedback on research direction

We’re open to all experience levels and backgrounds that meet the above criteria – you do not, for example, need prior experience with AI safety or ML. We particularly encourage applications from underrepresented groups in tech.

Strong candidates may also have:

• Experience with empirical ML research projects
• Experience working with Large Language Models
• Experience in one of the research areas (e.g. Interpretability)
• Experience with deep learning frameworks and experiment management
• Track record of open-source contributions

Candidates need not have:

• 100% of the skills needed to perform the job
• Formal certifications or education credentials

Interview process:

To ensure we can start onboarding Fellows in March 2025, we will conduct interviews on a rolling basis but set hard cut-off dates for each stage. If you are not able to make that stage’s deadline, we unfortunately will not be able to proceed with your candidacy.

We aim to onboard our first cohort of Fellows in March 2025, with the possibility of more cohorts depending on Fellow interest. Please note that if you are accepted into the March cohort, we expect that you will be available for several hours of mentor matching in March, although you may start the full-time program later.

1. Initial Application and References (Complete by Jan 20, 2025)
   • Submit your application below!
   • In the application, we’ll also ask you to provide references who can speak to what it’s like to work with you.
2. Technical Assessment (Complete by Feb 3, 2025)
   • You will complete a 90-minute coding screen in Python
     • As a quick note - we know most auto-screens are pretty bad. We think this one is unusually good and for some teams, give as much signal as an interview. It’s a bunch of reasonably straightforward coding that involves refactoring and adapting to new requirements, without any highly artificial scenarios or cliched algorithms you’d gain an advantage by having memorized.
   • We'll simultaneously collect written feedback from your references during this stage.
3. 15-minute Recruiter Screen + Final Interviews (Complete by Feb 17, 2025)
   • Before the final interviews, you will chat with a recruiter about your background and logistics.
   • We will complete interviews on a rolling basis until February 10, after which we will conduct interviews at specific timeslots on pre-specified days.
   • The final interviews consist of three interviews: 
     • Take-Home Project (5 hours work period + 30 minute review) – You'll work on a research-focused project that demonstrates your technical and analytical abilities
     • Culture Interview (40 minutes) – Discussion of your values, motivations, and reasoning as they relate to your work
     • Research Discussion (15 minutes) – Brainstorming session with Ethan Perez to explore research ideas and approaches
   • In parallel, we will conduct reference calls.
4. Offer decisions (Complete by Mar 10, 2025)
   • We aim to extend all offers by March 10, and finalise our cohort shortly after.
   • We will extend offers on a rolling basis and set an offer deadline of 1 week. However, if you need more time for the offer decision, please feel free to ask for it!
   • After we select our initial cohort, we will kick off mentor matching and project selection in mid-March. This will involve several-hour project brainstorming session in the week of March 17, and follow-up discussions.

At each stage, you'll receive more detailed instructions via email. While we have hard deadlines for each stage, we will be assessing candidates and making offer decisions on a rolling basis, so we encourage you to complete each stage as soon as possible.

Please note: The salary and logistics below this section does not apply to this job posting (for example, we will pay Fellows a weekly stipend of £1300 instead of an annual salary of £1300).

The expected salary range for this position is:

Annual Salary:£1,300—£1,300 GBP

Logistics

Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.

Visa sponsorship: We do sponsor visas! However, we aren't able to successfully sponsor visas for every role and every candidate. But if we make you an offer, we will make every reasonable effort to get you a visa, and we retain an immigration lawyer to help with this.

We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed.  Research shows that people who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy, so we urge you not to exclude yourself prematurely and to submit an application if you're interested in this work. We think AI systems like the ones we're building have enormous social and ethical implications. We think this makes representation even more important, and we strive to include a range of diverse perspectives on our team.

How we're different

We believe that the highest-impact AI research will be big science. At Anthropic we work as a single cohesive team on just a few large-scale research efforts. And we value impact — advancing our long-term goals of steerable, trustworthy AI — rather than work on smaller and more specific puzzles. We view AI research as an empirical science, which has as much in common with physics and biology as with traditional efforts in computer science. We're an extremely collaborative group, and we host frequent research discussions to ensure that we are pursuing the highest-impact work at any given time. As such, we greatly value communication skills.

The easiest way to understand our research directions is to read our recent research. This research continues many of the directions our team worked on prior to Anthropic, including: GPT-3, Circuit-Based Interpretability, Multimodal Neurons, Scaling Laws, AI & Compute, Concrete Problems in AI Safety, and Learning from Human Preferences.

Come work with us!

Anthropic is a public benefit corporation headquartered in San Francisco. We offer competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and a lovely office space in which to collaborate with colleagues.