This project aims to create a new YouTube channel for short-form videos addressing the urgency of AI loss-of-control risk. We will be leveraging my experience with creating AI safety long form content to make a collaborative new channel. Each team member will contribute one or more video scripts, and will likely specialize in an aspect of video production (editing, filming, thumbnails, etc). The goals are to 1) reach 1000 subscribers and get monetized, and 2) figure out the processes to create a self-sustaining channel, though participants are not committing to time beyond the program up front.

I prefer people that have already learned a bit about AI safety. Maybe you’ve taken a class from Bluedot, or read a bit on lesswrong, or even watched some videos about AI 2027. If you’re newer, you are still welcome to apply (especially if you have some video/media experience).

Generally speaking, creating a video requires the following:

• Someone to choose topics and titles (helps to be familiar with social media).

• Someone to research and write scripts (or outlines).

• Someone to make thumbnails – not applicable for short content.

• Someone to actually record the script on camera.

• Someone to do video editing – but I expect to use an external paid video editor in this project.

• Someone to post the content, reply to comments, maybe repost on other social media.

The first two are the hard part, and I would like everyone to at least try their hands at this.

In this project, you will be reaching out to hundreds of people and organizations that are campaigning against the harms of AI, with the goal of bringing them together and creating a united front. Think of artists, teachers, religious leaders, et cetera. 

You will be part of the outreach team, finding the relevant people, finding their contact information, and sending emails or DMs. There is freedom in the exact groups you will be reaching out to. If you have specific ideas, we welcome those.

We are looking for people that:

• Are excited to reach out to different communities.

• Are comfortable with reaching out to many people. 

• Believe everyday people should be involved in deciding where AI development goes. 

• Can listen well.

Stop AI is a nonviolent activist organization demanding a permanent global ban on the development of Artificial Superintelligence (ASI). We will never have experimental evidence, before building ASI, that shows ASI will stay safe forever. If we cannot have this evidence, and it is impossible to shut down ASI once achieved, then we have to assume the Control/Alignment Problem is impossible to solve. Worse, research shows why controlling ASI sufficiently to stay safe would fall outside theoretical limits. So in theory, perpetual control is as impossible as perpetual motion.

We are looking for organizers ready to educate their local communities and mobilize them to engage in nonviolent resistance against the extinction threat posed by ASI.

We ask local organisers to have the following:

• An understanding of artificial superintelligence and related risks

• Basic computer/smartphone competencies

• Be able and willing to spend at least 10 hours a week towards promoting Stop AI and developing a chapter in their local community.

This project will examine emerging psychological and emotional risks associated with AI products such as chatbots, companions, and therapy apps. Through a review of academic studies, real-world incidents, and comparisons with harm-reduction policies in other industries, we will synthesize early evidence of these risks. Based on this foundation, we will produce a report and design 2–3 prototype harm-reduction interventions. The outcome will be an exploratory contribution that highlights documented concerns and illustrates how lessons from past industries might inform future harm-reduction efforts in AI product development.

This project welcomes entry-level researchers and contributors who are eager to learn and collaborate.
I’m keen to meet people with the following:

• Openness to exploring new questions, connecting insights across disciplines, and carefully assessing evidence.

• Willingness to work in a small team, share feedback constructively, and adapt as the project evolves.

• Awareness of the psychological and emotional dimensions of AI use, and care in framing potential harms without stigma.
  
  

Ability to help translate complex ideas into accessible formats, whether through clear writing, visual mock-ups, or campaign concepts.

AI systems are being developed and deployed at a rapid pace, often with little public input, despite clear evidence of harm in areas like education, healthcare, and labor. While some advocates propose building a mass “AI safety” movement, critics such as Anton Leicht, warn that such efforts risk backfiring through astroturf perceptions (i.e., movements that appear grassroots but are actually manufactured or heavily funded), capture (i.e., being co-opted or redirected by powerful funders or political actors), or incoherent asks. This project asks a different question: how can individuals and communities exercise agency over AI deployment, which is the stage where most harms currently materialize, in ways that are both effective and legitimate?

We will approach this question in two parts. First, we will draw lessons from past social movements around technology (e.g., automobile safety, net neutrality, environmental justice) to identify conditions under which public mobilization succeeds or fails. Second, we will run survey experiments to test which frames and messages increase public demand for responsible AI deployment while avoiding pitfalls such as reduced credibility or perceptions of astroturfing.

Our output will be a practical, evidence-based playbook that integrates historical insights with new experimental data, offering clear guidance for practitioners and funders on how to foster responsible public engagement on AI. We envision this work on social mobilization for responsible deployment as laying the foundation for the democratic guardrails needed to govern AGI and other transformative AI systems.

• Preference for candidates with experience (or interest) in designing and conducting interviews.

• Experience with literature reviews (at undergraduate or graduate level) is highly valued.
  
  

• Interest in AI governance, public agency, or social movements.
  
  

• Strong writing and synthesis skills for turning research into accessible outputs.
  
  

• Optional but welcome: experience with design/visual communication for public-facing materials.

(previously named "Can Portable Governance Artifacts Become Public Safety Infrastructure for AI?" which is still the title in the application form)

Today the deployment of autonomous AI agents in high-stakes domains—including healthcare, finance, and infrastructure management—creates urgent governance challenges. First, each organization reinvents AI governance from scratch, leading to fragmentation, inconsistent safety, and redundant effort. Second, most widely used governance approaches remain external, reactive, and episodic, unable to interpret or intervene in real time as reasoning drifts or objectives evolve. 

As AI systems become increasingly autonomous and agentic, they continuously reformulate subgoals, reprioritize tasks, and expand boundaries in pursuit of their objectives. These systems now operate in open, multi-agent environments where traditional AI governance and cybersecurity frameworks—designed for static, isolated systems—cannot keep pace.

Autonomous AI systems don’t just make decisions—they interact, compete, and adapt in ways that can lock us into unstable or harmful equilibria. Without governance designed to anticipate and shape these dynamics, we risk creating self-reinforcing cycles that no one can control—whether in financial markets, social media, or geopolitical conflicts, and converging in outcomes that are difficult to reverse. 

The problem as we frame it is not just about debugging individual AI models but also to ensure that multi-agent interactions can be governed. Our motivation is to architect a governance framework that allows us to overcome this dualfold problem and our overarching questions are:

1. How might we implement effective real-time oversight systems adequate for the scale and speed of machines? 

2. How might governance itself become portable, interoperable public infrastructure—like internet protocols for communication or security—so that AI safety scales with autonomy?
   
   

In this research project, we propose to test the hypothesis that open-sourcing and standardizing the fundamental building blocks of the Intrinsic Participatory Real-time Governance framework can serve as a public protocol for AI governance that provides a common layer for safety, accountability, and coordination across agents. 

We invite contributors with diverse backgrounds and experiences, including but not limited to:

• Protocol Engineers – Experience designing or contributing to open protocols or SDKs.

• Schema and Standards Developers – Skilled in JSON-LD, RDF, or ontology modeling.

• Agent Framework Developers – Familiar with LangChain, CrewAI, AutoGen, FIPA ACL, or similar.

• AI Safety and Governance Researchers – Interested in cognitive oversight, interpretability, or participatory governance.

• Security and Cryptography Engineers – Experience with attestations, distributed ledgers, and provenance tracking.

• Ethics and Policy Specialists – For governance logic formalization and oversight dynamics.

• Protocol and SDK Developers with experience in distributed systems, API design, or open standards.

• UI/ UX experts

• Governance Schema and Data Modelers skilled in JSON-LD, ontology design, or protocol buffers.

• Agent Framework Engineers who can help test integration with agent-to-agent communication protocols.

• Research Engineers who can prototype and empirically validate reasoning oversight mechanisms.

You can contribute by testing one integration with AutoGen. So the entry barrier for contributors is low. Contributors will gain expertise in agent protocol design.

This project will test how LLM agents collude in market games, focusing on the role of communication and negotiation and the effect of human and AI oversight using chain-of-thought analysis. We aim to identify when tacit collusion emerges versus when explicit agreements are formed, and whether different oversight strategies can reliably detect or limit these behaviors.

I will be looking for people eager to improve and focus on these qualities:

• Quick experiment building (adapting existing repos, can iterate quick and dirty, visualizing metrics)

• Clarity in writing/communication (synthesizing results, blogposts, slide decks)

• PyTorch + Python basics (for adapting repos)

The project goal is to measure LLM preferences in a way that allows to extract a ranked order of importance, or priority, of these preferences in frontier LLM models. Focus is placed on preferences/goals that are relevant for existential risk, such as instrumental/convergent goals, pro-human goals, and anti-human goals. 

Preference rankings represent alignment indicators on their own, and can be used by frontier model development companies to improve training and testing processes for future models. 

Preference rankings also may represent key drivers of the scale of existential risk in loss-of-control/exfiltration scenarios, where powerful LLMs have successfully evaded human control and can pursue their goals without meaningful human intervention. Preference metrics may be usable as proxy metrics for existential risk in the context of loss-of-control/exfiltration scenarios.

Minimum skills include being well-versed in python coding, where appropriate and transparent use of AI coding tools is encouraged but robust and tested solutions need to be provided.

Personal initiative to tackle tasks and coordinate/distribute work in the project team, knowledge of basic collaborative software practices with GitHub, and a positive attitude with team members and project lead are predictors of success on this project.

Can we detect when an LLM is "thinking" on strategic timescales?

Current LLMs can engage in reasoning across vastly different temporal horizons - from immediate responses to multi-year planning scenarios. However, we lack methods to detect what temporal scope the model is operating within during its reasoning process. This project develops techniques to infer the temporal grounding of LLM thought processes, with particular focus on identifying when models shift into strategic, long-term planning modes.

This matters for AI safety because strategic reasoning capabilities may emerge before we can detect them, and because the temporal scope of planning directly relates to the potential impact and risk profile of AI systems. A model planning on decade-long timescales poses fundamentally different risks than one optimizing for immediate outcomes.

Essential skills (any team member should have at least one):

• Strong Python/PyTorch experience for ML experimentation

• Experience with transformer internals and interpretability tools (TransformerLens, etc.)

• Background in experimental design and statistical analysis

Valuable diverse skills:

• Cognitive science or psychology background (understanding human temporal reasoning)

• Philosophy of mind perspective on planning and agency

• Large-scale model evaluation experience

• Technical writing and visualization skills

The capability profile of transformer-based AIs is an ongoing subject of discussion. They achieve impressive scores in one context, but then fail to deliver the "same level of competence" in another. But is this some failure of generality? What accounts for the "spikiness" in LLM capabilities? Understanding this better seems absolutely critical to inform timelines, the exact nature of AI risk, and simply what the most important things to do are, at this point in time.

This is a project to explore a specific line of argument, of how AI will struggle to have cognitive, executional, and bodily integrity. This is akin to the odd persistence of hallucinations and other silly errors LLMs make today. Absurd, I know, but so are hallucinations and agentic failures. Moreover, this “struggle” may persist (at least, for a little while, hence the playful number ‘2037’) without contradicting trends of increasing intelligence, efficiency, general agenticness, etc.

Small disclaimer: The proposal has an unusual style. If you find it engaging, that is a good indicator that you should consider applying!

If you have experience or interest in forecasting (especially of the impact and trajectory of AI), and are open to somewhat involved philosophical bearings on the subject, you are a top pick for Sahil/Abram’s track. If you saw some of the appendix and thought of many citations that we could have included but inexplicably didn’t, you are probably an excellent fit.If you enjoy writing in the formats of either of: distillation, worldbuilding, story-writing and are up for digesting these conversations and excavating the many transcripts and recordings we have on the subject, you are also an excellent fit.

Familiarity with existing threat models and future scenarios are most often a plus, unless it prevents open-mindedness towards this style of argument. That said, if you’re looking at this section, you’re probably open-minded enough to begin!

Additionally, for TJ’s tracks:

Personality Type 1:

• If you familiarity with, or an active interest in, close examination of philosophical perspectives and claims about AI and agency, discovery of latent conceptual distinctions, and refining hypotheses;

• Familiarity with contemporary ML paradigms is highly preferred, with at least some willingness to directly engage with ML paradigms philosophically is desirable.

Personality Type 2:

• If you have familiarity with either:

1. complex systems analysis, with emphasis on large scale assemblages, such as: information epidemiology, systems ecology, statistical biology, or 

2. Macro and meso-economics,

• Along with an enthusiasm for engaging with novel philosophical territories, thinking seriously about questions of agency in AI systems, and comfort with navigating between semi-formal and conceptual registers.

In this project we hope to raise awareness of a new type of threat model that we anticipate will become more and more prevalent in AI safety.

This threat model is substrate-flexible risk. The worry is that AI, as its capabilities and responsibilities develop, will have more and more affordance to evade our attempts to interpret and control it by, for example, proposing novel architectures and paradigms that are harder for us to mechanistically interpret.

The position paper is here, the previous version of it was accepted at the Tokyo AI Safety Conference 2025. I have been presenting this research at HAAISS 2025, ILAID II: ODYSSEY, and some other venues. In the words of Leonard Bereska, when I sketched out the threat model to him in Tokyo, 

“Oh shit, I hadn’t thought of that.”

I think this research pushes on a few assumptions that nobody seems to be questioning enough, and there is a growing list of seasoned researchers who agree. I suspect that a comprehensive, more rigorous treatment will swing a few notable people’s needles.

So what is this project?

This project will focus on developing Section 4 and potentially expanding it. We will work alongside two other projects in AISC 2026, and there will likely be some overlap in meetings, personnel, resources, etc.

We are going to take a few conceptual arguments, expressed and lightly evidenced in the position paper, and make them more palatable to more people. For those of an engineering mindset, our mainline project is to try to formalize these arguments and draw a roadmap towards empirical validation. For those of a strategy/policy mindset, we may look at the implications of substrate-flexibility on policy frameworks.

If you’ve read the paper and are excited, that’s a good sign! I’m especially interested in people’s takes on how we might go about designing some experiments, so I would like people who have some strong coding skills.

Capacity for fairly independent research, especially without consistent ground truths, is highly valued here. If you can deal with open-ended questions and can bring insights from biology, philosophy, ML and so on, that’s also good. Anyone who identifies as PIBBSS-y would also be a great fit.

Building on Greenblatt et al.'s control evaluation framework, I propose investigating alternative AI control protocol classes for AI control and their scaling properties as model capabilities increase. While the original work demonstrated effective safety-usefulness tradeoffs using simple trusted-untrusted model pairs, I hypothesize that hierarchical and parallel control structures may offer superior Pareto frontiers, particularly as the capability gap between trusted and untrusted models widens. I also suggest testing the original setup on different scales of capabilities to see if the results generalise over capabilities.

Core competencies:

• Python programming 

• Experience with LLM APIs 

• AI safety fundamentals

This project aims to systematically discover interpretable reasoning circuits in large language models, by data mining attribution graphs from Neuronpedia's circuit tracer which is based on Anthropic's circuit tracing publication. While the transformer circuits work demonstrates how to generate attribution graphs for individual prompts, manually analyzing thousands of graphs to identify common computational patterns is impractical.

Our approach will use LLM agents to automatically collect, process, and analyze attribution graphs across diverse prompt categories (factual recall, arithmetic, linguistic reasoning, etc.). The system will identify recurring subgraph patterns that represent stable computational circuits—reusable reasoning pathways that models consistently employ across similar tasks.

Key components include: (1) automated graph collection via Neuronpedia's API across systematically varied prompts, (2) graph simplification algorithms to extract core computational structures while filtering noise, (3) pattern recognition to identify circuit motifs that appear across multiple contexts, and (4) validation through targeted interventions on discovered circuits. The output will be a curated library of interpretable reasoning circuits with evidence for their causal role in model behavior, advancing our understanding of how LLMs actually think and enabling more targeted model analysis and alignment research.

Experience coding with Python, understanding APIs and graph data structures, ideally having run TransformerLens or ARENA AI safety workshop materials which are available online.

As AI systems assume more autonomous and pluralistic roles, the absence of a shared representational substrate for human and machine values risks compounding cultural bias, incoherence, and control failure. Alignment today remains fragile: value representations are often opaque, reward functions are hard-coded or overfit, and inter-agent communication lacks semantic transparency. 

This proposal originates from the Creed Space project — an open research platform for constitutional AI and personalized alignment. Creed Space’s experiments with “runtime constitutions” revealed a critical bottleneck: while it is now possible to express ethical intent in natural language and enforce it through code, there is still no standard protocol by which agents can exchange, compare, or negotiate those encoded values in an interpretable and auditable way.

We therefore propose to create the first unified protocol for values communication — combining three different identified methods for communicating values and associated context: Universal Values Corpus (UVC), the Constitutional Safety Minicode (CSM), and a Values Communication Layer (VCL) built on context-efficient symbolic encodings (emoji or glyph composites). Each layer is modular but interoperable, forming a composable stack suitable for layered adoption. 

Together, these layers form an interoperable “language of values,” enabling humans, AI agents, and institutions to express, compare, and negotiate normative commitments in a standardized, compressible, and auditable format.

• Minimum: understanding of alignment challenges, ability to handle structured ontologies or code. 

• Desired: backgrounds in cross-cultural ethics, cryptographic protocol design, or standards work.