We will fund great founders and great companies who are broadly interested in building commercial tools which will help protect us from negative outcomes from the deployment of AI everywhere in business and society. These are some of the themes in which we are most interested, but this is not meant to be a complete list of all possible categories of AI safety.
Alignment and Interpretability Tools
Products that aid in the development and delivery of aligned models as well as clarify the decision making process of AI entities.
Security Infrastructure:
Products that help AI companies protect their intellectual property, including, for example, model weights.
Governance and Compliance Infrastructure
Products that regulators from industry or government can use to detect and ensure compliance.
Weapon Safety Systems
Products that can detect or prevent dangerous AI weapons from being developed and / or deployed, for example bioweapons or autonomous weapons.
Trust Preservation
Tools that help make society more resilient to mis/disinformation and other AI-driven disruptions, thereby helping to restore and maintain trust in important societal institutions. Included here would be tools to prevent or detect AI-generated attacks, scams, fraud, and even propaganda.
Model Safety and Benchmarking
Products to allow industry and / or regulators to rapidly determine in advance if a model or technology may be approaching dangerous capabilities.
Decision-making Tools
Tools that can help ensure that key decision makers have what they need to make better decisions. This might mean better AI advisors or better forecasting markets.
Negotiation Tools
AI systems that can explore potential trades between many parties, incorporate secret information without sharing it, and help suggest mutually-beneficial trades.