Anthropic Open-Sources Circuit Tracing Tools for AI Models

Anthropic has open-sourced its circuit tracing tools, allowing researchers to delve deeper into the inner workings of AI models. Announced on their website, the tools generate attribution graphs that partially reveal the decision-making steps of large language models. These graphs can be explored interactively via a frontend hosted by Neuronpedia.

The open-source library supports popular open-weight models, enabling users to trace circuits, visualize, annotate, and share graphs. Researchers can also test hypotheses by modifying feature values and observing changes in model outputs. This initiative, led by participants in the Anthropic Fellows program and in collaboration with Decode Research, aims to foster a broader understanding of AI model behaviors.

Anthropic's CEO, Dario Amodei, has emphasized the importance of interpretability research, noting that understanding AI's inner workings is crucial as AI capabilities advance. By making these tools available to the community, Anthropic hopes to encourage further exploration and improvement of AI interpretability tools.