Artificial Intelligence1 min read

Tools for exploring the souls of large language models

By · Published by Everything Blog

In short

The community researching internal model mechanics has expanded from a handful of experts in 2021 to hundreds of contributors today, despite persistent funding gaps across the broader safety landscape. To broaden access to these evaluation techniques, creators released an open-source project called the TinyGPT Visualizer on GitHub. Built to function like an X-ray machine, the program presents all working elements of a functional miniature model across one screen. Researchers believe any true evidence of artificial internal states or emotions will ultimately reside within these observable numerical values.

Key points

  • Foundation model developers face ongoing criticism for funneling high computing budgets i…: Foundation model developers face ongoing criticism for funneling high computing budgets into new systems rather than safety research.
  • Unlike the human brain, artificial intelligence models can be paused to let researchers i…: Unlike the human brain, artificial intelligence models can be paused to let researchers inspect active numerical matrices during inference.
  • Investigators argue that potential signs of internal states or emotions in models will ul…: Investigators argue that potential signs of internal states or emotions in models will ultimately be detected through concrete numeric values.

Sources checked by our analysis pipeline: 1

There is that old joke told by Woody Allen describing his experience as a university student: “I was thrown out of college for cheating on the metaphysics exam; I looked into the soul of the boy sitting next to me.” If he’d managed to pass his course, Woody Allen might have been ahead of his time.

Metaphysics is becoming a popular topic today, and it seems now everyone desperately wants to look for souls in large language models (LLMs). A recent conference last April was held by Anthropic with religious scholars and philosophical thinkers to discuss the nature of the soul when it came to their latest Claude models. It seems they needed guidance that Dario Amodei was unable to provide.

The time seems ripe to dig into how we might actually peer into the mind of the LLM sitting next to you. There’s been a lot of ongoing research; however, the self-same foundation model companies, Anthropic and OpenAI among them, have been criticized for dedicating more compute dollars into better models rather than the safety research. Mechanistic interpretability, the…

Artificial Intelligence