This newsletter made possible by MosaicML. Progress measures for grokking via mechanistic interpretability This paper is a win for mechanistic interpretability. If you haven’t heard that phrase, it means understanding the exact logic a model uses to map inputs to outputs.
If you liked the parametrisation in the Tracr, you'll LOVE Anthropic's "A Mathematical Framework for Transformer Circuits". This is where I've first seen this parametrisation, and it also contains some other nuggets for describing transformers in more intuitive and useful terms.
Thanks for your newsletter. I had one piece of feedback, is there a way to add a table of contents at the top in substack? Line or breaks betweenpapers are very hard to see in substacks at the moment.
Hi David,
If you liked the parametrisation in the Tracr, you'll LOVE Anthropic's "A Mathematical Framework for Transformer Circuits". This is where I've first seen this parametrisation, and it also contains some other nuggets for describing transformers in more intuitive and useful terms.
https://transformer-circuits.pub/2021/framework/index.html
Best,
Jan
Hi David,
Thanks for your newsletter. I had one piece of feedback, is there a way to add a table of contents at the top in substack? Line or breaks betweenpapers are very hard to see in substacks at the moment.
Thanks