Anthropic won't do it, but they published the j-lens to introspect the model- from what I understand it's roughly simply feeding a chosen layer straight into the final layers of the LLM for decoding into language:

https://github.com/anthropics/jacobian-lens

Looks like it should be easy to use on open weights models.