Finding the Words to Say: Hidden State Visualizations for Language Models
1–5 of 5 posts
Re: Finding the Words to Say: Hidden State Visualizations for Language Models
#2Hello HN, author here. Language models are absolutely fascinating tools. I believe it would pay for software engineers to have a sense of their capabilities and how they function. The article showcases a few views to expose the inner workings of the model, but also simple UI for interacting with a language model to get a sense for how they work and generate words.
If you prefer video, I have also recently released a video [1] with PyData to provide an intro to language models and their applications and how we're trying to make Transformer-based ones more transparent with Ecco[2]. Contributors are welcome!
[1] https://www.youtube.com/watch?v=rHrItfNeuh0
[2] https://www.eccox.io/ and https://github.com/jalammar/ecco
Thanks mods for merging submissions. Happy to get feedback , thoughts, or questions.
Re: Finding the Words to Say: Hidden State Visualizations for Language Models
#3Hello HN, author here. Language models are absolutely fascinating tools. I believe it would pay for software engineers to have a sense of their capabilities and how they function. The article showcases a few views to expose the inner workings of the model, but also simple UI for interacting with a language model to get a sense for how they work and generate words. If you prefer video, I have also recently released a…
Nice article, thanks for posting :-)
Re: Finding the Words to Say: Hidden State Visualizations for Language Models
#4Does some text generating tool exist which takes a badly written piece of prose (perhaps even in the form of bullet-points), and produces a high(er) quality text? (E.g., richer in vocabulary by using synonyms, removing repetition, rephrasing sentences so they don't all have the same general form, et cetera.)
Re: Finding the Words to Say: Hidden State Visualizations for Language Models
#5Does some text generating tool exist which takes a badly written piece of prose (perhaps even in the form of bullet-points), and produces a high(er) quality text? (E.g., richer in vocabulary by using synonyms, removing repetition, rephrasing sentences so they don't all have the same general form, et cetera.)
I didn't come across one yet personally