Practical approach for streaming UI from LLMs
timetler.com
Practical approach for streaming UI from LLMs
1–10 of 11 posts
Re: Practical approach for streaming UI from LLMs
#2Obviously the next step and already happening but for some reason I doubt it'll be this exact technology. We're probably going to see a lot of competition here.
Re: Practical approach for streaming UI from LLMs
#3This was a difficult read, not necessarily due to the merry-go-round of trying to find the right fit, but the overhead involved and the maintenance burden this is going to become over time. Once it starts it's never going to be just another thin layer.
I'd prefer taking a step back to see if this is the right approach or if this safety can be solved in a much simpler way. Looking at the title of the submission I don't consider it practical. I would welcome any other ideas, I'm thinking along the lines of a simple and boring bff which does the parsing and validation and sending strongly typed objects to the front end.
Re: Practical approach for streaming UI from LLMs
#4Chatgpt already looks a lot like that, tbh.
Re: Practical approach for streaming UI from LLMs
#5I quite like this, specially coming from the idea of creating custom UIs depending on the type of output required for the answer. Could be a great tool for visual learners.
Re: Practical approach for streaming UI from LLMs
#6Obviously the next step and already happening but for some reason I doubt it'll be this exact technology. We're probably going to see a lot of competition here.
Author here. I'd love to see it! I settled on this approach after not finding any other ways to do it, but it's definitely not the only way. I'd like to see what other approaches people might come up with.
Re: Practical approach for streaming UI from LLMs
#7Interesting- I wonder if we can integrate this approach with a framework like AG-UI (as opposed to copilotkit)
Re: Practical approach for streaming UI from LLMs
#8Really nice writeup!
I've been thinking about this approach. How do you handle updates on this setup? For example, a status component that goes from "loading" to "success".
Does the LLM go back and modify the previous response, sending deltas to the client? Effectively changing the component from loading to success. Or is something else happening behind the scenes?
Re: Practical approach for streaming UI from LLMs
#9Starred in GitHub.
I use assistant-ui to handle my frontend rendering. It would make sense for them to adopt something like this
Re: Practical approach for streaming UI from LLMs
#10Really nice writeup! I've been thinking about this approach. How do you handle updates on this setup? For example, a status component that goes from "loading" to "success". Does the LLM go back and modify the previous response, sending deltas to the client? Effectively changing the component from loading to success. Or is something else happening behind the scenes?
The components are standard react components so once it is rendered they can set their own state and effects and load in their own data.
I hope I understood your question correctly?