What will LLM-powered software look like in the medium-term future?

While I agree some longform articles could use a bulleted summary, I think the act of blog-writing is as much an exercise of the writer coming to terms with the subject as us enjoying the piece. Abridging an idea comes easier after exploring its breadth and width, and I am supportive of anyone who tries it out in the open.

You nailed it. In 2027 the AI is going to write the fluff for the Author and then extract the bullet points for the reader - Everybody saves time, winwin.

and for some there will be a "please elaborate" command of course

@vishnumenon, thanks for writing this up. I might do a followup blog on this, for now here is a comment :)

This article is close to my heart as I've been working on craftcraft.org with similar perspective.

1. Chat UX Consolidation: I agree that having crappy chat UIs everywhere is very suboptimal. Perhaps having a complete UX as a component is another solution here. We took many months to get http://chatcraft.org from prototype to an ergonomic productivity tool. Highly unlikely such attention will be paid to every chat UI integration.

2. Persistence Across Uses. This one is tricky. We keep all of our history client-side...but after using it this way, having a shared history server-side and having it pulled in as relevant context would be a nice improvement.

3. Universal Access: It's super weird to have LLMs restricted to providing output that you cut/paste. We have integrated pretty slick openai function interface to allow calling out to custom modules. So far we integrated: pdf conversion/ingestion, clickhouse analytics and system administration using a webrtcshell connector. Demo here: https://www.youtube.com/watch?v=UNsxDMMbm64

I've also investigated teaching LLMs consume UIs via accessibility UIs. I think this is underexplored. Blog post on that here: https://taras.glek.net/post/gpt-aria-experiment/

3b. LocalLLMs. These have been underwhelming so far vs openai ones(except maybe WizardCoder). Industry seems to be standardizing around openai-compatible REST interface(ala S3 clones). We have some support for this in a wip pull req, but not much reason to do that yet as the local models are relatively weak for interactive use.

4. Dynamically Generated UI & Higher Level Prompting: I do a lot of exploration by asking http://chatcraft.org to generate some code and run it to validate some idea. Friend of mine built basic UX for recruiting pipelines, where one can ingest resume pdfs into chatcraft and via custom system prompt have chatcraft become a supervised recruiting automation. We also do a lot generation of mermaid architecture diagrams when communicating about code. I think there a lot of room for UX exploration here.

Now a few categories that weren't covered:

1. Multi-modal interaction: It's so nice to be able to have chat with the assistant and then switch to voice while driving or to input some foreign language. I think extending UX from chat to voice and even video-based gestures will make for an even cooler AI assistant experience.

2. Non-linearity in conversations: Bots are not human, so it makes sense to undo steps in conversation, fork them, re-run them with different input params and different model params. Most of my conversations in chatcraft are me trying to beat llm into submission. Example: tuning chan-of-density prompt https://www.youtube.com/watch?v=6Vj0zwP3uBs&feature=youtu.be

Overall, really appreciate your blog post. Interesting to see how our intuition overlaps.

https://chat.openai.com (GPT-3.5) to summerise this article:

- summerise (then paste in the text)

- explain like I'm 10

Gave (the highlights)

-- ai here

1. Chat consolidation

2. Remembering stuff

3. Access to Everything

4. Making things easier

5. Being helpful

-- human here below

I'd love for the author to be right in his predictions. Arguably current AI has made understanding this article '4. easier' for me to understand already.

Nowhere near that, the largest problem will be when the misinformation will be manufactured on an unprecedented scale by these LLM. In the US alone some 300 000 people died from homemade artisan anti vaxx propaganda, it's very scary to think what will happen when GPT switches that to industrial scale. Doctorow called them "plausible sentence generators" and that's indeed what they are and it being plausible will make it deadly.

https://www.npr.org/2022/05/16/1099290062/how-many-of-americ...

> As the U.S. marks one million people dead from COVID-19, scientists suggest that nearly one third of those deaths could have been prevented if more people had chosen to be vaccinated.