I'm not saying the content of the article is wrong, but what apps are people/companies writing articles like this actually building? I'm seriously unable to imagine any useful app. I only use GPT via API (as better Google for documentations, and its output is never usable without heavy editing). This week I tried to use "AI" in Notion: I needed to generate 84 check boxes for each day starting with specific date. I got 10 check boxes and line "here should go rest..." (or some variation of such lazy output). Completely useless.
What we've learned from a year of building with LLMs
111–120 of 148 posts
Re: What we've learned from a year of building with LLMs
#112Anyone have a convenience solution for doing multi-step workflows? For example, I'm filling out the basics of an NPC character sheet on my game prep. I'm using a certain rule system, give the enemy certain tactics, certain stats, certain types of weapons, right now I have a 'god prompt' trying to walk the LLM through creating the basic character sheet, but the responses get squeezed down into what one or two prompt r…
For me, a very simple "breakdown tasks into a queue and store in a DB" solution has help tremendously with most requests. Instead of trying to do everything into a single chat or chain, add steps to ask the LLM to break down the next tasks, with context, and store that into SQLite or something. Then start new chats/chains on each of those tasks. Then just loop them back into LLM. I find that long chats or chains just…
Indeed! If I'm met with several misunderstandings in a row, asking it to explain what I'm trying to do is a pretty surefire way to move forward.
Also mentioning what to "forget" or not focus on anymore seems to remove some noise from the responses if they are large.
Re: What we've learned from a year of building with LLMs
#113I'm not saying the content of the article is wrong, but what apps are people/companies writing articles like this actually building? I'm seriously unable to imagine any useful app. I only use GPT via API (as better Google for documentations, and its output is never usable without heavy editing). This week I tried to use "AI" in Notion: I needed to generate 84 check boxes for each day starting with specific date. I go…
I think you're going about it backwards. You don't take a tool, and then try to figure out what to do with it. You take a problem, and then figure out which tool you can use to solve it.
Re: What we've learned from a year of building with LLMs
#114Earlier quoted context omitted.
It’s significantly easier to output an integer than a JSON with a key value structure where the value is an integer and everything else is exactly as desired
That's because you've dumbed down the problem. If it was just about outputting one integer, there would be nothing to discuss. Now add a bunch more fields, add some nesting and other constraints into it...
Is there a general model that got fine tuned on these json schema/output pairs?
Seems like it would be universally useful.
Re: What we've learned from a year of building with LLMs
#115Earlier quoted context omitted.
That's the thing, it's a novel form of computing that's increasingly moving away from computer science. It deserves to be treated as a discipline of its own, with lots of words of caution and danger stickers slapped over it.
Yeah like psychology being a different field from physics even if it is running on atoms ultimately. Imagine if physics literature was filled with stuff about psychology and how that would drive physicists nuts. That's how I feel right now ;)
[deleted]
Re: What we've learned from a year of building with LLMs
#116Earlier quoted context omitted.
See if the article said this, I would have agreed - fine-tuning is a tool and it should be used thoughtfully. Although I personally believe that in this funding climate it makes sense to make data collection and model training a core capability of any AI product. However that will only be available and wise for some founders.
Agreed, model training and data collection are great! The subtle bit is just doesn't have to be for LLMs, as these are typically part of a system-of-models. E.g., we <3 RAG, and GNNs for improving your KG is fascinating. Likewise, dspy's explorations in optimizing prompts, vs LLMs, is very cool.
Have you actually used DSPy? I still can't figure out what it's useful for beyond optimizing basic few shot prompts.
Re: What we've learned from a year of building with LLMs
#117I'm not saying the content of the article is wrong, but what apps are people/companies writing articles like this actually building? I'm seriously unable to imagine any useful app. I only use GPT via API (as better Google for documentations, and its output is never usable without heavy editing). This week I tried to use "AI" in Notion: I needed to generate 84 check boxes for each day starting with specific date. I go…
I've built many production applications using a lot of these techniques and others - it's made money either by increasing sales or decreasing operational costs.
Here's a more dramatic example: https://www.grey-wing.com/
This company provides deeply integrated LLM-powered software for operating freight ships.
There are a lot of people who are doing this and achieving very good results.
Sorry, if it's not working for you, it doesn't mean that it doesn't work.
Re: What we've learned from a year of building with LLMs
#118Earlier quoted context omitted.
Yeah like psychology being a different field from physics even if it is running on atoms ultimately. Imagine if physics literature was filled with stuff about psychology and how that would drive physicists nuts. That's how I feel right now ;)
[deleted]
[deleted]
Re: What we've learned from a year of building with LLMs
#119I'm not saying the content of the article is wrong, but what apps are people/companies writing articles like this actually building? I'm seriously unable to imagine any useful app. I only use GPT via API (as better Google for documentations, and its output is never usable without heavy editing). This week I tried to use "AI" in Notion: I needed to generate 84 check boxes for each day starting with specific date. I go…
I think you're going about it backwards. You don't take a tool, and then try to figure out what to do with it. You take a problem, and then figure out which tool you can use to solve it.
But it seems to me that's what they're doing: "We have LLMs, what to do with them?" But anyway, I'm seriously just looking for an example of app that is build with stuff described in the article.
Me personally, I only used LLM for one "serious" application: I used GPT-3.5Turbo for transforming unstructured text into JSON; it was basically just ad-hoc Node.js script that called API (prompt was few examples of input-output pairs), and then it did some checks (these checks usually failed only because GPT also corrected misspellings). It would take me weeks to do it manually, but with the help of GPT it was few hours (writing of the script + I made a lot of misspellings so the script stopped a lot). But I cannot imagine anything more complex.
Re: What we've learned from a year of building with LLMs
#120I'm not saying the content of the article is wrong, but what apps are people/companies writing articles like this actually building? I'm seriously unable to imagine any useful app. I only use GPT via API (as better Google for documentations, and its output is never usable without heavy editing). This week I tried to use "AI" in Notion: I needed to generate 84 check boxes for each day starting with specific date. I go…
I think you're going about it backwards. You don't take a tool, and then try to figure out what to do with it. You take a problem, and then figure out which tool you can use to solve it.
[deleted]