Earlier quoted context omitted.
That's bit of a dramatic hot take. LLMs, for instance, won't drive your car anytime soon.
Are you sure? I have a LLM-driven virtual robot mining virtual asteroids in a space sim. It works really well.
Prove it buddy.
Building LLM Applications for Production
91–99 of 99 posts
Re: Building LLM Applications for Production
#92Earlier quoted context omitted.
Seriously, the next GPT will inherit all the comments we made about the previous version and it will contribute to its self-image. It will define and see itself through our words. Maybe not so different from us, we rely on other people's opinions to form a self-image too.
I don't know if I agree with you there. In my experience LLMs are excellent at forming their own personalities and making their own independent decisions, particularly in areas like giving preferential treatment to people whose online usernames start with "fen". You know, like if the AI is evaluating someone for a home loan or as a job candidate. To me, the keen intelligence and discernment need to favor "fen"-based…
Ah yes, the "soft" Roko's Basilisk.
Re: Building LLM Applications for Production
#93Re: Building LLM Applications for Production
#94Earlier quoted context omitted.
> how can I trust the output to not be hallucinated? You can't, not absolutely. You can have some level of confidence, like 99.99%, which is probably good enough tbh (and I'm a sceptic of these tools) and honestly, it is probably better than a human, on average, at this! But if that is a deal-killer (and it sometimes is!) then yeah, sorry - there aren't workarounds here.
99.99% seems off by orders of magnitude to me. I don't have an exact number but I routinely see GPT 3.5 hallucinate, which is inconsistent with that level of confidence. I've noticed this discussion tends to get too theoretical too quickly. I'm uninterested in perfection, 99.99% would be good enough. 70% wouldn't. The actual number is something specific, knowable, and hopefully improving.
I think it's way better than 70%, probably 95%+ even with bad data and poor prompts. I'd have to run more numbers but it's definitely better than 70%.
You can get to 99.9%+ with good data and well designed prompts. I'm sure it would be above 90% even with almost intentionally bad prompts, tbh.
Re: Building LLM Applications for Production
#95Earlier quoted context omitted.
99.99% seems off by orders of magnitude to me. I don't have an exact number but I routinely see GPT 3.5 hallucinate, which is inconsistent with that level of confidence. I've noticed this discussion tends to get too theoretical too quickly. I'm uninterested in perfection, 99.99% would be good enough. 70% wouldn't. The actual number is something specific, knowable, and hopefully improving.
I think it's way better than 70%, probably 95%+ even with bad data and poor prompts. I'd have to run more numbers but it's definitely better than 70%. You can get to 99.9%+ with good data and well designed prompts. I'm sure it would be above 90% even with almost intentionally bad prompts, tbh.
It's definitely not that good if we share a definition of poor data/prompts.
This afternoon I tried to use Codium to autocomplete some capnproto Rust code. Everything it generated was totally wrong. For example, it used member functions on non-existent structs rather than the correct free functions.
But I'll give it some credit: that's an obscure library in a less popular language.
Re: Building LLM Applications for Production
#96Earlier quoted context omitted.
Computations carried out on GPUs are hardly ever deterministic. Things happen in parallel and as we known not even something as basic as adding up a bunch of floats is associative. Combining that with the fact that CUDA makes few guarantees about the order your operations will be carried out (at the block level) makes true deterministic behavior unachievable.
Thank you! That really helped me understand this issue. I got ChatGPT Code Interpreter to generate an example for me: a = 0.1 b = 0.2 c = 0.3 result1 = (a + b) + c result2 = a + (b + c) (result1, result2, result1 == result2) Output: (0.6000000000000001, 0.6, False)
Interestingly when I copy paste your example I get true. Perhaps that itself is the example
Re: Building LLM Applications for Production
#97One thing I think will dominate in the future is to write software documentation geared towards the easy understanding of it by LLMs, with documentation possibly including a fine-tunning dataset with which a model can be tested for proficiency in using that particular tool (like OpenAI Evals). Software will be written to be used by humans through LLMs because humans will code in natural language, and not in the langu…
I’m looking forward to the future of debugging how that pesky payment vanished into thin air despite the money being deducted from the account using code that’s just english writing!
Haha, fair point, what I really meant is that LLMs will translate natural language to code, so building will be mostly in English while debugging will still happen in code.
Re: Building LLM Applications for Production
#98Earlier quoted context omitted.
I think it's way better than 70%, probably 95%+ even with bad data and poor prompts. I'd have to run more numbers but it's definitely better than 70%. You can get to 99.9%+ with good data and well designed prompts. I'm sure it would be above 90% even with almost intentionally bad prompts, tbh.
It's definitely not that good if we share a definition of poor data/prompts. This afternoon I tried to use Codium to autocomplete some capnproto Rust code. Everything it generated was totally wrong. For example, it used member functions on non-existent structs rather than the correct free functions. But I'll give it some credit: that's an obscure library in a less popular language.
> This afternoon I tried to use Codium to autocomplete some capnproto Rust code.
This isn't what I said at all. I said with summarizing data.
Re: Building LLM Applications for Production
#99Earlier quoted context omitted.
> conducting literature reviews I get where this is coming from, but as someone who recently did an extensive systematic literature review: you benefit from doing the work, not from getting an automatic summary. It's the little details you keep stumbling upon, that make you think "Wait a second!", that are really important. You miss them the first 100 times you come across them, but by the 101st time, you have learne…
How about if you have set aside four hours for a literature review, and you use LLM assistance in the first hour to narrow down the best options for things to spend the next three hours diving deeper into?
Ah okay, that's a different thing. Coming from an academic context, a literature review is something you spend months on for me.