AI hypes every twenty years.
Instead of moving goalposts, let's accept that each one succeeded at mechanizing intelligence. Including boolean algebra, writing and arithmetic.
ChatGPT turned generative AI into an “anything tool”
81–90 of 245 posts
Re: ChatGPT turned generative AI into an “anything tool”
#82Earlier quoted context omitted.
> But back in a day, AGI was just what we have today -- an universal chat bot. I don't know anybody that used AGI to mean that. I've always thought of AGI as an AI that can learn how to perform tasks with very simple instructions, rather than spending weeks training a neural network on a data center full of GPGPUs. Basically, my idea of AGI has always been what you're calling the "new" definition of AGI.
Remember that people have decades of generalized training before becoming able to perform tasks with very simple instructions. I would consider years of pre-training to be acceptable if the model is usable and trainable quickly in practice as well
It only takes like 5 minutes to teach someone to play Fluxx, not decades.
Re: ChatGPT turned generative AI into an “anything tool”
#83Although let's see if it is sustainable. I pay for ChatGPT Plus but most people aren't, and from what I've read they're losing money like crazy. We may wind up seeing this as the technical high point before the various competitors cut back on expensive hardware/energy usage for the free tier, and the quality of the responses takes a dive. This might sound a bit conspiratorial so apologies for that but back when the c…
It’s sustainable. Llama.cpp is orders of magnitude more efficient than Llama because it’s in Rust. OpenAI hasn’t even touched low level languages yet.
C++ not Rust. And most ML libraries being called from python or whatever are themselves C++ but of course there's always room for improvement, esp if you reimplement the entire corebase in C++ and specialize the inference program for your architecture. Specialization often leads to possibility of more efficient code. You are not running a generic neural net but you know the architecture beforehand so you can design its memory layout etc for efficiency yes.
Re: ChatGPT turned generative AI into an “anything tool”
#84Although let's see if it is sustainable. I pay for ChatGPT Plus but most people aren't, and from what I've read they're losing money like crazy. We may wind up seeing this as the technical high point before the various competitors cut back on expensive hardware/energy usage for the free tier, and the quality of the responses takes a dive. This might sound a bit conspiratorial so apologies for that but back when the c…
> the responses to the various voice assistance was almost eerie in that it could infer what you wanted/needed. Then as things cooled off, they got gradually dumber/worse every year since then. They're legitimately bad today (Siri in particular It's a compelling argument but I find that difficult to believe given the timespan. Siri launched in 2011. The FLOP/s per dollar rates for GPUs involved in ML doubled every 2…
I don’t think Siri et al got worse, we just had time for the hype of the excellent voice to text and back again to wear off, and realized that they were too limited at figuring out actions to be very useful.
Re: ChatGPT turned generative AI into an “anything tool”
#85Earlier quoted context omitted.
I'm actually worried of the inevitable enshitification of Open AI. Right now, GPT 4 is great and I use it every day. I rely less and less on Google and save a ton of time. Google has gone downhill hard for a few years, thanks to all these silly blog engines like medium, substack and the like. I'm really worried and would like to 'freeze' GPT 4 as is, at least with the data up to Sep 2021, and for them to not change o…
That's why I think open source LLMs are the ones that will actually persist throughout the commercial enshittification and reign supreme at the end.
I'm more interested in Open Source/Local LLM's ability to consume private data as an internal knowledgeable. For example, feed in your entire source repo(s) and then ask it questions about where something is or how it works.
Re: ChatGPT turned generative AI into an “anything tool”
#86Although let's see if it is sustainable. I pay for ChatGPT Plus but most people aren't, and from what I've read they're losing money like crazy. We may wind up seeing this as the technical high point before the various competitors cut back on expensive hardware/energy usage for the free tier, and the quality of the responses takes a dive. This might sound a bit conspiratorial so apologies for that but back when the c…
> This might sound a bit conspiratorial so apologies for that but back when the competition for Google Assistant/Siri/Cortana/Alexa was super hot, the responses to the various voice assistance was almost eerie in that it could infer what you wanted/needed.
Siri at least was always a simple intent based system that was never very intelligent and very much a brute voice method where someone went in and manually configured patterns for it to recognize.
You can tell by how the API for Siri works for third party integrations.
On a much smaller scale, I’ve done something similar with Amazon Lex (the AWS version of Alexa).
I am not saying the underlying architecture of Siri or Alexa is simple. But how you build on top of it to get it to do things is just giving it patterns for it to recognize and various permutations.
The hardest part is the speech to text once you get everything in text form, the complexity is not that great.
Re: ChatGPT turned generative AI into an “anything tool”
#87Earlier quoted context omitted.
> I'd like to have LawGPT I tried ChatGPT 4 for a specific TX county's laws. This was after I had spent hours doing traditional search. ChatGPT got everything correct, and even gave me some more direction.
> ChatGPT got everything correct My anecdote with technical information is exactly the opposite, unfortunately. I routinely use it to help guide my research - generally about topics I know something about but want to know more. It often will provide some response that is entirely wrong but looks very good. When called out, it apologizes, then proceeds to produce more information that's also not entirely correct. Ther…
I'm having the same issues with it getting things confidently incorrect.
The flow I've been going through:
* Do this task with this constraint > GPT4 Does task
* Analyse all of the items and whether they adhere to the constraint > GPT4 "Apologies, I got some things wrong, here's what I did incorrect"
* Fix the things you did wrong, remember the constraint is of upmost importance > GPT4 "I have completed the task and fixed the constraint
* Analyse all of the items and whether they adhere to the constraint > GPT4 "Apologies, I got some things wrong, here's what I did incorrect"
...
At least it can analyse the results, but even then if you don't ask in the right way it will blatantly lie to you and tell you it's all correct when it's not.
Re: ChatGPT turned generative AI into an “anything tool”
#88Earlier quoted context omitted.
Games are solved since AlphaZero. Just set up an adversarial model, and voila. Any game, any conditions. Proactive models are simple — just generate the “internal monologue” periodically from the system prompt, mix it with the compressed state, then feed it to another model (the outputs are used as state updates). I think this is how human agency works as well. Movement is a different story (LLMs do not work here, ex…
AlphaZero is amazing, but it only plays 2D games with well defined win conditions.
[deleted]
Re: ChatGPT turned generative AI into an “anything tool”
#89Although let's see if it is sustainable. I pay for ChatGPT Plus but most people aren't, and from what I've read they're losing money like crazy. We may wind up seeing this as the technical high point before the various competitors cut back on expensive hardware/energy usage for the free tier, and the quality of the responses takes a dive. This might sound a bit conspiratorial so apologies for that but back when the c…
There are several breakthroughs actually, every few weeks, inching closer and closer to the next generation of hardware for ML models with optoelectronic compute.
This is just one of the most recent:
https://news.mit.edu/2023/system-could-yield-more-powerful-e...
Pretty much every single major tech university has their research/VC arm with a hand in photonics for AI.
The initial rollout is around interconnect, but compute is moving along even faster than I'd expected, with significant steps forward occurring frequently.
I'm skeptical we'll be seeing actual quantum computing any time soon for general purpose stuff due to error correction, but for the specific needs of training and running a neural network, it's pretty much the perfect marriage of hardware limitations with task specifications.
So yes, this software trend is going to end up being much cheaper and efficient. By orders of magnitude leaps, not just at the creeping pace of Moore's law.
Re: ChatGPT turned generative AI into an “anything tool”
#90Although let's see if it is sustainable. I pay for ChatGPT Plus but most people aren't, and from what I've read they're losing money like crazy. We may wind up seeing this as the technical high point before the various competitors cut back on expensive hardware/energy usage for the free tier, and the quality of the responses takes a dive. This might sound a bit conspiratorial so apologies for that but back when the c…
isn't ChatGPT just an investor demo anyway? I doubt they run out of cash at least until they fund building GPT 5. If it flops then who knows.
No, that was GPT, GPT-2, and GPT-3.