381–390 of 1001 posts
Earlier quoted context omitted.
AGI is the Sisyphean task of our age. We’ll push this boulder up the mountain because we have to, even if it kills us.
What has AGI got to do with this?
Earlier quoted context omitted.
No, I'm complaining that just because GPT-4 is called GPT-4 doesn't mean it's the fourth LLM from OpenAI. Off the top of my head: GPT-2, Codex, GPT-3 in three different flavors (babbage, curie, davinci), GPT-3.5. Suggesting that GPT-4 was "fourth" simply isn't credible. Just the other day they announced a jump from o1 to o3, skipping o2 purely because it's already the name of a major telecommunications brand in Europ…
It’s somehow funny to hear a British company being described as ‘in Europe’, but I suppose you’re technically correct…
Earlier quoted context omitted.
Have you ever heard of a local maxima? You don't get an attack helicopter by breeding stronger and stronger falcons.
For an industry that spun off of a research field that basically revolves around recursive descent in one form or another, there's a pretty silly amount of willful ignorance about the basic principles of how learning and progress happens. The default assumption should be that this is a local maximum, with evidence required to demonstrate that it's not. But the hype artists want us all to take the inevitability of LLM…
There might be many local maxima that cross the useful AI or even AGI threshold.
Earlier quoted context omitted.
> What do you think the last few years have been all about? Next token language-based predictors with no more intelligence than brute force GIGO which parrot existing human intelligence captured as text/audio and fed in the form of input data. 4o agrees: "What you are describing is a language model or next-token predictor that operates solely as a computational system without inherent intelligence or understanding. T…
Everything you said is parroting data you’ve trained on, two thirds of it is actual copy paste
I still don't buy the "we do the same as LLMs" discourse. Of course one could hypothesize the human brain language center may have some similarities to LLMs, but the differences in resource usage and how those resources are used to train humans and LLMs are remarkable and may indicate otherwise.
Earlier quoted context omitted.
Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.
AGI will arrive like self driving cars. it’s not that you will wake up one day and we have it. cars gained auto-braking, parallel parking, cruise control assist. and over a long time you get to something like waymo, which still is location dependent. i think AGI will take decades but sooner will be some special cases that are effectively the same
AGI is the holy grail of technology. A technology so advanced that not only does it subsume all other technology, but it is able to improve itself.
Truly general intelligence like that will either exist or not. And the instant it becomes public, the world will have changed overnight (maybe the span of a year)
Note: I don’t think statistical models like these will get us there.
Earlier quoted context omitted.
Just today I got Claude to convert a company’s PDF protocol specification into an actual working python implementation of that protocol. It would have been uncreative drudge work for a human, but I would have absolutely paid a week of junior dev time for it. Instead I wrote it alongside AI and it took me barely more than an hour. The best part is, I’ve never written any (substantial) python code before.
It would seem you don't care too much about verifying its output or about its correctness. If you did, it wouldn't take you just an hour. I guess you'll let correctness be someone else's problem.
Earlier quoted context omitted.
AGI will arrive like self driving cars. it’s not that you will wake up one day and we have it. cars gained auto-braking, parallel parking, cruise control assist. and over a long time you get to something like waymo, which still is location dependent. i think AGI will take decades but sooner will be some special cases that are effectively the same
I don’t think that’s true for AGI. AGI is the holy grail of technology. A technology so advanced that not only does it subsume all other technology, but it is able to improve itself. Truly general intelligence like that will either exist or not. And the instant it becomes public, the world will have changed overnight (maybe the span of a year) Note: I don’t think statistical models like these will get us there.
There may well be an upper limit on cognition (we are not really sure what cognition is - even as we do it) and it may be that human minds are close to it.
Earlier quoted context omitted.
Tokens don't need to be text either, you can move to higher level "take_action" semantics where "stream back 1 character to session#117" as every single function call. Training cheap models that can do things in the real world is going to change a huge amount of present capabilities over the next 10 years
can you share learning resources on this topic
Earlier quoted context omitted.
I don’t think that’s true for AGI. AGI is the holy grail of technology. A technology so advanced that not only does it subsume all other technology, but it is able to improve itself. Truly general intelligence like that will either exist or not. And the instant it becomes public, the world will have changed overnight (maybe the span of a year) Note: I don’t think statistical models like these will get us there.
If that is what AGI looks like. There may well be an upper limit on cognition (we are not really sure what cognition is - even as we do it) and it may be that human minds are close to it.
Earlier quoted context omitted.
I work on very complex problems. Some of my solutions have small, standard substeps that now I can reliably outsource to ChatGPT. Here are a few just from last week: - write cvxpy code to find the chromatic number of a graph, and an optimal coloring, given its adjecency matrix. - given an adjecency matrix write numpy code that enumerates all triangle-free vertex subsets. - please port this old code from tensorflow to…
To be honest, these don’t sound like hard problems. These sound like they have very specific answers that I might find in the more specialized stackoverflow sections. These are also the kind of questions (not in this domain) that I’ve found yield the best results from LLMs. In comparison asking an LLM a more project specific question “this code has a race condition where is it” while including some code usually is a…