51–60 of 77 posts
>Later, I’ll cover “unhobbling,” which you can think of as “paradigm-expanding/application-expanding” algorithmic progress that unlocks capabilities of base models.
I think this is probably on the mark. The LMMs are deep memory coupled to weak reasoning and without the recursive self-control and self evaluation of many threads of attention.
Earlier quoted context omitted.
I watched a secretary take business cards and 1 by 1 copy them into our client DB last week. All day. I showed her how to use the ChatGPT app, photo of them all, and convert it into an excel. "You just saved me weeks of work this year" Should be some fun shorting (we have over 300 secretaries alone, our support staff is massive).
Business card scanners have been around since the earliest versions of the iPhone, but I guess thank you ChatGPT for discovering OCR
As a software engineer I'm very familiar with "OOM"s and "orders of magnitude", and have never once heard the former used to mean the latter. Perhaps this is a term of art in harder science or maths. I can't help but think here it's likely to confuse the majority as they wonder why the author is conflating memory and compute. Something that might help is for the link to be amended to link to the page as a whole (and…
As a physician I have the same expectations as you for those two words. Especially given that the link is to an anchor on the middle of the page, I was thinking in units of "OOM". Essentially I was thinking "OK an OOM is a unit for the doubling of RAM" (wrong).
I’m very skeptical of any future prediction whose main evidence is an extrapolation of existing trendlines. Moore’s Law - frequently referenced in the original article - provides a cautionary tale for such thinking. Plenty of folks in the 90’s relied on a shallow understanding of integrated circuits and computers more generally to extrapolate extraordinary claims of exponential growth in computing power which obvious…
Earlier quoted context omitted.
Business card scanners have been around since the earliest versions of the iPhone, but I guess thank you ChatGPT for discovering OCR
Do you have a link to the one that I can put 50 of them on the floor and it will send me back an excel file? I'd like to test it out compared to ChatGPT as I'm going to be implementing "AI" across the whole 700+ person business.
Also from a month ago: https://news.ycombinator.com/item?id=40584237
There was also related discussion about another longform piece by the same author that I'm too lazy to look up at the moment.. In my opinion, this author has drunken the kool-aid and then some. There is simply no evidence that more scaling of LLMs will lead to AGI, and on the contrary there is plenty of evidence that the current "gaps" that LLMs have are innate and unsolvable with just more scaling.
> By the end of this, I expect us to get something that looks a lot like a drop-in remote worker. An agent that joins your company, is onboarded like a new human hire, messages you and colleagues on Slack and uses your softwares, makes .. I work at a company with ~50k employees each of whom has different data access rules governed by regulation. So either (a) you train thousands of models which is cost-prohibitive or…
Are separately trained models necessary for your case? As context windows get longer—Gemini 1.5 Pro now accepts up to two million tokens, and Google has talked of the goal of "infinite" context windows—couldn't a single base model be used with individualized contexts of sensitive data?
My question is whether this capability even exists.
And if it does how robust it is to workarounds.
It’s hard to make LLMs ignore what they were trained to generate. It’s easy for humans. Isn’t that an obstacle on the path to AGI? I was doing trivial tests that demand LLMs to swim against their probability distributions at inference time, and they don’t like this.
Earlier quoted context omitted.
What is an abstract reasoning task that your average 15 year old (who has "general intelligence") can do that you think LLMs can't do?
A 15 year old can reason about how to move their body through a complex obstacle course. They could reason about the nonverbal social cues in a complex interpersonal situation between multiple people, estimate the mood of each person even if there are very few words being exchanged, and determine how different possible actions would affect the situation. They could learn with brief instruction how to control their mu…
AGI is not a continuum from LLMs; true intelligence is characterized by comprehension, reasoning, and self-awareness, transcending mere data patterns.
Nonsense. You can't even define some of those words or know how to measure or identify them in humans. Well, foreign language learners do "reading comprehension tests" but an LLM can already ace that and it's not really the same meaning of the word. For reasoning you can write out the logic of your reasons, so there's that. But that's absolutely not required for AGI. People can already go a long way (often further th…