Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

381–390 of 512 posts

Re: Large Enough

#381

Post by stavros »

freediver wrote: post #22
TIPSIO wrote: post #11

Earlier quoted context omitted.

Have you (or anyone) swapped on Cursor with Anthropic API Key? For coding assistant, it's on my to do list to try. Cursor needs some serious work on model selection clarity though so I keep putting off.

I did it (fairly simple really) but found most of my (unsophisticated) coding these days to go through Aider [1] paired with Sonnet, for UX reasons mostly. It is easier to just prompt over the entire codebase, vs Cursor way of working with text selections. [1] https://aider.chat

Aider with Sonnet is so much better than with GPT. I made a mobile app over the weekend (never having touched mobile development before), and with GPT it was a slog, as it kept making mistakes. Sonnet was much, much better.

Re: Large Enough

#382

Post by groby_b »

SirMaster wrote: post #369
stavros wrote: post #348

Earlier quoted context omitted.

> How is a layman supposed to even know that it's testing on that? They're not, but laymen shouldn't think that the LLM tests they come up with have much value.

I'm saying a layman or say a child wouldn't even think this is a "test". They are just asking a language model a seemingly simple language related question from their point of view.

layman or children shouldn't use LLMs.

They're pointless unless you have the expertise to check the output. Just because you can type text in a box doesn't mean it's a tool for everybody.

Re: Large Enough

#383

Post by groby_b »

probably_wrong wrote: post #263
generalizations wrote: post #221

Earlier quoted context omitted.

Testing models on their tokenization has always struck me as kinda odd. Like, that has nothing to do with their intelligence.

I would counterargue with "that's the model's problem, not mine". Here's a thought experiment: if I gave you 5 boxes and told you "how many balls are there in all of this boxes?" and you answered "I don't know because they are inside boxes", that's a fail. A truly intelligent individual would open them and look inside. A truly intelligent model would (say) retokenize the word into its individual letters (which I'm op…

LLMs are not truly intelligent.

Never have been, never will be. They model language, not intelligence.

Re: Large Enough

#384

Post by ketzo »

ipsod wrote: post #284
ketzo wrote: post #79

Earlier quoted context omitted.

Uploading large codebases is particularly useful.

Is it? I've found that I get better results if I cherry pick code to feed to Claude 3.5, instead of pasting whole files. I'm kind of isolated, though, so maybe I just don't know the trick.

I've been using Cody from Sourcegraph, and it'll write some really great code; business logic, not just tests/simple UI. It does a great job using patterns/models from elsewhere in your codebase.

Part of how it does that is through ingesting your codebase into its context window, and so I imagine that bigger/better context will only improve it. That's a bit of an assumption though.

Re: Large Enough

#385

Post by RyanAdamas »

logicchains wrote: post #343
RyanAdamas wrote: post #315

Earlier quoted context omitted.

Multiple input and output processes in different languages has zero effect on associative learning and creative formulation in my estimations. We've already done studies that show there is no correlation between human intelligence and knowing multiple languages, after having to put up with decades of "Americans le dumb because..." and this is no different. The amount of discourse on a single topic has a limited degre…

Google mentioned this in one of their papers, they found for large enough models including more languages did indeed lead to an overall increase in performance.

Considering Googles progress and censorship history, I'm inclined to take their assessments with a grain of salt.

Re: Large Enough

#386

Post by lolinder »

impossiblefork wrote: post #341
lolinder wrote: post #129

Earlier quoted context omitted.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs Yes. This is exactly why I'm skeptical of AI doomerism/saviorism. Too many people have been looking at the pace of LLM development over the last two (2) years, modeled it as an exponential growth function, and come to the conclusion that AGI is inevitable in the next ${1-5} years and we're headed for ${(dys|u)topia}. B…

Notice though, that all these improvements have been with pretty basic transformer models that output all their tokens-- no internal thoughts, no search, no architecture improvements and things are only fed through them once. But we could add internal thoughts-- we could make the model generate tokens that aren't part of its output but are there for it to better figure out its next token. This was tried QuietSTAR. Ho…

> Also, presumably GPT-5 isn't far away.

Why do we presume that? People were saying this right before 4o and then what came out was not 5 but instead a major improvement on cost for 4.

Is there any specific reason to believe OpenAI has a model coming soon that will be a major step up in capabilities?

Re: Large Enough

#387

Post by jpadkins »

lolinder wrote: post #129
Liquix wrote: post #10

These companies full of brilliant engineers are throwing millions of dollars in training costs to produce SOTA models that are... "on par with GPT-4o and Claude Opus"? And then the next 2.23% bump will cost another XX million? It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs; that an ARC prize level breakthrough is needed to move the needle any farther at this point.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs Yes. This is exactly why I'm skeptical of AI doomerism/saviorism. Too many people have been looking at the pace of LLM development over the last two (2) years, modeled it as an exponential growth function, and come to the conclusion that AGI is inevitable in the next ${1-5} years and we're headed for ${(dys|u)topia}. B…

> we're already seeing that OpenAI has pivoted from improving on GPT-4's benchmarks to reducing cost, while competitors (including free ones) catch up.

What if they have two teams? One dedicated to optimizing (cost, speed, etc) the current model and a different team working on the next frontier model? I don't think we know the growth curve until we see gpt5.

Re: Large Enough

#388

Post by SirMaster »

groby_b wrote: post #382
SirMaster wrote: post #369

Earlier quoted context omitted.

I'm saying a layman or say a child wouldn't even think this is a "test". They are just asking a language model a seemingly simple language related question from their point of view.

layman or children shouldn't use LLMs. They're pointless unless you have the expertise to check the output. Just because you can type text in a box doesn't mean it's a tool for everybody.

Well they certainly aren't being marketed or used that way...

I'm seeing everyone and their parents using chatgpt.

Re: Large Enough

#389

Post by robbiep »

dTal wrote: post #371
jahsome wrote: post #301

Earlier quoted context omitted.

Is anyone in the know, aside from mainstream media (god forgive me for using this term unironically) and civillians on social media claiming LLMs are anything but word calculators? I think that's a perfect description by the way, I'm going to steal it.

I think it's a very poor intuition pump. These 'word calculators' have lots of capabilities not suggested by that term, such as a theory of mind and an understanding of social norms. If they are a "merely" a "word calculator", then a "word calculator" is a very odd and counterintuitively powerful algorithm that captures big chunks of genuine cognition.

They’re trained on the available corpus of human knowledge and writings. I would think that the word calculators have failed if they were unable to predict the next word or sentiment given the trillions of pieces of data they’ve been fed. Their training environment is literally people talking to each other and social norms. Doesn’t make them anything more than p-zombies though.

As an aside, I wish we would call all of this stuff pseudo intelligence rather than artificial intelligence

Re: Large Enough

#390

Post by michaelmrose »

VincentEvans wrote: post #366
microtonal wrote: post #342

Earlier quoted context omitted.

How the tokenizer works explains why a model can’t answer the question, what the name of the cat is doesn’t explain anything. This is Hacker News, we are usually interested in how things work.

Indeed, I appreciate the explanation, it is certainly both interesting and informative to me, but to somewhat echo the person you are replying to - if I wanted a boat, and you offer me a boat, and it doesn’t float - the reasons for failure are perhaps full of interesting details, but perhaps the most important thing to focus on first - is to make the boat float, or stop offering it to people who are in need of a boat…

The test problem is emblematic of a type of synthetic query that could fail but of limited import in actual usage.

For instance you could ask it for a JavaScript function to count any letter in any word and pass it r and strawberry and it would be far more useful.

Having edge cases doesn't mean its not useful it is neither a free assastant nor a coder who doesn't expect a paycheck. At this stage it's a tool that you can build on.

To engage with the analogy. A propeller is very useful but it doesn't replace the boat or the Captain.

Post reply on HN