Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

241–250 of 512 posts

Re: Large Enough

#241

Post by cloverich »

flir wrote: post #64
satvikpendem wrote: post #28

Earlier quoted context omitted.

Indeed, it used to output all the code I needed but now it only outputs a draft of the code with prompts telling me to fill in the rest. If I wanted to fill in the rest, I wouldn't have asked you now, would've I?

It's doing something different for me. It seems almost desperate to generate vast chunks of boilerplate code that are only tangentially related to the question. That's my perception, anyway.

This is also my experience. Previously it got good at giving me only relevant code which, as an experienced coder, is what i want. my favorites were the one line responses.

Now it often falls back to generating full examples, explanations, restating the question and its approach. I suspect this is by design as (presumably) less experienced folks want or need all that. For me, i wish i could consistently turn it into one of those way too terse devs that replies with the bare minimum example, and expects you to infer the rest. Usually that is all i want or need, and i can ask for elaboration when not the case. I havent found the best prompts to retrigger this persona from it yet.

Re: Large Enough

#242

Post by ToValueFunfetti »

cjalmeida wrote: post #139
lolinder wrote: post #129

Earlier quoted context omitted.

> It seems increasingly apparent that we are reaching the limits of throwing more data at more GPUs Yes. This is exactly why I'm skeptical of AI doomerism/saviorism. Too many people have been looking at the pace of LLM development over the last two (2) years, modeled it as an exponential growth function, and come to the conclusion that AGI is inevitable in the next ${1-5} years and we're headed for ${(dys|u)topia}. B…

Indeed.All exponential growth curves are sigmoids in disguise.

This is something that is definitionally true in a finite universe, but doesn't carry a lot of useful predictive value in practice unless you can identify when the flattening will occur.

If you have a machine that converts mass into energy and then uses that energy to increase the rate at which it operates, you could rightfully say that it will level off well before consuming all of the mass in the universe. You just can't say that next week after it has consumed all of the mass of Earth.

Re: Large Enough

#243

Post by Zambyte »

recursive wrote: post #111
Tepix wrote: post #54

Earlier quoted context omitted.

LLMs think in tokens, not letters. It's like asking someone who is dyslexic about spelling. Not their strong suit. In practice, it doesn't matter much, does it?

Sometimes it does, sometimes it doesn't. It is evidence that LLMs aren't appropriate for everything, and that there could exist something that works better for some tasks.

Language models are best treated like consciousness. Our consciousness does a lot less than people like to attribute to it. It is mostly a function of introspection and making connections, rather than being the part of the brain where higher level reasoning and the functions of the brain that tell your body how to stay alive (like beating your heart).

By allowing a language model to do function calling, you are essentially allowing it to do specialized "subconscious" thought. The language model becomes a natural language interface to the capabilities of its "subconsciousness".

A specific human analogy could be: I tell you to pick up a pen off of the table, and then you do it. Most of your mental activity would be subconscious, orienting your arm and hand properly to pick up the pen, actually grabbing the pen, and picking it up. The linguistic representation of the action would exist in your concious mind (pick up the pen), but not much else.

A language model could very easily call out to a text processing function to correctly do things like count the number of r's in the word strawberry. That is a job that your concious mind can dispatch to your subconciousness.

Re: Large Enough

#244

Post by nabakin »

wesleyyue wrote: post #203
nabakin wrote: post #194

Earlier quoted context omitted.

Are you sure the chat history is being passed when the second message is sent? That looks like the kind of response you'd expect if it only received the prompt "in python" with no chat history at all.

Yes, I built the extension. I actually also just went to send another message asking what the first msg was just to double check I didn't have a bug and it does know what the first msg was.

Thanks, that's some really bad accuracy/performance

Re: Large Enough

#246

Post by troupo »

zooq_ai wrote: post #144
troupo wrote: post #118

Earlier quoted context omitted.

> however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives Has there been any indication that we're improving the lives of millions of people?

Yes, just like internet, power users have found use cases. It'll take education / habit for general users

Ah yes. We're in the crypto stages of "it's like the internet".

Re: Large Enough

#247

Post by troupo »

machiaweliczny wrote: post #195
troupo wrote: post #118

Earlier quoted context omitted.

> however this progress is still worth it if we can deploy it to improve millions and eventually billions of people's lives Has there been any indication that we're improving the lives of millions of people?

Just me coding 30% faster is worth it

I haven't found a single coding problem where any of these coding assistants where anything but annoying.

If I need to babysit a junior developer fresh out of school and review every single line of code it spits out, I can find them elsewhere

Re: Large Enough

#248

Post by wruza »

generalizations wrote: post #221
drewnick wrote: post #27

Earlier quoted context omitted.

All 3 models you ranked cannot get "how many r's are in strawberry?" correct. They all claim 2 r's unless you press them. With all the training data I'm surprised none of them fixed this yet.

Testing models on their tokenization has always struck me as kinda odd. Like, that has nothing to do with their intelligence.

It doesn’t test “on tokenization” though. What happens when an answer is generated is few abstraction levels deeper than tokens. A “thinking” “slice” of an llm is completely unaware of tokens as an immediate part of its reasoning. The question just shows lack of systemic knowledge about strawberry as a word (which isn’t surprising, tbh).

Re: Large Enough

#249

Post by vorticalbox »

drewnick wrote: post #27
tikkun wrote: post #5

Links to chat with models that released this week: Large 2 - https://chat.mistral.ai/chat Llama 3.1 405b - https://www.llama2.ai/ I just tested Mistral Large 2 and Llama 3.1 405b on 5 prompts from my Claude history. I'd rank as: 1. Sonnet 3.5 2. Large 2 and Llama 405b (similar, no clear winner between the two) If you're using Claude, stick with it. My Claude wishlist: 1. Smarter (yes, it's the most intelligent, and y…

All 3 models you ranked cannot get "how many r's are in strawberry?" correct. They all claim 2 r's unless you press them. With all the training data I'm surprised none of them fixed this yet.

I just tried llama 3.1 8 b this is its reply.

According to multiple sources, including linguistic analysis and word breakdowns, there are 3 Rs in the word "strawberry".

Re: Large Enough

#250

Post by itishappy »

rahimnathwani wrote: post #226
est31 wrote: post #220

Earlier quoted context omitted.

Humans also constantly make mistakes that are due to proximity in their internal representation. "Could of"/"Should of" comes to mind: the letters "of" have a large edit distance from "'ve", but their pronunciation is very similar. Especially native speakers are prone to the mistake as they grew up learning english as illiterate children, from sounds only, compared to how most people learning english as second langua…

Until I was ~12, I thought 'a lot' was a single word.

https://hyperboleandahalf.blogspot.com/2010/04/alot-is-bette...
Post reply on HN