Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Enough

mistral.ai

Post reply on HN
Print view View original

231–240 of 512 posts

Re: Large Enough

#231

Post by kridsdale3 »

bla3 wrote: post #128
crystal_revenge wrote: post #122

Earlier quoted context omitted.

> the only thing you can compete on is how many parameters it takes and how cheaply you can serve that to users. The problem with this strategy is that it's really tough to compete with open models in this space over the long run. If you look at OpenAI's homepage right now they're trying to promote "ChatGPT on your desktop", so it's clear even they realize that most people are looking for a local product. But once ag…

I think their desktop app still runs the actual LLM queries remotely.

This. It's a mac port of the iOS app. Using the API.

Re: Large Enough

#232

Post by throwadobe »

flir wrote: post #64
satvikpendem wrote: post #28

Earlier quoted context omitted.

Indeed, it used to output all the code I needed but now it only outputs a draft of the code with prompts telling me to fill in the rest. If I wanted to fill in the rest, I wouldn't have asked you now, would've I?

It's doing something different for me. It seems almost desperate to generate vast chunks of boilerplate code that are only tangentially related to the question. That's my perception, anyway.

This is also my perception using it daily for the last year or so. Sometimes it also responds with exactly what I provided it with and does not make any changes. It's also bad at following instructions.

GPT-4 was great until it became "lazy" and filled the code with lots of `// Draw the rest of the fucking owl` type comments. Then GPT-4o was released and it's addicted to "Here's what I'm going to do: 1. ... 2. ... 3. ..." and lots of frivolous, boilerplate output.

I wish I could go back to some version of GPT-4 that worked well but with a bigger context window. That was like the golden era...

Re: Large Enough

#233

Post by antisthenes »

soneca wrote: post #206
GenerWork wrote: post #186

Earlier quoted context omitted.

>If you insert characters to breaks the tokens down, it find the correct result: how many r's are in "s"t"r"a"w"b"e"r"r"y" ? The issue is that humans don't talk like this. I don't ask someone how many r's there are in strawberry by spelling out strawberry, I just say the word.

This is only an issue if you send commands to a LLM as you were communicating to a human.

> This is only an issue if you send commands to a LLM as you were communicating to a human.

Yes, it's an issue. We want the convenience of sending human-legible commands to LLMs and getting back human-readable responses. That's the entire value proposition lol.

Re: Large Enough

#234

Post by kridsdale3 »

machiaweliczny wrote: post #199
lawlessone wrote: post #184

Earlier quoted context omitted.

> Most of the weights in llms are 0, that's interesting. Do you have a rough percentage of this? Does this mean these connections have no influence at all on output?

My uneducated guess is that with many layers you can implement something akin to graph in brain by nulling lots of previous later outputs. I actually suspect that current models aren’t optimal with layers all of the same size but i know shit

This is quite intuitive. We know that a biological neural net is a graph data structure. And ML systems on GPUs are more like layers of bitmaps in Photoshop (it's a graphics processor). So if most of the layers are akin to transparent pixels, in order to build a graph by stacking, that's hyper memory inefficient.

Re: Large Enough

#235

Post by johntb86 »

tcgv wrote: post #201
tikkun wrote: post #39

Earlier quoted context omitted.

When using a prompt that involves thinking first, all three get it correct. "Count how many rs are in the word strawberry. First, list each letter and indicate whether it's an r and tally as you go, and then give a count at the end." Llama 405b: correct Mistral Large 2: correct Claude 3.5 Sonnet: correct

Chain-of-Thought (CoT) prompting to the rescue! We should always put some effort into prompt engineering before dismissing the potential of generative AI.

By this point, instruction tuning should include tuning the model to use chain of thought in the appropriate circumstances.

Re: Large Enough

#237

Post by throwadobe »

rvnx wrote: post #77
chipdart wrote: post #56

Earlier quoted context omitted.

> I think GPT5 will be the signal of whether or not we have hit a plateau. I think GPT5 will tell if OpenAI hit a plateau. Sam Altman has been quoted as claiming "GPT-3 had the intelligence of a toddler, GPT-4 was more similar to a smart high-schooler, and that the next generation will look to have PhD-level intelligence (in certain tasks)" Notice the high degree of upselling based on vague claims of performance, and…

PhD level-of-task-execution sounds like the LLM will debate whether the task is ethical instead of actually doing it

I wish I could frame this comment

Re: Large Enough

#238

Post by coder543 »

freediver wrote: post #126
tikkun wrote: post #76

Earlier quoted context omitted.

Correct

That would make each API call cost at least $3 ($3 is price per million input tokens). And if you have a 10 message interaction you are looking at $30+ for the interaction. Is that what you would expect?

Gemini 1.5 Pro charges $0.35/million tokens up to the first million tokens or $0.70/million tokens for prompts longer than one million tokens, and it supports a multi-million token context window.

Substantially cheaper than $3/million, but I guess Anthropic’s prices are higher.

Re: Large Enough

#239

Post by [deleted] »

satvikpendem wrote: post #28
bugglebeetle wrote: post #12

Earlier quoted context omitted.

GPT-4 was probably as good as Claude Sonnet 3.5 at its outset, but OpenAI ran it into the ground with whatever they’re doing to save on inference costs, otherwise scale, align it, or add dumb product features.

Indeed, it used to output all the code I needed but now it only outputs a draft of the code with prompts telling me to fill in the rest. If I wanted to fill in the rest, I wouldn't have asked you now, would've I?

[deleted]

Re: Large Enough

#240

Post by visarga »

satvikpendem wrote: post #28
bugglebeetle wrote: post #12

Earlier quoted context omitted.

GPT-4 was probably as good as Claude Sonnet 3.5 at its outset, but OpenAI ran it into the ground with whatever they’re doing to save on inference costs, otherwise scale, align it, or add dumb product features.

Indeed, it used to output all the code I needed but now it only outputs a draft of the code with prompts telling me to fill in the rest. If I wanted to fill in the rest, I wouldn't have asked you now, would've I?

> I wouldn't have asked you now, would've I?

That's what I said to it - "If I wanted to fill in the missing parts myself, why would I have upgraded to paid membership?"

Post reply on HN