Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on the New Deepseek v3

composio.dev

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Notes on the New Deepseek v3

#11

Post by antirez »

knicholes wrote: post #7
antirez wrote: post #2

I'm testing it for system programming brainstorming, code reviews and Python test units writing, and my impression is that it's a Sonnet 3.5 level model for most tasks. I said a few things here: https://www.youtube.com/watch?v=xjCqi9JK440 but in general this is really an open weights frontier model, the first one that we get (IMHO llama 3.1 405B does not fit the definition, and the actual model quality is far from th…

Do you mind sharing your workflow with how you use this model? Is it integrated with an IDE, are you using the command line, maybe a UI with each question then copy/pasting the results?

Sure! I'll do several videos in the future with my workflow. No ide, I just use the web interface.

Re: Notes on the New Deepseek v3

#13

Post by RockyMcNuts »

maeil wrote: post #4

A lot of talk about how much cheaper it is than all other models. It remains to be seen what the pricing will be when run by non-Deepseek providers. They might be loss leading. The comparison for cheap models should also be Gemini 2.0 Flash Exp. I could see it being even cheaper when it stops being free - if it does at all. There's definitely a scenario where Google just keeps it freeish for a long time with relative…

exactly ... Gemini 2.0 Flash ranks better on quality, is faster, and cheaper if you assume same pricing as 1.5 (might go down).

these models are being commoditized.

https://artificialanalysis.ai/models/deepseek-v3

Re: Notes on the New Deepseek v3

#14

Post by bangaladore »

maeil wrote: post #4

A lot of talk about how much cheaper it is than all other models. It remains to be seen what the pricing will be when run by non-Deepseek providers. They might be loss leading. The comparison for cheap models should also be Gemini 2.0 Flash Exp. I could see it being even cheaper when it stops being free - if it does at all. There's definitely a scenario where Google just keeps it freeish for a long time with relative…

Per available providers on OpenRouter right now:

DeepSeek - 0.14$ per million tokens input, 0.28$ million tokens output (66 tokens per/s)

Fireworks - 0.9$ per million tokens input, 0.9$ million tokens output (23 tokens per/s)

DeepInfra - 1$ per million tokens input, 2$ million tokens output (1.27 tokens per/s)

Compared to Llama 3.1 405B (smaller model than this afaik):

Cheapest is 0.8/0.8$ at 24 t/s all the way to 4$/4$ at 8 t/s

So third party cost seems similar, but there aren't many people hosting DeepSeek right now.

Re: Notes on the New Deepseek v3

#16

Post by maccam912 »

egnehots wrote: post #8

If you understand how LLMs work, you should disregard tests such as: - How many 'r's are in Strawberry? - Finding the fourth word of the response These tests are at odds with the tokenizer and next-word prediction model. They do not accurately represent an LLM's capabilities. It's akin to asking a blind person to identify colors.

I'm not going to argue these are good tests, if you asked a coworker these questions they'd look at you weird, but what surprised me is how well you can encode a sentence never written down before, put it through base64 encoding, and then ask an llm to decode it. And the good models can do this surprisingly well.

Re: Notes on the New Deepseek v3

#17

Post by Arcuru »

bangaladore wrote: post #14
maeil wrote: post #4

A lot of talk about how much cheaper it is than all other models. It remains to be seen what the pricing will be when run by non-Deepseek providers. They might be loss leading. The comparison for cheap models should also be Gemini 2.0 Flash Exp. I could see it being even cheaper when it stops being free - if it does at all. There's definitely a scenario where Google just keeps it freeish for a long time with relative…

Per available providers on OpenRouter right now: DeepSeek - 0.14$ per million tokens input, 0.28$ million tokens output (66 tokens per/s) Fireworks - 0.9$ per million tokens input, 0.9$ million tokens output (23 tokens per/s) DeepInfra - 1$ per million tokens input, 2$ million tokens output (1.27 tokens per/s) Compared to Llama 3.1 405B (smaller model than this afaik): Cheapest is 0.8/0.8$ at 24 t/s all the way to 4$…

Just a minor clarification, DeepSeek's pricing for this model is temporary to match their previous model. They announced [1] that it will be the following after February 8:

DeepSeek - 0.27$ per million tokens input, 1.10$ million tokens output (66 tokens per/s)

Still much cheaper than the others though for input pricing.

[1] https://api-docs.deepseek.com/news/news1226#-api-pricing-upd...

Re: Notes on the New Deepseek v3

#19

Post by bangaladore »

Arcuru wrote: post #17
bangaladore wrote: post #14

Earlier quoted context omitted.

Per available providers on OpenRouter right now: DeepSeek - 0.14$ per million tokens input, 0.28$ million tokens output (66 tokens per/s) Fireworks - 0.9$ per million tokens input, 0.9$ million tokens output (23 tokens per/s) DeepInfra - 1$ per million tokens input, 2$ million tokens output (1.27 tokens per/s) Compared to Llama 3.1 405B (smaller model than this afaik): Cheapest is 0.8/0.8$ at 24 t/s all the way to 4$…

Just a minor clarification, DeepSeek's pricing for this model is temporary to match their previous model. They announced [1] that it will be the following after February 8: DeepSeek - 0.27$ per million tokens input, 1.10$ million tokens output (66 tokens per/s) Still much cheaper than the others though for input pricing. [1] https://api-docs.deepseek.com/news/news1226#-api-pricing-upd...

Yeah, I was assuming they are selling for cheap to get people to try the model.

But still certainly cheaper than everyone else at the moment.

Re: Notes on the New Deepseek v3

#20

Post by darksaints »

egnehots wrote: post #8

If you understand how LLMs work, you should disregard tests such as: - How many 'r's are in Strawberry? - Finding the fourth word of the response These tests are at odds with the tokenizer and next-word prediction model. They do not accurately represent an LLM's capabilities. It's akin to asking a blind person to identify colors.

The funny thing about those "tests" is that LLMs are judged by their ability to do that themselves, as opposed to their ability to write code that does it. The best LLMs still fail at doing the task themselves, because they fundamentally are not designed to do anything except predict tokens. But they absolutely can write code that does it perfectly, and can write code that does so many things better than that.
Post reply on HN