Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The impact of competition and DeepSeek on Nvidia

youtubetranscriptoptimizer.com

Post reply on HN
Print view View original

141–150 of 500 posts

Re: The impact of competition and DeepSeek on Nvidia

#141

Post by manojlds »

>With the advent of the revolutionary Chain-of-Thought ("COT") models introduced in the past year, most noticeably in OpenAI's flagship O1 model (but very recently in DeepSeek's new R1 model, which we will talk about later in much more detail), all that changed. Instead of the amount of inference compute being directly proportional to the length of the output text generated by the model (scaling up for larger context windows, model size, etc.), these new COT models also generate intermediate "logic tokens"; think of this as a sort of scratchpad or "internal monologue" of the model while it's trying to solve your problem or complete its assigned task.

Is this right? I thought CoT was a prompting method and are we calling the reasoning models as CoT models?

Re: The impact of competition and DeepSeek on Nvidia

#142

Post by veesahni »

manojlds wrote: post #141

>With the advent of the revolutionary Chain-of-Thought ("COT") models introduced in the past year, most noticeably in OpenAI's flagship O1 model (but very recently in DeepSeek's new R1 model, which we will talk about later in much more detail), all that changed. Instead of the amount of inference compute being directly proportional to the length of the output text generated by the model (scaling up for larger context…

Reasoning models are a result of the learnings from CoT prompting.

Re: The impact of competition and DeepSeek on Nvidia

#143

Post by infecto »

zby wrote: post #101
simonw wrote: post #75

Earlier quoted context omitted.

Honestly I tried searching his recent tweets for AMD and there was way too much noise in there to figure out his current position!

" we are going to move it off AMD to our own or partner silicon. We have developed it to be very portable." https://x.com/__tinygrad__/status/1879617702526087346

Honest question. That sounds more difficult that getting things to play with commodity hardware. Maybe I am oversimplifying it though.

Re: The impact of competition and DeepSeek on Nvidia

#144

Post by billconan »

toisanji wrote: post #31
breadwinner wrote: post #26

Great article but it seems to have a fatal flaw. As pointed out in the article, Nvidia has several advantages including: - Better Linux drivers than AMD - CUDA - pytorch is optimized for Nvidia - High-speed interconnect Each of the advantages is under attack: - George Hotz is making better drivers for AMD - MLX, Triton, JAX: Higher level abstractions that compile down to CUDA - Cerbras and Groq solve the interconnect…

I want the NVIDIA monopoly to end, but there is no real competition still. * George Hotz has basically given up on AMD: https://x.com/__tinygrad__/status/1770151484363354195 * Groq can't produce more hardware past their "demo". It seems like they haven't grown capacity in the years since they announced, and they switched to a complete SaaS model and don't even sell hardware anymore. * I dont know enough about MLX, Tr…

I also noticed that Groq's Chief Architect now works for NVIDIA.

https://research.nvidia.com/person/dennis-abts

Re: The impact of competition and DeepSeek on Nvidia

#145

Post by s1mplicissimus »

veesahni wrote: post #142
manojlds wrote: post #141

>With the advent of the revolutionary Chain-of-Thought ("COT") models introduced in the past year, most noticeably in OpenAI's flagship O1 model (but very recently in DeepSeek's new R1 model, which we will talk about later in much more detail), all that changed. Instead of the amount of inference compute being directly proportional to the length of the output text generated by the model (scaling up for larger context…

Reasoning models are a result of the learnings from CoT prompting.

I'm curious what are the key differences between "a reasoning model" and good old CoT prompting. Is there any reason to believe that the fundamental limitations of prompting don't apply to "reasoning models"? (hallucinations, plainly wrong output, bias towards to training data mean etc.)

Re: The impact of competition and DeepSeek on Nvidia

#146

Post by [deleted] »

veesahni wrote: post #142
manojlds wrote: post #141

>With the advent of the revolutionary Chain-of-Thought ("COT") models introduced in the past year, most noticeably in OpenAI's flagship O1 model (but very recently in DeepSeek's new R1 model, which we will talk about later in much more detail), all that changed. Instead of the amount of inference compute being directly proportional to the length of the output text generated by the model (scaling up for larger context…

Reasoning models are a result of the learnings from CoT prompting.

[deleted]

Re: The impact of competition and DeepSeek on Nvidia

#147

Post by kimbler »

dismalaf wrote: post #83
breadwinner wrote: post #37

Earlier quoted context omitted.

AMD P/E ratio is 109, NVDA is 56. Which stock is overvalued?

NVDA is valued at $3.5 trillion, which means investors think it will grow to around $1 trillion in yearly revenue. Current revenue is around $35 billion per quarter, so call it $140 billion yearly. Investors are betting on a 7x increase in revenue. Not impossible, sounds plausible but you need to assume AMD, INTC, GOOG, AMZN, and all the others who make GPUs/TPUs either won't take market share or the market will be w…

I thought the valuation of public companies at 3x revenues or 5x earnings has long since sailed?

Re: The impact of competition and DeepSeek on Nvidia

#148

Post by sebmellen »

adastra22 wrote: post #93
isatty wrote: post #92

Earlier quoted context omitted.

Sorry, I don’t know who George Hotz is, but why isn’t AMD making better drivers for AMD?

George Hotz is a hot Internet celebrity that has basically accomplished nothing of value but has a large cult following. You can safely ignore. (Famous for hacking the PS3–except he just took credit for a separate group’s work. And for making a self-driving car in his garage—except oh wait that didn’t happen either.)

Comma.ai works really well. I use it every day in my car.

Re: The impact of competition and DeepSeek on Nvidia

#149

Post by itchyjunk »

s1mplicissimus wrote: post #145
veesahni wrote: post #142

Earlier quoted context omitted.

Reasoning models are a result of the learnings from CoT prompting.

I'm curious what are the key differences between "a reasoning model" and good old CoT prompting. Is there any reason to believe that the fundamental limitations of prompting don't apply to "reasoning models"? (hallucinations, plainly wrong output, bias towards to training data mean etc.)

The level of sophistication for CoT model varies. "good old CoT prompting" is you hoping the model generates some reasoning tokens prior to the final answer. When it did, the answers tended to be better for certain class of problems. But you had no control over what type of reasoning tokes it was generating. There were hypothesis that just having a tokens in between generated better answers as it allowed n+1 steps to generate an answer over n. I would consider Meta's "continuous chain of thought" to be on the other end of "good old CoT prompting" where they are passing back the next tokens from the latent space back to the model getting a "BHF" like effect. Who knows what's happening with O3 and Anthropics O3 like models.. The problems you mentioned is very broad and not limited to prompting. Reasoning models tend to outperform older models on math problems. So I'd assume it does reduce hallucination on certain class of problems.

Re: The impact of competition and DeepSeek on Nvidia

#150

Post by kimbler »

Nvidia seem to be one step ahead of this and you can see their platform efforts are pushing towards creating large volumes of compute that are easy to manage for whatever your compute requirements are, be that training, inference or whatever comes next and whatever form. People are maybe tackling some of these areas in isolation but you do not want to build datacenters where everything is ringfenced per task or usage.
Post reply on HN