Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Promising results from DeepSeek R1 for code

simonwillison.net

Post reply on HN
Print view View original

71–80 of 765 posts

Re: Promising results from DeepSeek R1 for code

#71

Post by happyopossum »

nine_zeros wrote: post #50
jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

> Our trade has changed forever, and there's no going back. When companies claim that AI will replace developers, it isn't entirely bluster. Jobs are going to be lost unless there's somehow a demand for more applications This is a key insight - the trade has changed. For a long time, hoarding talent - who could conceive and implement such PRs - was a competitive advantage. It no longer is because companies can hire a…

> literally a single Deepseek release yesterday destroyed large market cap companies

Nobody was “destroyed” - a handful of companies had their stock price drop, a couple had big drops, but most of those stocks are up today, showing that the market is reactionary.

Re: Promising results from DeepSeek R1 for code

#72

Post by mjr00 »

jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

> I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.

Look at the code that was changed[0]. It's a single file. From what I can tell, it's almost purely functional with clearly specified inputs and outputs. There's no need to implement half the code, realize the requirements weren't specified properly, and go back and have a conversation with the PM about it. Which is, you know, what developers actually do.

This is the kind of stuff LLMs are great at, but it's not representative of a typical change request by Java Developer #1753 at Fortune 500 Enterprise Company #271.

[0] https://github.com/ggerganov/llama.cpp/pull/11453/files

Re: Promising results from DeepSeek R1 for code

#73

Post by simonw »

punkpeye wrote: post #67
simonw wrote: post #43

Earlier quoted context omitted.

Using the smaller distilled versions. I'm running this one, which only needs 20GB of VRAM (or regular RAM on Apple Silicon): https://ollama.com/library/deepseek-r1:32b

Do we know how do these distilled versions perform in benchmarks?

DeepSeek published a bunch of benchmarks when they released the models: https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-fil...

I'd like to see detailed benchmarks run by other unaffiliated organizations.

Re: Promising results from DeepSeek R1 for code

#74

Post by 20k »

jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

Eh it performed a 1:1 conversion of ARM NEON to wasm SIMD, which with the greatest will in the world is pretty trivial work. Its something that ML is good at, because its the same problem area as "translate this from english to french", but more mechanistic

This is a task that would likely have taken as long to write by hand as the AI took to do it, given how long the actual task took to execute. 98% of the work is find and replace

Don't get me wrong - this kind of thing is useful and cool, but you're mixing up the easy coding donkey work with the stuff that takes up time

If you look at the actual prompt engineering part, its clear that this prompting produced extensively wrong results as well, which is tricky. Because it wasn't produced by a human, it requires extensive edge case testing and review, to make sure that the AI didn't screw anything up. If you have the knowledge to validate the output, it would have been quicker to write it by hand instead of reverse engineering the logic by hand. Its bumping the work off from writing it by hand, to the reviewers who now have to check your ML code because you didn't want to put in the work by hand

So overall - while its extremely cool that it was able to do this, it has strong downsides for practical projects as well

Re: Promising results from DeepSeek R1 for code

#75

Post by chrisguilbeau »

jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

I'm a developer that primarily uses gh copilot for python dev. I find it pretty useful as an intelligent auto-completer that understands our project's style, and unusual decorators we use.

What tools would you tell a copilot dev to try? For example, I have a $20/mo ChatGPT account and asking it to write code or even fix things hasn't worked very well. What am I missing?

Re: Promising results from DeepSeek R1 for code

#76

Post by cynicalpeace »

jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

It's making programming more boring and more of an admin task- which is sure to attract different types of people to the field.

Re: Promising results from DeepSeek R1 for code

#77

Post by fofoz »

When these models succeed in building a whole program and a whole system then the software industry that creates products and services will disappear. Any person and any organization will create from scratch the software they need perfectly customized to their needs and the AI system will evolve it over time. At most they will have to cooperate on communication protocols. In my opinion we are less than 5 years away from this event.

Re: Promising results from DeepSeek R1 for code

#78

Post by sitkack »

simonw wrote: post #52
jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

"Jobs are going to be lost unless there's somehow a demand for more applications." That's why I'm not worried. There is already SO MUCH more demand for code than we're able to keep up with. Show me a company that doesn't have a backlog a mile long where most of the internal conversations are about how to prioritize what to build next. I think LLM assistance makes programmers significantly more productive, which makes…

We have already entered a new paradigm of software development, where small teams build software for themselves to solve their own problems rather than making software to sell to people. I think selling software will get harder in the future unless it comes with special affordances.

Re: Promising results from DeepSeek R1 for code

#79

Post by simonw »

mjr00 wrote: post #72
jeswin wrote: post #31

> 99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1 I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these.…

> I hope we can put to rest the argument that LLMs are only marginally useful in coding - which are often among the top comments on many threads. I suppose these arguments arise from (a) having used only GH copilot which is the worst tool, or (b) not having spent enough time with the tool/llm, or (c) apprehension. I've given up responding to these. Look at the code that was changed[0]. It's a single file. From what I…

"Yeah, but LLMs can't handle millions of lines of crufty old Java" is a guaranteed reply any time this topic comes up.

(That's not to say it isn't a valid argument.)

Short answer: LLMs are amazingly useful on large codebases, but they are useful in different ways. They aren't going to bang out a new feature perfectly first time, but in the right hands they can dramatically accelerate all sorts of important activities, such as:

- Understanding code. If code has no documentation, dumping it into an LLM can help a lot.

- Writing individual functions, classes and modules. You have to be good at software architecture and good at prompting to use them in this way - you take on the role of picking out the tasks that can be done independently of the rest of the code.

- Writing tests - again, if you have the skill and experience to prompt them in the right way.

Re: Promising results from DeepSeek R1 for code

#80

Post by TheBigSalad »

simonw wrote: post #60
TheBigSalad wrote: post #55

Earlier quoted context omitted.

LLMs are only marginally useful for coding. You have simply chosen to dismiss or or 'give up' on that fact. You've chosen what you want to believe in contrast to the reality that we are all experiencing.

LLMs are incredibly useful for coding, if you learn how to apply them effectively. You have simply chosen to dismiss or 'give up' on that fact.

They are a useful tool, but not 'incredibly useful'. The simple, repetitive code in this example is what they are good at. It's like 1% of what I do working on products. Writing code isn't even that impressive, the whole job is figuring out exactly what people want.
Post reply on HN