Live data from Hacker News

Can LLMs write better code if you keep asking them to “write better code”?

minimaxir.com

451–460 of 461 posts

Re: Can LLMs write better code if you keep asking them to “write better code”?

#451
post #431

Earlier quoted context omitted.

Is there source code / a GitHub link with more information?

https://github.com/internetarchive/openlibrary

Is there a specific issue with more context? I looked at the repo already but it’s not obvious which operations are slowest / most important to optimize.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#452

Earlier quoted context omitted.

The answer to your question is "can AI be a doctor".

This doesn't make any sense: it's a question, not the answer. I don't see the relevance of doctors to the current topic anymore. Your initial reference made sense as an analogy (although the analogy itself was irrelevant), but the new reference doesn't make any sense whatsoever.

Apart from our discussion, I guess no one is reading our comments. Therefore, we should stop sharing our thoughts and analogies.

Nice conversation with you, Victor!

Re: Can LLMs write better code if you keep asking them to “write better code”?

#453

> how to completely uninstall and reinstall postgresql on a debian distribution without losing the data in the database. https://www.phind.com/search?cache=lrcs0vmo0wte5x6igp5i3607 Still seem to struggle on basic instructions, and even understanding what it itself is doing. sudo rm -rf /etc/postgresql sudo rm -rf /var/lib/postgresql sudo rm -rf /var/log/postgresql > This process removes all PostgreSQL components, cle…

I have just opened your link and it does not contain the exact text you quoted anymore, now it is:

> This process removes all PostgreSQL components except the data directory, ensuring existing databases are retained during the reinstall. It provides a clean slate for PostgreSQL while maintaining continuity of stored data. Always backup important data before performing major system changes.

And as the first source it cites exactly your comment, strange

> https://news.ycombinator.com/item?id=42586189

Re: Can LLMs write better code if you keep asking them to “write better code”?

#454
post #431

Earlier quoted context omitted.

https://github.com/internetarchive/openlibrary

Is there a specific issue with more context? I looked at the repo already but it’s not obvious which operations are slowest / most important to optimize.

Unfortunately the discussion mostly is on their public Slack channel which you have to fill out the volunteer form to join.

But there are several more specific issues about performance: https://github.com/internetarchive/openlibrary/issues?q=is%3...

Re: Can LLMs write better code if you keep asking them to “write better code”?

#455

Earlier quoted context omitted.

O1 found it. https://chatgpt.com/share/67782b6b-6248-8012-882d-238b600ef9...

Did it found it before the HN comment? O1 has access to the web so I'm just asking

No it doesn't.

"When did Jeff Baena die?"

> There is no record or credible report indicating that Jeff Baena has passed away. As of the most recent information available, he is still alive. My training data includes information up to October 2023. Events or details that emerged after that date may not be reflected in my responses.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#456
post #55

I often run into LLMs writing "beginner code" that uses the most fundamental findings in really impractical ways. Trained on too many tutorials I assume. Usually, specifying the packages to use and asking for something less convoluted works really well. Problem is, how would you know if you have never learned to code without an LLM?

This ^

Re: Can LLMs write better code if you keep asking them to “write better code”?

#457
post #318

Earlier quoted context omitted.

Amazing. Next step would be to propose hardcoding 99930-3999 as the O(1) result and live with the output just being wrong sometimes. The bug rate is then in the ballpark of most modern software, including LLMs', so I'd say ship it.

Doesn’t this line of thinking constantly redefine success until all software is only bugs?

That's exactly what has been happening for decades now and what things like scrum actively encourage to happen.

Given work item does not fit into allotted timebox? Relax Definition of Done until it does ¯\_(ツ)_/¯

Re: Can LLMs write better code if you keep asking them to “write better code”?

#458
post #356

Earlier quoted context omitted.

> However, with T steps of CoT, constant-depth transformers using constant-bit precision and O(logn) embedding size can solve any problem solvable by boolean circuits of size T There is a difference between being equivalent to a circuit and prediction of the output of the BVSP. That is what I was suggesting learning descriptive complexity theory would help with.

Why does the limit on computational complexity of single decoder transformers matter for obtaining superhuman coding ability? Is there a theory of what level of complexity is needed for the task of coding according to a spec? Or the complexity for translation/optimization of a code? Even if there were, and one could show that a plain decoder transformer is insufficient, you probably only need to add a tool in the mid…

Rice's theorem means you can't choose to decide if a program is correct, but you have to choose an error direction and accept the epislon.

The Curry–Howard–Lambek correspondence is possibly a good tool to think about it.

The reason I suggested graduate level complexity theory is because the undergrad curriculum is flawed in that it seems that you can use brute force with a TM to stimulate a NTM with NP.

It is usually taught that NP is the set of decision problems that can be solved by a NTM in polynomial time.

But you can completely drop the NTM and say it is the set of decision problems that are verifiable by a DTM in poly time.

Those are equivalent.

Consider the The Approximate Shortest Vector Problem (GapSVP), which is NP-HARD, and equivalent to predicting the output of a 2 layer NN (IIRC).

Being NPH, it is no longer a decision problem.

Note that for big 0, you still have your scaler term. Repeated operations are typically dropped.

If you are in contemporary scale ML, parallelism is critical to problems being solvable, even with FAANG level budgets.

If you are limited to DLOGTIME-uniform TC0, you can't solve NC1- complete problems, and surely can't do P-complete problems.

But that is still at the syntactic level, software in itself isn't worth anything, it is the value it provides to users that is important.

Basically what you are claiming is that feed forward NN solve the halting problem, in a generalized way.

Training an LLM to make safe JWT refresh code is very different from generalized programming. Mainly because most of the ability for them to do so is from pre-training.

Inference time is far more limited, especially for transformers and this is well established.

https://arxiv.org/abs/2309.06926

Re: Can LLMs write better code if you keep asking them to “write better code”?

#459

Earlier quoted context omitted.

This is not what premature optimization is the root of all evil means. It’s a tautological indictment of doing unnecessary things. It’s not in support of making obviously naive algorithms. And if it were it wouldn’t be a statement worth focusing on. As the point of the article is to see if Claude can write better code from further prompting so it is completely appropriate to “optimize” a single implementation.

I have to disagree. Naive algorithms are absolutely fine if they aren’t performance issues. The comment you are replying to is making the point that “better” is context dependent. Simple is often better. > There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at e…

> Writing naive algorithms

Depends on the circumstance, and how difficult an appropriate algorithm is to write, but in my experience, if code performance is important, this tends to yield large, painful rewrites down the road.

Re: Can LLMs write better code if you keep asking them to “write better code”?

#460
post #298

The headline question here alone gets at what is the biggest widespread misunderstanding of LLMs, which causes people to systematically doubt and underestimate their ability to exhibit real creativity and understanding based problem solving. At it's core an LLM is a sort of "situation specific simulation engine." You setup a scenario, and it then plays it out with it's own internal model of the situation, trained on…

> At it's core an LLM is a sort of "situation specific simulation engine." You setup a scenario, and it then plays it out with it's own internal model of the situation, trained on predicting text in a huge variety of situations. This includes accurate real world models of, e.g. physical systems and processes, that are not going to be accessed or used by all prompts, that don't correctly instruct it to do so.

This idea of LLMs doing simulations of the physical world I've never heard before. In fact a transformer model cannot do this. Do you have a source?

Post reply on HN