I thought it was a joke?
Ask HN: Is prompt engineering just snake oil?
201–206 of 206 posts
Re: Ask HN: Is prompt engineering just snake oil?
#202Earlier quoted context omitted.
> The article seems reasonable but ... closer to linting then testing... they don't check the interesting parts can you elaborate a bit more on what those interesting parts are? It could just be a limitation of computation.
We are helping our users with qa tasks involving code generation, where the answers may be either JSON, executable code, or markdown discussions involving the same. We are tuning for a bunch of tools following that pattern so our users don't have to. It's easy to make a labeled training set for grading our homework (catching regressions, ...) in the case of classifiers, and that's basically what the blog post showed.…
I don't think LLMs are going to be able to solve that. There are a number of things that are assumed are true, but may not necessarily be true. This can potentially lead to multiple possible answers (outputs) given the same inputs.
For example determinism in code, its required for computation and its a system's property, but generalizing a test for it is really hard. Its a property, and by knowing its true or false you can make inferences on whether a system maintains those properties, but most of this is abstracted away at lower levels and since the context can't ever be fully shared with an LLM for evaluation, nor can it automatically switch contexts when evaluation fails, this most likely will never be solveable by computers when there exists one single input that produces two separate (different) outputs, at least from what I know about automata theory and computability.
Its generally considered a class of problems that can't be solved by turing machines.
https://en.wikipedia.org/wiki/Theory_of_computation
https://medium.com/@tarcisioma/limits-of-computation-231bf28... (overview)
https://en.wikipedia.org/wiki/Undecidable_problem (crux of the problem)
Re: Ask HN: Is prompt engineering just snake oil?
#203Earlier quoted context omitted.
Political engineering is pretty common in societies where deep state or establishment has a firm control and wants to engineer results their way. Most Western democracies are not that.
would you mind answering this question, please? it's been ignored twice despite being kind of the key point: > is social engineering snake oil, too?
Already answered above:
>For social or political engineering, you can gain definite knowledge (spying, observing, researching, information extraction) and then devise something.
Re: Ask HN: Is prompt engineering just snake oil?
#204Earlier quoted context omitted.
would you mind answering this question, please? it's been ignored twice despite being kind of the key point: > is social engineering snake oil, too?
Already answered above: >For social or political engineering, you can gain definite knowledge (spying, observing, researching, information extraction) and then devise something.
unfortunately not: that doesn't answer whether social engineering, too, is snake oil, which is a yes or no question
straight answers to straight questions tend to be better
Re: Ask HN: Is prompt engineering just snake oil?
#205It's okay; software engineering isn't real either. (In many countries, the word "engineer" is regulated -- you can't call yourself an engineer without professional qualifications and oversight.) https://en.m.wikipedia.org/wiki/Regulation_and_licensure_in_...
Isn't real? You write about people who sling PHP or Javascript spaghetti? I have engineering degree in computer science (I think it is mostly thing in eastern Europe, you have bunch of electronics, math, computer architecture to learn), I worked on automotive software which is I have to say fairly regulated. Where you have to care for things like ISO or MISRA standards in your code. I don't mind people calling themse…
> Isn't real?
> I have engineering degree in computer science
For the record, I also have bad news about our shared degree's use of the word "science".
Re: Ask HN: Is prompt engineering just snake oil?
#20610000000%