Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Sell ∀ ∃ as ∃ ∀ Scam

win-vector.com

Post reply on HN
Print view View original

31–40 of 148 posts

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#31

Post by debaserab2 »

badloginagain wrote: post #10

If I understand correctly, the meat of the argument is "that is a system for every (∀) task, there exists (∃) a setting that gives the correct answer for that one task." My understanding of this (correct me if I'm wrong) is that the scam is convincing users that GPT-X can do anything with say, the correct prompts. This argument misses the mark for me. It's not that it solves all the problems, it's that the problems i…

> Significantly economically impactful in some cases- obvious examples of call centers and first-line customer support.

Is it that obvious?

Yesterday I had a trivial but uncommon issue with my pharmacy. I reached out to them online - their chatbot was the only channel available. I tried, over the course of 20 minutes and 3 restarted sessions, to communicate an issue that a human would have been able to respond to in 30 seconds. Eventually I just gave up and got the prescription filled elsewhere.

No doubt this pharmacy saved money by cutting support staff. I just think it's easy to see these solutions and cost savings without bothering to look at how much of a frustrating experience it can be for a customer.

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#32

Post by akkartik »

jameshart wrote: post #26
version_five wrote: post #11

Earlier quoted context omitted.

He, imo correctly, puts them both in the category of extra degrees of freedom that can allow the user to overfit and get results that appear more impressive than the underlying reality about how the model has generalized.

But generating new prompts for GPT is incredibly cheap! You just rephrase and ask again. That there are prompts which generate impressive results with GPT is the point . Because anyone can generate prompts - and get impressive results. Whereas hyper parameter tuning is expensive . A system that generates good results with the right tuning doesn’t tell you much about your ability to use it to generate good results, be…

Think about when you use GPT to generate prompts, something that seems to be growing common. Once you have a pipeline like that, changing the (meta)prompt can be expensive. You designed the pipeline thinking you only have to come up with the meta-prompt once, and it would work from now on. But you find you have to keep tuning your Kubernetes/upgrading your dependencies/tweaking your meta prompt.

"Scam" is overblown, but I think OP is right to warn of a possible future issue. It's an issue endemic to all software, so not something worth calling out recent AI advances in particular for. But it seems something we should all be trying to get better about. Monitor the ongoing costs of the systems we create. Do they really pay for themselves or are we waiting for a Godot that never arrives?

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#33

Post by bitL »

This has always been present in subfields of AI. For example, in classical computer vision, one had to figure out specific parameters working just for a single image or video scene by hand. Machine learning can in theory at least make these parameters learnable at the cost of complexity.

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#34

Post by tialaramex »

TZubiri wrote: post #29

Are compilers a scam as well? There exists a program every problem you have, you just have to find the code.

Crucially there is not a program for every problem. Many (presumably "Almost all" in a mathematical sense) problems are Undecidable and so a program can't do that.

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#35

Post by [deleted] »

debaserab2 wrote: post #31
badloginagain wrote: post #10

If I understand correctly, the meat of the argument is "that is a system for every (∀) task, there exists (∃) a setting that gives the correct answer for that one task." My understanding of this (correct me if I'm wrong) is that the scam is convincing users that GPT-X can do anything with say, the correct prompts. This argument misses the mark for me. It's not that it solves all the problems, it's that the problems i…

> Significantly economically impactful in some cases- obvious examples of call centers and first-line customer support. Is it that obvious? Yesterday I had a trivial but uncommon issue with my pharmacy. I reached out to them online - their chatbot was the only channel available. I tried, over the course of 20 minutes and 3 restarted sessions, to communicate an issue that a human would have been able to respond to in…

[deleted]

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#36

Post by tanseydavid »

debaserab2 wrote: post #31
badloginagain wrote: post #10

If I understand correctly, the meat of the argument is "that is a system for every (∀) task, there exists (∃) a setting that gives the correct answer for that one task." My understanding of this (correct me if I'm wrong) is that the scam is convincing users that GPT-X can do anything with say, the correct prompts. This argument misses the mark for me. It's not that it solves all the problems, it's that the problems i…

> Significantly economically impactful in some cases- obvious examples of call centers and first-line customer support. Is it that obvious? Yesterday I had a trivial but uncommon issue with my pharmacy. I reached out to them online - their chatbot was the only channel available. I tried, over the course of 20 minutes and 3 restarted sessions, to communicate an issue that a human would have been able to respond to in…

Do you have any reason to believe that the Chatbot was GPT3.5 or GPT4 based?

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#37

Post by _royadar »

alanbernstein wrote: post #21

My biggest takeaway from the article is my new favorite word: cryptomorphic, meaning equivalent, but not obviously so.

Thanks for pointing that out, I'll add it to the list of words I'm supposed to use to sound smart.

However, I did find the main argument compelling, through my own waste of time (yes I learned it the hard way) I've come to acknowledge that the tradeoff for prompting GPT in order to get a valuable answer, is just not worth it.

However, it does seem that most of the people are intrigued by "it can answer anything with the right prompt" promise, and they devote a lot of time in order to fulfill it.

You cannot trust GPT, you cannot rely on it and it can't replace anyone, until it learns to prompt itself.

I think what this article really implies is that - we the humans are doing the quality assurance for the GPT answers, if we take that out of the equation, and we don't give it quality prompts, etc.

It's worth nothing.

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#38

Post by Waterluvian »

tanseydavid wrote: post #36
debaserab2 wrote: post #31

Earlier quoted context omitted.

> Significantly economically impactful in some cases- obvious examples of call centers and first-line customer support. Is it that obvious? Yesterday I had a trivial but uncommon issue with my pharmacy. I reached out to them online - their chatbot was the only channel available. I tried, over the course of 20 minutes and 3 restarted sessions, to communicate an issue that a human would have been able to respond to in…

Do you have any reason to believe that the Chatbot was GPT3.5 or GPT4 based?

I’d be surprised if something like a pharmacy managed to adopt a tech that quickly. In my experience non-tech industries often take quite a while to adopt.

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#39

Post by mitchellh »

badloginagain wrote: post #10

If I understand correctly, the meat of the argument is "that is a system for every (∀) task, there exists (∃) a setting that gives the correct answer for that one task." My understanding of this (correct me if I'm wrong) is that the scam is convincing users that GPT-X can do anything with say, the correct prompts. This argument misses the mark for me. It's not that it solves all the problems, it's that the problems i…

Since my blog post is linked, I wanted to clarify something. While this appears to be the broad message, I don't think the author intended to imply this about my post specifically, but I still feel the need that I point out the following in my prompt eng blog post (linked by the OP)[1]:

> To start, you must have a problem you're trying to build a solution for. The problem can be used to assess whether prompting is the best solution or if alternative approaches exist that may be better suited as a solution. Engineering starts with not using a method for the method's sake, but driven by the belief that it is the right method.

[1]: https://mitchellh.com/writing/prompt-engineering-vs-blind-pr...

Re: The Sell ∀ ∃ as ∃ ∀ Scam

#40

Post by LegitShady »

_Microft wrote: post #2

Since there are certainly people unfamiliar with the symbols, here is the Wikipedia article on them: https://en.wikipedia.org/wiki/Quantifier_(logic)

I just want to know what they're called if someone were to say them out loud. Upside down A and backwards E probably aren't accurate.
Post reply on HN