Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 vision prompt injection

blog.roboflow.com

Post reply on HN
Print view View original

111–118 of 118 posts

Re: GPT-4 vision prompt injection

#111

Post by mjburgess »

YeGoblynQueenne wrote: post #110
mjburgess wrote: post #35

Earlier quoted context omitted.

This is a great example of the myopia of computer scientists. The meaning here is obvious, and the MP is remarkably insightful. When I ask a question with a mistake in it, a human will either correct that mistake or ask me questions to clarify it. Such is an essential component to real communication. If communication is just a procedural activity where, either by wrote or by statistics, an answer is derived by algori…

This was in the 1850s. Babbabe was not trying to make a machine that thinks like a human. He designed a mechanical calculator capable of automatically solving differential equations, not a chatbot capable of holding a conversation with the user. Perhaps the Difference Engine was described as a "mechanical brain" or something similar and that gave the MP the wrong expectation. He wasn't being insightful, only confused…

babbage was very much selling it as a miracle machine -- i think these replies echo the debate today.

one myopic side of engineers, another with an intuitive understanding of ecological rationality... a complete chasm of understanding whereby the machinist thinks of themselves as a series of cogs

Babbage here, is being archetypally dumb -- the dumbness of his ilk reduced down in this perfectly condescending quote

Re: GPT-4 vision prompt injection

#112

Post by YeGoblynQueenne »

mjburgess wrote: post #111
YeGoblynQueenne wrote: post #110

Earlier quoted context omitted.

This was in the 1850s. Babbabe was not trying to make a machine that thinks like a human. He designed a mechanical calculator capable of automatically solving differential equations, not a chatbot capable of holding a conversation with the user. Perhaps the Difference Engine was described as a "mechanical brain" or something similar and that gave the MP the wrong expectation. He wasn't being insightful, only confused…

babbage was very much selling it as a miracle machine -- i think these replies echo the debate today. one myopic side of engineers, another with an intuitive understanding of ecological rationality... a complete chasm of understanding whereby the machinist thinks of themselves as a series of cogs Babbage here, is being archetypally dumb -- the dumbness of his ilk reduced down in this perfectly condescending quote

Why dumb? Because he understands how his machine works? I don't get it.

Re: GPT-4 vision prompt injection

#113

Post by bytefactory »

mathgorges wrote: post #108
bytefactory wrote: post #93

Earlier quoted context omitted.

Interesting. I like your idea in one of your posts of separating out system prompts and user inputs. Seems promising.

Thus separating the model’s logic from the model’s data. All that was old is new again :) [0] 0: s/model/program/

It's interesting how this is not presumably the case within the weights of the LLM itself. Those probably encode data as well as logic!

Re: GPT-4 vision prompt injection

#114

Post by mjburgess »

YeGoblynQueenne wrote: post #112
mjburgess wrote: post #111

Earlier quoted context omitted.

babbage was very much selling it as a miracle machine -- i think these replies echo the debate today. one myopic side of engineers, another with an intuitive understanding of ecological rationality... a complete chasm of understanding whereby the machinist thinks of themselves as a series of cogs Babbage here, is being archetypally dumb -- the dumbness of his ilk reduced down in this perfectly condescending quote

Why dumb? Because he understands how his machine works? I don't get it.

dumb in his inability to understand the question he was being asked, because he could only think in terms of his machine

Re: GPT-4 vision prompt injection

#115

Post by zamadatix »

famouswaffles wrote: post #89
zamadatix wrote: post #82

Earlier quoted context omitted.

As the resulting combined system of anything you use a book, search engines, wikis, and forums as part of is unlike the raw source information by itself sure. The ChatGPT "AI" isn't an autonomous thinker performing its own actions based on reasoning of what's fed to it. In all it's in no different than any of our previous systems in that it's "just" (still very useful) compression and next-token-predictor which is so…

>The ChatGPT "AI" isn't an autonomous thinker performing its own actions based on reasoning of what's fed to it. Yes it is. Or it very well could be. Agency is trivial to implement in LLMs. https://github.com/microsoft/autogen https://arxiv.org/abs/2307.07924 The intelligence and tool access of the LLM in question is the only thing stopping things from being particularly dangerous (to humanity).

The only way you'll get intelligence is if these models start a permanent training cycle, feeding input and output between them makes a larger model not one computing its data in a new way.

Re: GPT-4 vision prompt injection

#116

Post by zamadatix »

thfuran wrote: post #94
zamadatix wrote: post #82

Earlier quoted context omitted.

As the resulting combined system of anything you use a book, search engines, wikis, and forums as part of is unlike the raw source information by itself sure. The ChatGPT "AI" isn't an autonomous thinker performing its own actions based on reasoning of what's fed to it. In all it's in no different than any of our previous systems in that it's "just" (still very useful) compression and next-token-predictor which is so…

>As the resulting combined system of anything you use a book, search engines, wikis, and forums as part of is unlike the raw source information by itself sure And the GP was clearly referring to such a system, so insisting that it's just a book seems, charitably, off topic.

The point is combining these systems doesn't result in intelligence so such a combined system doesn't either, not that such combined systems are an exception to my point.

Re: GPT-4 vision prompt injection

#117

Post by zamadatix »

pixl97 wrote: post #106
zamadatix wrote: post #82

Earlier quoted context omitted.

As the resulting combined system of anything you use a book, search engines, wikis, and forums as part of is unlike the raw source information by itself sure. The ChatGPT "AI" isn't an autonomous thinker performing its own actions based on reasoning of what's fed to it. In all it's in no different than any of our previous systems in that it's "just" (still very useful) compression and next-token-predictor which is so…

Humans suck at systems thinking. A snowflake is harmless. A million of them and you might freeze. And a trillion of them may bury your entire city under an avalanche. Add in the AI-effect where when we learn how something works it's no longer AI, and eventually we'll get to the point of having super capable 'intelligent' digital systems where a huge portion of the population is in denial of their capabilities.

I think we'll get there eventually, and maybe not that long of an eventually in the grand scheme of things. If one wants to bash the potential future handling of AI developments because of this I have no qualms. I only take issue with the idea anything ChatGPT Code in particular is doing is related to those concerns.

Re: GPT-4 vision prompt injection

#118

Post by YeGoblynQueenne »

mjburgess wrote: post #114
YeGoblynQueenne wrote: post #112

Earlier quoted context omitted.

Why dumb? Because he understands how his machine works? I don't get it.

dumb in his inability to understand the question he was being asked, because he could only think in terms of his machine

What other terms should he be trying to think in? He was asked about his machine! And he understood the question perfectly well. The asker thought his machine is some kind of Victorian ChatGPT that enters a dialogue with the user.

I mean, imagine the Wright brothers: "Does your machine need to build a nest to lay its eggs? Does it migrate in the winter?". What are they supposed to think, no, the question makes sense because our machine flies like a bird so it should be expected to behave like a bird in other ways also?

Post reply on HN