Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT Enterprise

openai.com

Post reply on HN
Print view View original

301–310 of 532 posts

Re: ChatGPT Enterprise

#301

Post by wunderwuzzi23 »

feralderyl wrote: post #269
wunderwuzzi23 wrote: post #89

Wonder if for the Enterprise version they will fix the Image Markdown Data Exfiltration vulnerability that's been known for a while. https://embracethered.com/blog/posts/2023/chatgpt-webpilot-d... Seems like a no-go for companies if an attacker can steal stuff.

can you briefly explain the vulnerability here. I'm having difficulty understanding as all I see is him recalling already previously recorded chat history of his own session. thanks.

The vulnerability is the automatic insecure rendering of image markdown. One way to trigger it is with an indirect prompt injection payload. The scenario is that the user analyzes some text/data, which contains malicious instructions. The owner of the text doesn't have access to the chat history (it's just some random text somewhere), it could be a comment on a webpage, text inside a pdf file, copy/pasting, or even instructions hidden inside an image the user analyzes and sends to the LLM. You can find many examples of indirect prompt injections on my blog (e.g. analyzing YouTube transcripts,...). Just yesterday I put up a video explaining the various TTPs (and also fixes companies put in place): https://www.youtube.com/watch?v=L_1plTXF-FE Hope that helps.

Re: ChatGPT Enterprise

#303

Post by earthboundkid »

PeterCorless wrote: post #298
lijok wrote: post #295

Earlier quoted context omitted.

Should get better over time as the tool gets better incorporated into the industry. People always misuse new tools

In such cases, I believe there are (and should be) guardrails. It already has some guardrails internally set: ChatGPT already knows how to say: "I am not a lawyer, but I can provide you with some general information on this topic." The same thing for medicine: "I'm not a medical professional, but I can provide some general information..." But, for instance, ask it to design a canard for a 4th generation supersonic fi…

One of the bounding boxes should be requests to introspect. The models absolutely are incapable of introspection. They just simulate a human introspecting, but they think totally differently from a human, so the output is less than useless. If you ask “why did you say X” it should just refuse to answer, the same as if you asked it to swear or say something racist.

Re: ChatGPT Enterprise

#304

Post by PeterCorless »

PeterCorless wrote: post #288

I hereby dub this "Baron-von-Munchausen-as-a-Service." Now you can pay real money for a chatbot to make stuff up about your company and its products. While it is pretty incredible stuff, until or unless they have a veracity bit — some sort of "please don't lie" flag in it, I'd be wary of what it produces. Will ChatGPT offer off-the-cuff pricing, discounts, rebates and refunds that are in line with your actual busines…

By the way, I asked ChatGPT itself about how it would respond to the message above. Its answer:

The statement you provided raises valid concerns about the capabilities and potential risks of advanced language models like ChatGPT Enterprise. Here's a response that addresses these concerns:

"Thank you for sharing your thoughts on ChatGPT Enterprise. It's important to recognize that advanced language models like ChatGPT do indeed have a remarkable capacity to generate human-like responses. However, they lack inherent understanding, critical thinking, and ethical judgment.

OpenAI acknowledges the concerns you've highlighted about accuracy, veracity, and potential consequences of generated content. It's crucial to understand that ChatGPT does not have an inherent "veracity bit" or a built-in mechanism to discern truth from falsehood. As such, it's important for users to exercise caution and discretion when relying on the output generated by the model.

In situations where accuracy and compliance are paramount, it's advisable to rely on established procedural logic, expert systems, and human review. ChatGPT is not a substitute for human oversight, particularly in sensitive areas such as legal, medical, or business transactions.

OpenAI is actively working to improve the control and guidance mechanisms for ChatGPT, and they acknowledge the importance of striking the right balance between automation and human review. As you pointed out, there are instances where generative AI is appropriate and others where traditional procedural logic and human intervention are necessary.

While ChatGPT can be a powerful tool for generating content and ideas, it is incumbent upon users to apply their expertise and judgment to evaluate the output, particularly in contexts where accuracy and adherence to policies are critical.

Your concerns align with ongoing discussions in the AI community about the ethical use and potential limitations of these technologies. As AI systems continue to advance, responsible development, transparency, and user education will remain important factors in their deployment."

It's important to remember that while AI models like ChatGPT have the potential to assist and enhance various tasks, they do not replace the need for human expertise, ethical considerations, and careful oversight in complex and critical scenarios.

Re: ChatGPT Enterprise

#305

Post by andrewjl »

PeterCorless wrote: post #288

I hereby dub this "Baron-von-Munchausen-as-a-Service." Now you can pay real money for a chatbot to make stuff up about your company and its products. While it is pretty incredible stuff, until or unless they have a veracity bit — some sort of "please don't lie" flag in it, I'd be wary of what it produces. Will ChatGPT offer off-the-cuff pricing, discounts, rebates and refunds that are in line with your actual busines…

Companies already hire (and pay them real money too) people to make stuff up about their products.

Re: ChatGPT Enterprise

#307

Post by blibble »

agnokapathetic wrote: post #207
blibble wrote: post #192

Earlier quoted context omitted.

> We do not train on your business data or conversations, and our models don’t learn from your usage. doesn't say they're not selling it to someone else who might...

This would be a violation of GDPR and CCPA and would expose them to massive litigation liability.

"fair use"

Re: ChatGPT Enterprise

#308

Post by suby »

gcanyon wrote: post #233
swores wrote: post #113

Earlier quoted context omitted.

I still have access in my $20/m non-Enterprise Pro account, though it has indeed just updated its name from Code Interpreter to Advanced Data Analysis. I haven't personally noticed it being any better than standard GPT4 even for generation of code that can't be run by it (ie non-Python code).

I also have a pro account, and I’ve looked for and not seen code interpreter in my account. Am I just missing it?

You may have to go into settings and enable it under beta options.

Re: ChatGPT Enterprise

#309

Post by PeterCorless »

tsunamifury wrote: post #300
PeterCorless wrote: post #288

I hereby dub this "Baron-von-Munchausen-as-a-Service." Now you can pay real money for a chatbot to make stuff up about your company and its products. While it is pretty incredible stuff, until or unless they have a veracity bit — some sort of "please don't lie" flag in it, I'd be wary of what it produces. Will ChatGPT offer off-the-cuff pricing, discounts, rebates and refunds that are in line with your actual busines…

This sounds a bit like word salad, and your argument is worse than an LLM output. You even contradict yourself at the end. Also I think you need to better understand how much hallucination has been driven down recently and its clear path going forward.

“This isn’t fixable,” said Emily Bender, a linguistics professor and director of the University of Washington’s Computational Linguistics Laboratory. “It’s inherent in the mismatch between the technology and the proposed use cases.”

https://apnews.com/article/artificial-intelligence-hallucina...

Re: ChatGPT Enterprise

#310

Post by lijok »

biophysboy wrote: post #297
lijok wrote: post #295

Earlier quoted context omitted.

Should get better over time as the tool gets better incorporated into the industry. People always misuse new tools

But why will it get better at accuracy if it isnt trained to be accurate?

What I mean is, the situation of people misusing it should get better over time
Post reply on HN