Live data from Hacker News

How is ChatGPT's behavior changing over time?

arxiv.org

181–187 of 187 posts

Re: How is ChatGPT's behavior changing over time?

#181

Earlier quoted context omitted.

> Also sounds like you haven't actually used Azure OpenAI On the contrary, I am using Azure OpenAI daily at work, and I'm explicitly not allowed to use "regular" OpenAI offerings. > It has the same 30 day retention for legal reasons unless you manually request (just like OpenAI) It doesn't, at least not for us. > Azure OpenAI has a narrower built in filter that you can't modify without again, a separate request. I'm…

> It doesn't, at least not for us. So then you filled out the request because the default is exactly the same as OpenAI: retained unless you manually apply for an exception. https://customervoice.microsoft.com/Pages/ResponsePage.aspx?... > I'm not sure if it's narrower, but it is there and I have a strong suspicion that MS is just trying to extract additional rent from companies that really want to turn the filter of…

> I don't know if you actually believe this or you're just not aware, but the companies that don't play fast and loose aren't using OpenAI period: Azure flavored or otherwise.

They do, and that's the biggest value proposition of Azure OpenAI right now: strong contractual guarantees, from a reputable partner (that's easy to hit with lawsuits should they go rogue :)).

The current situation is that it's pretty unwise for any company to ignore GPT models. OpenAI itself is a wildcard, but getting the same from Microsoft isn't "playing fast and loose with data" any more than using Windows and Office 365 across the organization is. Most large corporations and governments have been building their office work and communication around those tools for decades now, so - questions of antitrust aside - all the kinks have been worked out. I don't think you appreciate how big a difference this makes.

I mean, it's either that or all the company communication I got on this was bullshit.

> OpenAI has SOC2, GDPR and CCPA compliance. They comply with HIPPA and offer BAs.

That's the first I hear of it, but since I never dealt with OpenAI itself on that level, I accept this was my ignorance speaking; thanks for clarifying.

> You're pretty much proving the value of Azure in your comment: it's a veneer of familiarity that coaxes people who are convinced the new kid on the block must be untrustworthy.

I think you're underestimating the importance of this. What you call "veneer of familiarity" translates to billions of dollars of differences in terms of security risk.

As mentioned before, MS has been in this space for a while, and has decades of trust and experience built with governments and corporations and other big organizations. Microsoft is a known, trusted quantity. That alone is worth a lot.

But then, there are also technical aspects too - like how deploying to a tenant on Azure integrates properly with all the other services you use to run half the company. In practical terms, this means all use is monitored and auditable by in-house teams, and all the in-house policies are being enforced. OpenAI can't begin to offer this level of integration - they have neither technical nor legal resources for that.

> If OpenAI can't promise something Azure can't either: They're entirely dependent on OpenAI for this. Every idiosyncrasy behind Azure OpenAI maps back 1:1 to OpenAI.

None of that matters here. The models are what they are - peculiar large matrix multiplication as a service. By themselves, they're pretty much pure functions. The part that matters is operations - both technical and legal aspects - and this is where Microsoft and OpenAI are independent and have different offerings.

Also, looking at the way money flows, I think it's OpenAI that's dependent on Microsoft right now, not the other way around. They kinda pretend to be just friends with benefits, but it's obvious who the dependent party is.

Re: How is ChatGPT's behavior changing over time?

#182
post #82

Earlier quoted context omitted.

You keep repeating that. You don't even know if the people commenting to you use the API or the "web app". I use the API and I noticed the same stuff others have.

> Have you seen the 25 messages/3 hours limitation for GPT-4? If can't tell if that's about the API or the web app, I don't think you're familiar enough with the subject to speak on it.

I don't need to be familiar with it to see that you're not being genuine in your interpretation of peoples' comments and in the way you're responding to people in this thread. Case in point: your reply to me.

Re: How is ChatGPT's behavior changing over time?

#183

Earlier quoted context omitted.

> It doesn't, at least not for us. So then you filled out the request because the default is exactly the same as OpenAI: retained unless you manually apply for an exception. https://customervoice.microsoft.com/Pages/ResponsePage.aspx?... > I'm not sure if it's narrower, but it is there and I have a strong suspicion that MS is just trying to extract additional rent from companies that really want to turn the filter of…

> I don't know if you actually believe this or you're just not aware, but the companies that don't play fast and loose aren't using OpenAI period: Azure flavored or otherwise. They do, and that's the biggest value proposition of Azure OpenAI right now: strong contractual guarantees, from a reputable partner (that's easy to hit with lawsuits should they go rogue :)). The current situation is that it's pretty unwise fo…

I'm advising on calls with firms that have existed since the 1800s: their clients don't even want LLMs involved in output, regardless of who's hosting what.

Companies that don't play fast and loose are not using LLMs yet. They use "old school" ML at most with much narrower scope because at this point it's simply less of a liability.

You seem to think I'm underestimating what Azure's name adds to OpenAI: I fully understand how bureaucratic organizations work off vibes under the guise of name recognition and my point is I simply have no respect for it.

If you genuinely care about customer data, then the value of being able to sue MS instead of OpenAI is moot. You also probably aren't going to use a service that shouts from the roof tops about not using your data then quietly keeps it for 30 days unless you manually opt-out. You probably don't use some model with unsolved copyright/PII questions. And a million other unknowns

> The part that matters is operations - both technical and legal aspects - and this is where Microsoft and OpenAI are independent and have different offerings.

You might want to check OpenAI's subprocessor list if you think that they're not the same technically...

https://platform.openai.com/subprocessors/openai-subprocesso...

And Azure's subprocessor list is a superset of that list, not a subset.

Re: How is ChatGPT's behavior changing over time?

#184
post #43

Earlier quoted context omitted.

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

It’s not about the results, it’s about its ability to “reason”. Math is about as close to pure reasoning we get so I don’t get the pessimism. If it is bad at math and can’t be taught, then you have a fundamental problem. It’s a matter of time before this limit gets hit in other domains.

Your thinking is very emotional and shows your inability to understand that "conversational output" is basically an accidental side-effect of a complex tool that just replicates our speech without understanding what it is saying.

Through continuous use, I have found that it does not "reason". That doesn't mean it's not valuable in many ways, and I have found it to be very helpful in a multitude of diverse applications, including helping me reflect on my own life through my own interpretations of its output. It's also a great interface for JSTOR, wikipedia, and basically any language learning.

I'm having a hard time making the jump from "this must be a calculator" to "this must be a philosopher" to be useful. When did we ever have those requirements for a tool?

This tool is just not made for math. Most of its logic processing abilities seem to surpass mine if I am only given 5 minutes to understand a problem. If you understand the tool, you will get the most out of it. Stop anthropomorphizing it, and stop pretending that it can't generate both highly beneficial or highly harmful content simply because it doesn't have a soul/d*ck or whatever.

Re: How is ChatGPT's behavior changing over time?

#185
post #172

Earlier quoted context omitted.

The point of doing $non-llm-optimal-thing on an llm is the hope that it let's you skip on formal syntaxes, which are mentally taxing. It's far easier even for an expert to communicate what they want in natural language than it is in a formal syntax for all but the most trivial things. It should be a goal of these tools to do this correctly.

> the hope that it let's you skip on formal syntaxes, which are mentally taxing agree completely, but the LLM should be focused, then, strictly on formulating an "execution plan" of sorts and handing that off, not on performing math itself. In other words, when asked "if i have 349 blueberries and one blueberry turns into a cherry per hour, how many of each fruit will I have in 93478 minutes?" it shouldn't be doing t…

This is my second ChatGPT comment so I hate to sound like an evangelist, but it basically does this, reproducibly:

You describe a complex relationship between several nodes(people, cities, etc), and then ask ChatGPT to draw the relationship as a graph data structure. It will create a formula that Mathematica can render, and then send the formula to Mathematica before presenting an ascii drawing. Usually. Sometimes it just complains that it can't draw and explains what the graph looks like to you in a written formal/human-readable syntax.

In other words, yes it just summarizes the problem, converts it to a formal syntax, and sends it off to some other tool.

Re: How is ChatGPT's behavior changing over time?

#186
post #182

Earlier quoted context omitted.

> Have you seen the 25 messages/3 hours limitation for GPT-4? If can't tell if that's about the API or the web app, I don't think you're familiar enough with the subject to speak on it.

I don't need to be familiar with it to see that you're not being genuine in your interpretation of peoples' comments and in the way you're responding to people in this thread. Case in point: your reply to me.

What was I supposed to reply to your factless baseless anecdote with?

Millions of dollars in spend from products predicated on the API not randomly changing?

Re: How is ChatGPT's behavior changing over time?

#187
post #166
post #70

Just yesterday I gave ChatGPT a summarization task and it performed horribly. I even tried multiple times and got the identical answer. Then I gave the identical prompt to gpt-3.5-turbo via the API and I immediately got the expected good answer.

Which gpt-3.5-turbo API? Legacy completion or chat?

Turbo is always chat.
Post reply on HN