How An AI math breakthrough ignited a controversy

You know, everyone's focused on the math and the human drama aspects, but to me the real story should be about this accusation that they somehow read the stream of data these guys were generating and then reacted to it at all on a human level is pure poison for their hopes for enterprises to be using their software.

Even if these two mathematicians were using the AI services under a clause that nominally allows OpenAI to train on their data, it is still another level above for there to exist some pathway within OpenAI to know that someone is doing something very valuable and important with their AI and to swoop in to try to steal the value of that work. It doesn't matter exactly what that pathway is, just that it exists is a big deal. Today it's a Millennium Prize, but tomorrow, is it the next hot new product category? Is it the patent someone is working on? Does OpenAI constitutionally believe that you may be using their product but whatever you produce really belongs to them and you are just borrowing it? Can we trust contracts that promise that they don't believe that if this is how they act?

Even people who are consciously aware that their sessions are being used to train the AI don't expect the AI companies to be in some way scanning the stream of what AI is working on for high-value propositions to snipe. That changes the relationship between these companies and enterprises, and anyone who thinks they may be doing this sort of work, deeply and fundamentally.

It isn't just that OpenAI is now something we should be suspicious about for math. It's all high-value science. Is there anyone trying to figure out how to develop a next-generation AI architecture using OpenAI? Better stop, OpenAI could see it and scoop you out of who knows how much value.

There is an old story attached to many historical figures that goes:

    Churchill: Madam, would you sleep with me for a million pounds?

    Woman: My goodness… well, I suppose I would.

    Churchill: Would you sleep with me for a pound?

    Woman: Certainly not! What kind of woman do you think I am?

    Churchill: Madam, we’ve already established that. Now we are 
    just haggling over the price.

If this accusation is true, which I'm not completely confident about, but if it is true, that is the situation we would be in... we would have established that OpenAI is willing to steal from its customers, the only question is where the line is now, and where the line might be in the future.

Honestly the human drama, while the compelling story, is in monetary terms orders of magnitude less important than the question of whether OpenAI has a mechanism to steal ideas from its customers. Has this happened before and they just successfully covered it up? Or maybe the people stolen from thought it was innocent and just a coincidence that they were scooped? If it established that there is a price, I have a lot of questions about that price.

You can’t do anything novel with these models from scratch and let it fly. I’ve observed something over the past few months

Work on something novel -> llm is kinda useless and low value-add -> Keep at it and in the process feed it more information -> keep doing this periodically -> a few months go by and you realise the model outputs are almost like-for-like regurgitations of what was inputted in some prior period.

Once it’s accumulated new info can it produce something automated that is somewhat useful? Sure.

But by itself - absolutely not.

I clearly see humans will be needed - the best ones that is. For ‘rote work’ and stuff that is not IP sensitive firms will be ok with employees putting that as inputs into models.

But I’d wary about trusting the labs. They will push the letter of the law to the max.

Personally I’ve stopped doing anything novel with these models. If I do use a model on something adjacent but not totally novel I have to craft the inputs in a strategic way not to give much away.

I’d wager firms will soon realise this and that growth rate of revenues of the frontier labs will become questionable. The economic cost that firms have brought out thus far is only financial. There’s a whole bunch of other costs people aren’t talking about.

> This right here, or at least the thought of this, is why in the not so far future, businesses can't (won't?) be using these LLMs services.

Or they can simply turn off the setting that allows OpenAI to use their chats to improve the model. Or they can use the API where it is off by default.. If the advantage of using OpenAI models will be significant enough for the business in question, these are the options.

If that's true, either they both deserve credit or neither of them do. You don't just average 0 and 2 and decide that = 1 person deserves credit.

Even in the past a lot of “aristocrats” had to earn a living. Among scholars, many worked as teachers or had sinecures that paid the bills, or held clergy posts with minimal responsibilities.

Aristocrats were a dime a dozen.