Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FrontierMath was funded by OpenAI

lesswrong.com

Post reply on HN
Print view View original

151–160 of 212 posts

Re: FrontierMath was funded by OpenAI

#151

Post by echoangle »

alphan0n wrote: post #88
lmm wrote: post #76

Earlier quoted context omitted.

> the copyright holder’s tort would be with the source of the infringing distribution, not the people who read the material. Someone who just reads the material doesn't infringe. But someone who copies it, or prepares works that are derivative of it (which can happen even if they don't copy a single word or phrase literally), does. > would I then owe the authors of the books I learned from a fee to apply that knowled…

Right, but the onus of responsibility being on the end user publishing the song or creative work in violation of copyright, not the text editor, word processor, musical notation software, etc, correct? A text prediction tool isn’t a person, the data it is trained on is irrelevant to the copyright infringement perpetrated by the end user. They should perform due diligence to prevent liability.

How is the end user the one doing the infringement though? If I chat with ChatGPT and tell it „give me the first chapter of book XYZ“ and it gives me the text of the first chapter, OpenAI is distributing a copyrighted work without permission.

Re: FrontierMath was funded by OpenAI

#152

Post by echoangle »

teleforce wrote: post #99
agnosticmantis wrote: post #13

“… we have a verbal agreement that these materials will not be used in model training” Ha ha ha. Even written agreements are routinely violated as long as the potential upside > downside, and all you have is verbal agreement? And you didn’t disclose this? At the time o3 was released I wrote “this is so impressive that it brings out the pessimist in me”[0], thinking perhaps they were routing API calls to human workers…

>perhaps they were routing API calls to human workers Honest question, did they?

How would that even work? Aren’t the responses to the API equally fast as the Web interface? Can any human write a response with the speed of an LLM?

Re: FrontierMath was funded by OpenAI

#153

Post by aithrowawaycomm »

numba888 wrote: post #118
rrr_oh_man wrote: post #29

Earlier quoted context omitted.

> if they used it in training it should be 100% hit. Not necessarily, no. A statistical model will attempt to minimise overall loss, generally speaking. If it gets 100% accuracy on the training data it's usually an overfit. (Hugging the data points too tightly, thereby failing to predict real life cases)

you are mostly right. but seeing almost perfectly reconstructed images from training set it's obvious model -can- memorize samples. in this case it would reproduce the answers too close to the original to be just 'accidental'. should be easy to test. My guess samples could be used to find good enough stopping point for o1, o3 models. which is hardcoded.

The subtlety here is that an almost-memorized picture of a lady is the same picture with a few artifacts, and an almost-memorized NYT article is the same article with a few words changed, but an almost-memorized computation or proof is likely to be plain wrong. So even if OpenAI's benchmark was data contamination (as I suspect) it still says something about o1's abilities to execute a given problem-solving strategy without confabulating. It's just not what OpenAI wants you to think: much closer to Mathematica than an actual mathematician.

Re: FrontierMath was funded by OpenAI

#154

Post by YeGoblynQueenne »

echoangle wrote: post #152
teleforce wrote: post #99

Earlier quoted context omitted.

>perhaps they were routing API calls to human workers Honest question, did they?

How would that even work? Aren’t the responses to the API equally fast as the Web interface? Can any human write a response with the speed of an LLM?

No but a human can solve a problem that an LLM can't solve and then an LLM can generate a response to the original prompt including the solution found by the human.

Re: FrontierMath was funded by OpenAI

#155

Post by [deleted] »

ripped_britches wrote: post #82

Do people actually think OpenAI is gaming benchmarks? I know they have lost trust and credibility, especially on HN. But this is a company with a giant revenue opportunity to sell products that work. What works for enterprise is very different from “does it beat this benchmark”. No matter how nefarious you think sama is, everything points to “build intelligence as rapidly as possible” rather than “spin our wheels mes…

[deleted]

Re: FrontierMath was funded by OpenAI

#156

Post by bhouston »

Rebuff5007 wrote: post #33
asadotzler wrote: post #18

Earlier quoted context omitted.

OpenAI doesn't respect copyright so why would they let a verbal agreement get in the way of billion$

Can somehow explain to me how they can simply not respect copyright and get away with it? Also is this a uniquely open-ai problem, or also true of the other llm makers?

> Can somehow explain to me how they can simply not respect copyright and get away with it? Also is this a uniquely open-ai problem, or also true of the other llm makers?

Uber showed the way. They initially operated illegally in many cities but moved so quickly as to capture the market and then they would tell the city that they need to be worked with because people love their service.

https://www.theguardian.com/news/2022/jul/10/uber-files-leak...

Re: FrontierMath was funded by OpenAI

#157

Post by Xcelerate »

asadotzler wrote: post #18
agnosticmantis wrote: post #13

“… we have a verbal agreement that these materials will not be used in model training” Ha ha ha. Even written agreements are routinely violated as long as the potential upside > downside, and all you have is verbal agreement? And you didn’t disclose this? At the time o3 was released I wrote “this is so impressive that it brings out the pessimist in me”[0], thinking perhaps they were routing API calls to human workers…

OpenAI doesn't respect copyright so why would they let a verbal agreement get in the way of billion$

Why do HN commenters want OpenAI to be considered in violation of copyright here? Ok, so imagine you get your wish. Now all the big tech companies enter into billion dollar contracts with each other along with more traditional companies to get access to training data. So we close off the possibility of open development of AI even further. Every tech company with user-generated content over the last 20 years or so is sitting on a treasure trove now.

I’d prefer we go the other direction where something like archive.org archives all publicly accessible content and the government manages this, keeps it up-to-date, and gives cheap access to all of the data to anyone on request. That’s much more “democratizing” than further locking down training data to big companies.

Re: FrontierMath was funded by OpenAI

#158

Post by alphan0n »

echoangle wrote: post #151
alphan0n wrote: post #88

Earlier quoted context omitted.

Right, but the onus of responsibility being on the end user publishing the song or creative work in violation of copyright, not the text editor, word processor, musical notation software, etc, correct? A text prediction tool isn’t a person, the data it is trained on is irrelevant to the copyright infringement perpetrated by the end user. They should perform due diligence to prevent liability.

How is the end user the one doing the infringement though? If I chat with ChatGPT and tell it „give me the first chapter of book XYZ“ and it gives me the text of the first chapter, OpenAI is distributing a copyrighted work without permission.

Can you do that though? Just ask ChatGPT to give you the first chapter of a book and it gives it to you?

Re: FrontierMath was funded by OpenAI

#159

Post by echoangle »

alphan0n wrote: post #158
echoangle wrote: post #151

Earlier quoted context omitted.

How is the end user the one doing the infringement though? If I chat with ChatGPT and tell it „give me the first chapter of book XYZ“ and it gives me the text of the first chapter, OpenAI is distributing a copyrighted work without permission.

Can you do that though? Just ask ChatGPT to give you the first chapter of a book and it gives it to you?

https://news.ycombinator.com/item?id=42767775

Not a book chapter specifically but this could already be considered copyright infringement, I think.

Re: FrontierMath was funded by OpenAI

#160

Post by alphan0n »

lmm wrote: post #107
alphan0n wrote: post #88

Earlier quoted context omitted.

Right, but the onus of responsibility being on the end user publishing the song or creative work in violation of copyright, not the text editor, word processor, musical notation software, etc, correct? A text prediction tool isn’t a person, the data it is trained on is irrelevant to the copyright infringement perpetrated by the end user. They should perform due diligence to prevent liability.

> A text prediction tool isn’t a person, the data it is trained on is irrelevant to the copyright infringement perpetrated by the end user. They should perform due diligence to prevent liability. Huh what? If a program "predicts" some data that is a derivative work of some copyrighted work (that the end user did not input), then ipso facto the tool itself is a derivative work of that copyrighted work, and illegal to…

If the output of a mathematical model trained on an aggregate of knowledge that contains copyrighted material is derivative and infringing, then ipso facto, all works since the inception of copyright are derivative and infringing.

You learned English, math, social studies, science, business, engineering, humanities, from a McGraw Hill textbook? Sorry, all creative works you’ve produced are derivative of your educational materials copyrighted by the authors and publisher.

Post reply on HN