Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New GPT-3 model: text-DaVinci-003

beta.openai.com

Post reply on HN
Print view View original

31–40 of 216 posts

Re: New GPT-3 model: text-DaVinci-003

#31

Post by joe_the_user »

ctoth wrote: post #3

This one seems weirdly more verbose, in my brief testing. Definitely uses more tokens on average than 002 when answering. The writing style seems oddly-pretentious and yet somehow uninformed, perhaps there were too many copies of HN in the dataset? I kid, I kid.

After the failure of FB's Galactica model, I wonder whether LLM are starting to hit diminishing return. I've read of several ways diminishing returns could happen:

* Not enough training data - you've used up the Internet (even a percentage of the Internet might be as much as is usable by clever brute force).

* Not enough compute time to fully train (we're not close to that)

* The model covers such a large area that testing is impossible

One thing I'd speculate about is perhaps the more different subjects the program is expected to combine, the more it learns to spout plausible bullshit and clever quips, since for clever humans, that how they relate to stuff they don't know. So "pretentious but uninformed" might be a sign.

Re: New GPT-3 model: text-DaVinci-003

#32

Post by blindot »

oldgradstudent wrote: post #17
zoba wrote: post #11

Oh wow! It can rhyme! Write a rhyming poem comparing love to a beautiful flower: A flower so beautiful and fair Love can be found everywhere Its petals so vibrant and bright A love that will last day and night The beauty of love, like a flower It's a beauty that will never cower A feeling of joy that will last Like a flower, love stands fast The beauty of love is so true Like a blooming flower for me and for you No m…

Have it compose a poem — a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter S!!

Sleek strands of sadness, soon severed and shorn;

Solace sought in scissors, slicing and scorned.

Sublime sorrow, set sorrowfully still;

Stoic heart to surrender, so swiftly and shrill.

Secrets spilled in soft, sorrowful swish

Sorrowful sounds, soon silenced with snip.

Re: New GPT-3 model: text-DaVinci-003

#33

Post by nathell »

tspike wrote: post #24
oldgradstudent wrote: post #17

Earlier quoted context omitted.

Have it compose a poem — a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter S!!

Stately she stands, so solemnly still She sharpens the scissors, and sharpens her skill She'll snip and she'll shape, and her fingers so swift Will swiftly restore a style so befitting She slices the strands with a silent salute To sacredly sanctify the silhouette

My attempt with text-davinci-002: https://twitter.com/nathell/status/1551557069043650560/photo... (a corresponding Polish version can be found in the thread as well)

Re: New GPT-3 model: text-DaVinci-003

#34

Post by robbomacrae »

I've been trying out various models for summarization and I have to say, on first glance, this davinci-003 doesn't impress compared to davinci-002 or facebook/bart-large-cnn. If you throw in "summarize the following text:" and copy in the contents of https://en.wikipedia.org/wiki/Hacker_News you get the following (temperature 0.0 max tokens 100)[0]:

davinci-003

Hacker News is a social news website focusing on computer science and entrepreneurship, run by Y Combinator. It was created by Paul Graham in 2007 and is moderated by Daniel Gackle. It is intended to recreate a community similar to the early days of Reddit, but with a karma system that requires users to accumulate 501 points before they can downvote content. It also has automated flame and spam detectors and active human moderation, as well as a "voting rings" detector. It

davinci-002

Hacker News is a social news website focusing on computer science and entrepreneurship. It is run by the investment fund and startup incubator Y Combinator. The site was created by Paul Graham in February 2007. Initially called Startup News or occasionally News.YC., it became known by its current name on August 14, 2007.

facebook/bart-large-cnn

Hacker News is a social news website focusing on computer science and entrepreneurship. It is run by the investment fund and startup incubator Y Combinator. In general, content that can be submitted is defined as anything that gratifies one's intellectual curiosity. The site has a proactive attitude in moderating content, including automated flame and spam detectors and active human moderation.

Curious if anyone knows any good alternatives or tricks to improve the summaries? For now I'm using the bart one as it has the advantage of being downloadable from huggingface so you can run it yourself [1].

[0]: https://beta.openai.com/playground?model=text-davinci-003

[1]: https://huggingface.co/facebook/bart-large-cnn

Re: New GPT-3 model: text-DaVinci-003

#35

Post by Bilal_io »

rpjt wrote: post #26
nl wrote: post #22

Earlier quoted context omitted.

This is a lot more wide-spread than "some kids on Reddit". Maybe 30% of the kids in my son's class are using this or related tools.

Any ideas on how to solve this issue of kids cheating with GPT3 essays?

It's the parents' responsibility. No one outside the household can do anything about it imo.

Using AI to write will cause the same issues as:

- phones, some people don't try to remember directions, phone number or addresses

- calculators, some people cannot do easy math

- computers, some people cannot write with a pen, cannot spell without spellcheck

Re: New GPT-3 model: text-DaVinci-003

#36

Post by TaylorAlexander »

joe_the_user wrote: post #31
ctoth wrote: post #3

This one seems weirdly more verbose, in my brief testing. Definitely uses more tokens on average than 002 when answering. The writing style seems oddly-pretentious and yet somehow uninformed, perhaps there were too many copies of HN in the dataset? I kid, I kid.

After the failure of FB's Galactica model, I wonder whether LLM are starting to hit diminishing return. I've read of several ways diminishing returns could happen: * Not enough training data - you've used up the Internet (even a percentage of the Internet might be as much as is usable by clever brute force). * Not enough compute time to fully train (we're not close to that) * The model covers such a large area that t…

I still need to read past the first page, but I was reminded this morning of the 2021 paper “On the dangers of stochastic parrots: can language models be too big?” which discusses the harms and diminishing returns of moving to ever larger language models.

I had to rush out the door today after seeing this paper come up so I can’t speak much to its content right now. But if anyone wants to read it and reflect here I’d like to hear it.

https://dl.acm.org/doi/10.1145/3442188.3445922

Re: New GPT-3 model: text-DaVinci-003

#37

Post by typpo »

In case it interests anyone, I built a document editor + GPT and updated it to use text-davinci-003: https://github.com/typpo/arkose/

If you want to try GPT-3 but don't have an OpenAI API key, I've set up a quick demo here until I hit my billing cap (normally users would supply their own API key): https://arkose.pages.dev/

Re: New GPT-3 model: text-DaVinci-003

#38

Post by version_five »

joe_the_user wrote: post #31
ctoth wrote: post #3

This one seems weirdly more verbose, in my brief testing. Definitely uses more tokens on average than 002 when answering. The writing style seems oddly-pretentious and yet somehow uninformed, perhaps there were too many copies of HN in the dataset? I kid, I kid.

After the failure of FB's Galactica model, I wonder whether LLM are starting to hit diminishing return. I've read of several ways diminishing returns could happen: * Not enough training data - you've used up the Internet (even a percentage of the Internet might be as much as is usable by clever brute force). * Not enough compute time to fully train (we're not close to that) * The model covers such a large area that t…

It's also likely we're just seeing the limits of what the tech can do. It's not actually "intelligent" it's effectively a cool trick, and at some point the gap between what the trick can do and what a "perfect" AI can do becomes obvious. It's really not that different from Eliza being cool for a few minutes before being obviously found wanting. It just takes longer for the shine to wear off as we get to a better technology. Ditto for self driving cars - a steep climb to a 90% solution, then an impossible chasm to cross to something that actually handles the edge cases.

Re: New GPT-3 model: text-DaVinci-003

#39

Post by simonw »

rpjt wrote: post #26
nl wrote: post #22

Earlier quoted context omitted.

This is a lot more wide-spread than "some kids on Reddit". Maybe 30% of the kids in my son's class are using this or related tools.

Any ideas on how to solve this issue of kids cheating with GPT3 essays?

I honestly don't think it's possible to solve, other than by increasing the amount of evaluation that's done in locked down conditions.

I cannot imagine a detection mechanism that could not itself be defeated by some tweaks to the prompts being used to generate the essays.

It's effectively the same problem as "prove that this kid didn't get their friend/cousin to write the essay for them".

Re: New GPT-3 model: text-DaVinci-003

#40

Post by brookst »

_jal wrote: post #18
faizshah wrote: post #12

Earlier quoted context omitted.

It can't have been trained on HN, I tried generating my comment for this thread and it would only output positive marketing speak instead of "This name seems to collide with the artist Da Vinci."

I knew it wasn't trained on HN when it didn't immediately try to correct my word choice.

I think you mean "refine" rather than "correct".
Post reply on HN