It seems to write in the generic "style" of GPT, instead of in the style I would recognise as a HN poster. Is that because of something baked into how the training process works? It lacks a sort of casualness or air of superiority ;)
Show HN: AskHN
11–20 of 139 posts
Re: Show HN: AskHN
#12I love this! I used to append "reddit" to my Google search queries to get best results, but the quality of dialog over there has really dropped in recent years. These days I've switched to appending "hackernews", but this is even better.
Re: Show HN: AskHN
#13Is there a way to opt out of one's comments being used for this?
If you're willing to pay for the retraining? ;)
Re: Show HN: AskHN
#14I'm a little surprised that Hacker News comments weren't already in the GPT-3 training set. I just assumed that OpenAI had vacuumed up most of the web already.
I am guessing they already were? But this is 100% pure, concentrated HN not contaminated with nonsense from the rest of the web :)
If it's really trained exclusively off of HN comments, I expect most of the bot's responses will evade the actual question but spend several paragraphs debating the factual specifics of every possible related tangential point, followed by an thinly-veiled insult questioning the user's true motivations.
Re: Show HN: AskHN
#15Can anyone help me to guide some tutorials using GPT-3 model on a certain dataset. I am a Python programmer.
Re: Show HN: AskHN
#16Is there a way to opt out of one's comments being used for this?
Yes, don’t post on online forums.
Re: Show HN: AskHN
#17"He only went and did it... " !
Re: Show HN: AskHN
#18Am I correct in understanding that this doesn't actually generate answers based on HN, but instead finds semantically-near comments and sends them verbatim to GPT to summarize? Seems like a good enough hack, though I'd love a detailed writeup of how to actually specialize an existing LLM with additional training data (like HN).
Re: Show HN: AskHN
#19> I trained on a corpus of over 6.5 million Hacker News comments How long did it take to scrape them and train the "corpus" on this content?
That was mentioned in the article
In the « Ingesting and filtering HN corpus » … 30min …
Re: Show HN: AskHN
#20Earlier quoted context omitted.
I am guessing they already were? But this is 100% pure, concentrated HN not contaminated with nonsense from the rest of the web :)
I have to assume that targeted/curated LLM training sets will have a tendency to be less accurate than very general, just by the very nature of how they work. (edited for clarity)
I know it's not quite analogous, but I fine-tuned GPT-3 on a small (200 examples) data set and it performed extremely poorly compared to the untrained version.
This surprised me, I thought it wouldn't do much better, but I wasn't expecting that specializing it on my target data would reduce performance! I had fewer examples than the minimum OpenAI recommends, so maybe it was a case of overfitting or something like that.