Show HN: AskHN • Hacker News Board

Re: Show HN: AskHN

#61

Post by bayan1234 » Wed, Feb 22, 2023, 7:07 PM UTC

la64710 wrote: post #34

Is there any LLM model that can be self hosted and fed a corpus of data to ingest for question answering? The part I find difficult is how to feed (not train) the open LLM models with entire dataset not available to public?

The hack to solve this is to embed each paragraph in your large corpus. Find paragraphs most similar to the user query using embeddings. Put the paragraphs and the raw user query into a prompt template. Send the final generated prompt to gpt3.

This actually works surprisingly well.

Check out the OpenAI cookbook for examples.

Re: Show HN: AskHN

#62

Post by reacharavindh » Wed, Feb 22, 2023, 7:08 PM UTC

have_faith wrote: post #11

It seems to write in the generic "style" of GPT, instead of in the style I would recognise as a HN poster. Is that because of something baked into how the training process works? It lacks a sort of casualness or air of superiority ;)

Now that you say it, it will train itself for it while it learns from your comments ;-)

Re: Show HN: AskHN

#63

Post by osigurdson » Wed, Feb 22, 2023, 7:08 PM UTC

I have an experiment that uses the embeddings to visualize clusterings of HN comments (using tsne). Not super useful but interesting to view the comments in 3D and seeing how similar ones cluster together into mostly relevant themes.

Re: Show HN: AskHN

#64

Post by tomr75 » Wed, Feb 22, 2023, 7:13 PM UTC

could you do this for medical journal articles?

Re: Show HN: AskHN

#65

Post by jerpint » Wed, Feb 22, 2023, 7:15 PM UTC

Nice! We built something very similar recently, it is more like "Ask your documentation" but very similar implementations otherwise.

See a demo on the huggingface transformers documentation: https://huggingface.co/spaces/jerpint/buster

code: https://github.com/jerpint/buster

Re: Show HN: AskHN

#66

Post by jerpint » Wed, Feb 22, 2023, 7:17 PM UTC

georgelyon wrote: post #18

Am I correct in understanding that this doesn't actually generate answers based on HN, but instead finds semantically-near comments and sends them verbatim to GPT to summarize? Seems like a good enough hack, though I'd love a detailed writeup of how to actually specialize an existing LLM with additional training data (like HN).

The summary itself is still generated, but has all the context to do summarization in the prompt.

It's very difficult to otherwise finetune existing LLMs. GPT itself is closed-sourced, and doesn't allow for finetuning (except via an opaque API and with limited amounts of data). Other open models are either very difficult to load in memory and/or simply not as expressive as GPT

Re: Show HN: AskHN

#67

Post by vidarh » Wed, Feb 22, 2023, 7:18 PM UTC

bradwood wrote: post #48

britzkopf wrote: post #35

Earlier quoted context omitted.

To truly capture the HN experience, the user should provide a parameter for the number of "well actually"'s they want to receive. So initial response should demonstrate clear expertise and make a great concise point in response to question, and then start the cascade of silly nitpicking.

I think you'll find "I think you'll find" trumps "well actually". ;)

I wish the results were reversed, so I could "well actually" your comment, but 'site:news.ycombinator.com "well actually"' gives ca. 4k results in Google and 'site:news.ycombinator.com "I think you'll find"' gives close to 17k results, so you appear to be right.

Re: Show HN: AskHN

#68

Post by gnicholas » Wed, Feb 22, 2023, 7:19 PM UTC

Love that it includes sources — this makes it much more valuable because you can tell if it's giving useful information or just blowing smoke.

Re: Show HN: AskHN

#69

Post by triyambakam » Wed, Feb 22, 2023, 7:20 PM UTC

Minor49er wrote: post #42

egypturnash wrote: post #9

Is there a way to opt out of one's comments being used for this?

Just post a healthy amount of random nonsense along with any of your actual posts to dilute the effects Banana Sebastian housewares fly swimmingly under terrestrial Zruodroru'th Memphis Steve Jobs archipelagos

> Banana Sebastian housewares fly swimmingly under terrestrial Zruodroru'th Memphis Steve Jobs archipelagos

It's actually more likely to require a bathtub to increase the volume of the reticulated lorries, so I really don't think a farmer's market is the ideal place.

Re: Show HN: AskHN

#70

Post by stnmtn » Wed, Feb 22, 2023, 7:23 PM UTC

btbuildem wrote: post #53

agolio wrote: post #46

Earlier quoted context omitted.

Scroll a bit further down and you will see > 7. Put top matching content into a prompt and ask GPT-3 to summarize > 8. Return summary along with direct links to comments back to Discord user

Ah got it. Perhaps they should edit the intro then, it's misleading.

I agree, that language could be very improved. This is not a GPT-like LLM whose training corpus is HN comments, which I found to be an extremely interesting idea. Instead, it looks like it's finds relevant HN threads and tells GPT-3 (the existing model) to summarize them.

To be clear, I think this is still very cool, just misleading.