Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Answer Overflow – Indexing Discord content into the web

answeroverflow.com

Post reply on HN
Print view View original

81–90 of 99 posts

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#81

Post by sprremix »

leobg wrote: post #69

Indexing Discord is going to be tough. The reason is that context is all over the place: Question in one message. Then two unrelated messages. Then a partial answer by somebody. And so on. It’s even worse than indexing a PDF. Just breaking stuff into paragraphs and generating embeddings isn’t going to cut it.

I imagine this will only work (and only index) threads. So the context can be gathered from the thread title/body and underlying messages reflect the discussion.

Some communities I'm in have #support channels which only support threads. So you create a thread, add a title and a body message and people can reply to your thread by clicking on it. There's no way to post individual messages; only comments in threads.

Thread overview: https://i.imgur.com/jfvrRtG.png

Opening a thread: https://i.imgur.com/pqGrARI.png

This solves your context problem. Still not sure if this is the right direction we want to go in. This just proves to me that Discord is not right tool for the problem at hand.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#82

Post by andybak »

lohnjemon wrote: post #57
michaelmior wrote: post #24

Earlier quoted context omitted.

I would argue it makes no sense to index the vast majority of content without good search . If your search is good enough, you can index everything and then surface only the good stuff at query time.

Indexing trillions of messages is outrageous.

That's why "archive to the web and let Google handle it" is a good solution.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#83

Post by andybak »

Rhys - are you sure the consent functionality is working? I'm seeing indexed posts by users who are in a time zone that makes it very unlikely they have consented in the last hour or so.

The one user whom I contacted said they had never clicked the green consent button.

EDIT - turns out those posts were only visible to me when I was logged in to both sites (which makes sense).

It wasn't obvious this was the case and checking incognito shows things correctly.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#84

Post by andybak »

freediver wrote: post #73

There are several issues with surfacing search results from Discord as mentioned before in the thread, and even if all of them are resolved the biggest one remains relevance. Unless a general purpose web search engine introduces a special Discord 'tab', like Images/News/Videos already exist, there is no way for a search engine to assign relevance to anything said on Discord because there is no authority or link graph…

I'm mostly interested in surfacing content on pretty specific topics with clear keywords.

But also either answeroverflow.com will gain some domain authority over time, or the communities will be hosted on domains that already have some.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#87

Post by rhyssullivan1 »

andybak wrote: post #83

Rhys - are you sure the consent functionality is working? I'm seeing indexed posts by users who are in a time zone that makes it very unlikely they have consented in the last hour or so. The one user whom I contacted said they had never clicked the green consent button. EDIT - turns out those posts were only visible to me when I was logged in to both sites (which makes sense). It wasn't obvious this was the case and…

Glad we got this resolved and it was all working properly, the site does need to do more to make it clearer when viewing a private message while signed in added it to the backlog sorry about that!

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#88

Post by Geezus_42 »

TeMPOraL wrote: post #77
chillfox wrote: post #61

Earlier quoted context omitted.

Not everything should be preserved forever. It's actually really nice to be able to talk online and not have it form a permanent record that can be instantly referenced by anyone.

Correct. However, some things should be preserved forever and accessible. It's really nice to be able to put a piece of scientific jargon, or a text of an error message, into a search box, and get back links and references to material you can peruse at your own pace and discretion - as opposed to having to join a closed community and keep asking people, hoping someone who knows the answer and is willing to help spots…

Also, there's the perpetual issue of people who just want to be assholes to other people on line and not actually help. This has existed in internet based chat platforms and is why they will never be better than documentation.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#89

Post by Geezus_42 »

Geezus_42 wrote: post #88
TeMPOraL wrote: post #77

Earlier quoted context omitted.

Correct. However, some things should be preserved forever and accessible. It's really nice to be able to put a piece of scientific jargon, or a text of an error message, into a search box, and get back links and references to material you can peruse at your own pace and discretion - as opposed to having to join a closed community and keep asking people, hoping someone who knows the answer and is willing to help spots…

Also, there's the perpetual issue of people who just want to be assholes to other people on line and not actually help. This has existed in internet based chat platforms and is why they will never be better than documentation.

> Also, there's the perpetual issue of people who just want to be assholes to other people on line and not actually help. This has [always] existed in internet based chat platforms and is why they will never be better than documentation.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#90

Post by esafak »

TeMPOraL wrote: post #76
esafak wrote: post #58

Earlier quoted context omitted.

The solution to the growth problem is incubating complementary products.

For regular growth, maybe, but not for investment-fueled hyper-growth.

Every big, lasting company has a product portfolio: Google, Amazon, Meta, Microsoft... A single product will leave you vulnerable to disruption, and the aforementioned problem.
Post reply on HN