Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: An AI-Generated Encyclopedia

mycyclopedia.co

Post reply on HN
Print view View original

11–20 of 54 posts

Re: Show HN: An AI-Generated Encyclopedia

#11

Post by nickpsecurity »

mahouk wrote: post #10

Update: Sorry guys, but it seems the server has crashed due to a sudden influx of traffic, and I'm attending a funeral service at the moment so I don't have access to my laptop. Will try to get the site back up asap!

Sorry to hear you lost someone close to you. I’ve prayed that Jesus Christ provides comfort to you and others involved. Do what you need to do and we’ll look at the site whenever it’s back up. No rush.

Edit: Forgot to add about the site crashing on heavy traffic. You might want to consider a CDN. Cloudflare is No 1 with a free option. StackPath was great but just shut down their CDN. I’m trying BunnyCDN now since it’s pennies per GB.

Re: Show HN: An AI-Generated Encyclopedia

#13

Post by visarga »

We need an AI-generated encyclopedia - not for us, but for AI. It should have a trillion articles covering all known entities and concepts, written using RAG over the web. Controversial topics should report the controversy or the distribution of opinions. We can put this big synthetic text corpus in the training set of future models.

Why? Because AI needs long form, in-depth texts to train on, and the web doesn't provide it in sufficient quantity and quality. We need chain-of-thought to capture relations between concepts in explicit language. Synthetic data makes it possible to have balanced coverage of topics and combinatorial coverage of skills to improve reasoning. It's also better from a copyright stand point to train models on synthetic data.

Re: Show HN: An AI-Generated Encyclopedia

#14

Post by swatcoder »

ShamelessC wrote: post #12

Perhaps best not to assume everyone on the planet worships Jesus Christ. edit: It's really not a big deal though. Sorry for stirring a controversy unnecessarily.

They didn't. It's just their way of showing sympathy.

The OP may not find the prayer itself practical, as indeed many on the planet mightn't, but the expression doesn't assume so. Common secular expressions like "Sending good thoughts" or "I feel you" are just immaterial and seemingly ineffectual but similarly get across the point that someone has slowed down long enough to show care.

Re: Show HN: An AI-Generated Encyclopedia

#16

Post by [deleted] »

visarga wrote: post #13

We need an AI-generated encyclopedia - not for us, but for AI. It should have a trillion articles covering all known entities and concepts, written using RAG over the web. Controversial topics should report the controversy or the distribution of opinions. We can put this big synthetic text corpus in the training set of future models. Why? Because AI needs long form, in-depth texts to train on, and the web doesn't pro…

[deleted]

Re: Show HN: An AI-Generated Encyclopedia

#17

Post by jfk13 »

visarga wrote: post #13

We need an AI-generated encyclopedia - not for us, but for AI. It should have a trillion articles covering all known entities and concepts, written using RAG over the web. Controversial topics should report the controversy or the distribution of opinions. We can put this big synthetic text corpus in the training set of future models. Why? Because AI needs long form, in-depth texts to train on, and the web doesn't pro…

> the web doesn't provide it in sufficient quantity and quality

Do you seriously think that "an AI-generated encyclopedia" would provide a better-quality training set? What would the "AI generator's" articles be derived from?

Re: Show HN: An AI-Generated Encyclopedia

#19

Post by ShamelessC »

swatcoder wrote: post #14
ShamelessC wrote: post #12

Perhaps best not to assume everyone on the planet worships Jesus Christ. edit: It's really not a big deal though. Sorry for stirring a controversy unnecessarily.

They didn't. It's just their way of showing sympathy. The OP may not find the prayer itself practical, as indeed many on the planet mightn't, but the expression doesn't assume so. Common secular expressions like "Sending good thoughts" or "I feel you" are just immaterial and seemingly ineffectual but similarly get across the point that someone has slowed down long enough to show care.

It's really not a big deal. I understand what they meant. I just think the secular variety of expression is preferable to (some) who have lost someone. Religions deal with the afterlife in different ways, and so it could be seen as someone else making "judgemental" assumptions.

Re: Show HN: An AI-Generated Encyclopedia

#20

Post by Philpax »

jfk13 wrote: post #17
visarga wrote: post #13

We need an AI-generated encyclopedia - not for us, but for AI. It should have a trillion articles covering all known entities and concepts, written using RAG over the web. Controversial topics should report the controversy or the distribution of opinions. We can put this big synthetic text corpus in the training set of future models. Why? Because AI needs long form, in-depth texts to train on, and the web doesn't pro…

> the web doesn't provide it in sufficient quantity and quality Do you seriously think that "an AI-generated encyclopedia" would provide a better-quality training set? What would the "AI generator's" articles be derived from?

The idea is that you can standardise the quality of the training data by taking source articles and synthesizing new data with the same "voice" and structure, as well as being able to collate insights from multiple sources.

This is the line of thinking behind the Phi lineup of models [0], as well as efforts to generate synthetic textbooks for training [1].

[0]: https://arxiv.org/abs/2309.05463

[1]: https://twitter.com/ocolegro/status/1712327588255809667

Post reply on HN