> Enterprise: Contact Us If there is no certifications or compliance information then I don't think there is anything to discuss about any enterprise plan.
Gotta start somewhere :)
Show HN: Turn any website into a knowledge base for LLMs
51–60 of 135 posts
Re: Show HN: Turn any website into a knowledge base for LLMs
#52I like this a lot! But: I feel the more of these services come to being, the more likely it is that every website starts putting up gates to keep the bots away. Sort of like a weird GenAI take on Cixin Liu's Dark Forest hypothesis ( https://en.wikipedia.org/wiki/Dark_forest_hypothesis ). (Edited to add a reference.)
This would be amazing
Re: Show HN: Turn any website into a knowledge base for LLMs
#53I spent a lot of time thinking about how to manage embeddings for docs sites. This is basically the same solution that I landed on but never got around to shipping as a general-purpose product.
A key question that the docs should answer (and perhaps the "How it works" page too): chunking. You generate an embedding for the entire page? Or do you generate embeddings for sections? And what's the size limit per page? Some of our docs pages have thousands of words per page. I'm doubtful you can ingest all that, let alone whether the embedding would be that useful in practice.
Re: Show HN: Turn any website into a knowledge base for LLMs
#54I spent a lot of time thinking about how to manage embeddings for docs sites. This is basically the same solution that I landed on but never got around to shipping as a general-purpose product. A key question that the docs should answer (and perhaps the "How it works" page too): chunking. You generate an embedding for the entire page? Or do you generate embeddings for sections? And what's the size limit per page? Som…
I chunk pages and generate embeddings for each chunk. So there's no real size limit per page.
Re: Show HN: Turn any website into a knowledge base for LLMs
#55Which LLM model is it using?
Re: Show HN: Turn any website into a knowledge base for LLMs
#56I like this a lot! But: I feel the more of these services come to being, the more likely it is that every website starts putting up gates to keep the bots away. Sort of like a weird GenAI take on Cixin Liu's Dark Forest hypothesis ( https://en.wikipedia.org/wiki/Dark_forest_hypothesis ). (Edited to add a reference.)
Responding just because it's a pet peeve of mine: Cixin Liu did not invent the dark forest hypothesis. People were discussing it, and writing science fiction books about it, for decades before the 3BP books were published. Nothing against him, and he definitely helped popularize the concept, but I think it's incorrect to refer to it as "Cixin Liu's hypothesis".
But he is responsible for the name, not the concept. So yes it is Cixin Liu's Dark Forest hypothesis.
Re: Show HN: Turn any website into a knowledge base for LLMs
#57Earlier quoted context omitted.
Responding just because it's a pet peeve of mine: Cixin Liu did not invent the dark forest hypothesis. People were discussing it, and writing science fiction books about it, for decades before the 3BP books were published. Nothing against him, and he definitely helped popularize the concept, but I think it's incorrect to refer to it as "Cixin Liu's hypothesis".
But he is responsible for the name, not the concept. So yes it is Cixin Liu's Dark Forest hypothesis.
Just like Amerigo Vespucci put the name "America" on a map and people starting referring to the New World as such, although he didn't discover it himself.
Re: Show HN: Turn any website into a knowledge base for LLMs
#58[deleted]
Re: Show HN: Turn any website into a knowledge base for LLMs
#59In my opinion this is a transitional niche.
Soon websites/apps whatever you want to call them will have their own built-in handling for AI.
It's inefficient and rude to be scraping pages for content. Especially for profit.
Re: Show HN: Turn any website into a knowledge base for LLMs
#60I spent a lot of time thinking about how to manage embeddings for docs sites. This is basically the same solution that I landed on but never got around to shipping as a general-purpose product. A key question that the docs should answer (and perhaps the "How it works" page too): chunking. You generate an embedding for the entire page? Or do you generate embeddings for sections? And what's the size limit per page? Som…
I chunk pages and generate embeddings for each chunk. So there's no real size limit per page.
The more detail, the better. If `` elements are found you chunk those? Do you do it recursively or do you stop after a certain level? And when section elements don't exist, you use ``, ``, etc. to infer logical chunks?