Earlier quoted context omitted.
> A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. What do you reckon the cost of doing this would be? Just doing the napkin math for say a Mojeek sized index (couple of billion docs) doesn't seem to justify a particularly astronomi…
7 billion+ actually, and our investment level mentioned here: https://news.ycombinator.com/item?id=36517505
Kagi raises $670k
161–170 of 367 posts
Re: Kagi raises $670k
#162Re: Kagi raises $670k
#163Earlier quoted context omitted.
> A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. What do you reckon the cost of doing this would be? Just doing the napkin math for say a Mojeek sized index (couple of billion docs) doesn't seem to justify a particularly astronomi…
>What do you reckon the cost of doing this would be? That is indeed the key question. I tried to find out before commenting but the information I found is very vague. Internet Archive spends millions per year, but their index is updated far too slowly for a search engine. I have no idea what it costs to create a Google-size index. Do you think the Mojeek index is good enough to compete with Google?
Assuming they'll need to index 820 billion pages (the number of pages preserved in the internet archive), at 100kb each, and assuming they use a database with 0.3x text data compression efficiency, they'll need at least 24600 TB to store those text data. Assuming $300 per 16TB disk, then they'll need to spend at least $7,380,000 for disk alone. This is a lot of money just for storage and we haven't included stuff like replication and backup, indexing metadata overhead, etc.
Re: Kagi raises $670k
#164Re: Kagi raises $670k
#165Earlier quoted context omitted.
I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important. The absolute worst way you can arrange a search engine project is as some sort of manhattan project with a humongous budget and an army of professors and experts. History is littered with bold and ambitious Google killers that went nowhere. You can throw almost any amount of money at an operation, and it will gobbl…
Yeah, there's a lot of truth in this, and it matches my experience, with the caveat that the 3-4 guys need to stay very clear eyed and pragmatic with their technology choices. Kind of covered by the "talented" part but I've seen so many otherwise talented people fall into holes because of this, and I'd guess it applies doubly for building a search engine.
Re: Kagi raises $670k
#166Earlier quoted context omitted.
>What do you reckon the cost of doing this would be? That is indeed the key question. I tried to find out before commenting but the information I found is very vague. Internet Archive spends millions per year, but their index is updated far too slowly for a search engine. I have no idea what it costs to create a Google-size index. Do you think the Mojeek index is good enough to compete with Google?
Internet archive snapshot the whole page while google only index the first few kb of text data on every page (forgot the number, was it 100kb max per page?), so maybe it'll cost lower to build a search engine index compared to an internet preservation project? Assuming they'll need to index 820 billion pages (the number of pages preserved in the internet archive), at 100kb each, and assuming they use a database with…
They changed that limit somewhere in the mid-2000s. Just as well, there's some CMS's out there where there's several hundred kilobytes of inline JS and CSS before any body text.
Re: Kagi raises $670k
#167>Kagi Search requires an account only because it is a paid service which requires an account for the transaction. ref: https://help.kagi.com/kagi/getting-started/faqs.html#why-doe... Eh... This is a tall order when competition is 'free'. Anyways, I wish them good luck as competition is inherently good.
Competition is free if you don't care about your privacy or your personal data. Many don't, sure, but it seems enough do care such that Kagi has continued to grow and improve over time. A tall order for what? They already have plenty of paying customers and it's a self-sustaining business already. What more do you want?
Specially so for technical queries or prior art queries it really feels like a 2007 google crawler, which is actually a complement.
Re: Kagi raises $670k
#168"Kagi is building a novel ad-free, paid search engine and a powerful web browser as a part of our mission to humanize the web." With 670K? Cuil went through about $30 million to develop a standalone search engine. And that was fifteen years ago, when search was simpler. There may be a market for a search engine company that profitably runs a low-cost operation with very few ads and makes real efforts to keep out spam…
Re: Kagi raises $670k
#169Earlier quoted context omitted.
I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important. The absolute worst way you can arrange a search engine project is as some sort of manhattan project with a humongous budget and an army of professors and experts. History is littered with bold and ambitious Google killers that went nowhere. You can throw almost any amount of money at an operation, and it will gobbl…
>I think a small operation is exactly the sort of outfit to do it. A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. As a consequence, they will always be at the mercy of Google/Bing. The range of things they can innovate on will alw…
Sure. But Google/Bing has shown lack of interest in innovating for power searchers. I believe they'll continue doing so.
And even so, this space is big, I'm sure there are many niches that would fit a scrappy, subscription based player.
Re: Kagi raises $670k
#170Kagi is really, really good. Lenses (where you can setup set filters for particular search use cases) are particularly good. Very happy to see them raising, and very happy to give them my $10 each month. Money well spent.