111–120 of 153 posts
Cool demo. Searching for phases like "there was a" and "and there is" take a really long time. I presume that since the words are common, the document IDs mapped to those individual tokens are too long as well, so intersections etc. take longer?
How are you dealing with the fact common crawl updates their data much less regularly than commercial search engines? And that each update is only a partial refresh? Edit: And I will say your site design is very nice.
How are you dealing with the fact common crawl updates their data much less regularly than commercial search engines? And that each update is only a partial refresh? Edit: And I will say your site design is very nice.
Earlier quoted context omitted.
I am really starting to feel that co-location will make a big comeback. It seems cloud costs are just becoming too high for the convenience they once offered. For small projects and scale probably makes a ton of sense, but at some point the costs to scale aren't worth the up front developer cost savings.
Where they get you is that it very rarely makes financial sense to do both cloud and colo/on-prem (unless you're a massive company). It ends up being way more expensive to use the cloud, but also hire engineers to work on making an on-prem cloud. Most companies have a mixed bag of projects that are either better served by the cloud, or are okay with colo and the savings it can bring. Assuming you don't want to do a h…
> In-house systems tend to be poorly documented, and doing anything slightly outside the norm becomes a "set up a meeting with the infra team" kind of thing.
I usually wasn’t really happy with AWS’ documentation. But now, considering the alternative, it find it quite lovely. Thank you for making me realize that.
Earlier quoted context omitted.
AWS is a scam not because it can’t save you money, but because they actively try to trick you into spending more money. That’s practically the definition of a scam. Go to the AWS console and try to answer even simply things like how much did the last hour/day/week cost me? Or how about some notifications if that new service you just added is going to cost vastly more than you where expecting. I know of a few people g…
Disclosure: Work at AWS. I've never seen AWS actively try to trick people into spending more money. I've seen Premium Support, product service teams, solutions architects, and account managers all suggest not to use AWS services if it doesn't fit the customer usecase. I've personally recommended non-AWS options for customers who are trying to fit a square peg into a round hole. Can the billing console be better? Yes.…
If clarity causes customers to spend less it is disincentivized and since clarity is hard and requires active investment to maintain it decays naturally.
It's easy to see how you can end up with a system that the users experience as a dishonest attempt to get more of their money and operators, who are necessarily very familiar with the system experience as merely messy but transparent.
Neither is precisely wrong however your users don't have your experience or training and many are liable to interact with a computer not you. Your system is then exactly as honest and transparent as your UI as perceived by your average user.
How are you dealing with the fact common crawl updates their data much less regularly than commercial search engines? And that each update is only a partial refresh? Edit: And I will say your site design is very nice.
We indexed Common Crawl only for the purpose of this demo so this is one-time thing, we won't deal with updates.
Then update it with git like diff versioning, utilize IPNS to point to HEAD of the latest chain of the index.