Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Searching the web for under $1000/month

quickwit.io

Post reply on HN
Print view View original

91–100 of 153 posts

Re: Searching the web for under $1000/month

#91

Post by heipei »

ryanlol wrote: post #78
heipei wrote: post #56

Earlier quoted context omitted.

Huge fan of Hetzner, but dedicated servers do not invalidate the value proposition of the cloud. Ordering a server at Hetzner can take anywhere between a few minutes and a few days. Each server has a fixed setup cost of around the monthly rent. They only have two datacenters in Europe. They don't have any auxillary services (databases, queues, scalable object storage, etc.). They are unbeatable for certain use-cases,…

> They only have two datacenters in Europe Nonsense, Hetzner operates like 25 datacenters.

Sorry, let's call it "regions" then, they have multiple DCs in different cities in Germany, but for latency purposes I would consider these part of one region.

Re: Searching the web for under $1000/month

#92

Post by snidane »

Chaos Search seems to be doing this architecture already and according to the podcast episode [1], it uses a highly optimized storage layout.

Never used it, so would be interested if somebody could comment on it.

[1] https://www.dataengineeringpodcast.com/chaos-search-with-pet...

Re: Searching the web for under $1000/month

#93

Post by rossmohax »

curryst wrote: post #72
onebot wrote: post #48

Earlier quoted context omitted.

I am really starting to feel that co-location will make a big comeback. It seems cloud costs are just becoming too high for the convenience they once offered. For small projects and scale probably makes a ton of sense, but at some point the costs to scale aren't worth the up front developer cost savings.

Where they get you is that it very rarely makes financial sense to do both cloud and colo/on-prem (unless you're a massive company). It ends up being way more expensive to use the cloud, but also hire engineers to work on making an on-prem cloud. Most companies have a mixed bag of projects that are either better served by the cloud, or are okay with colo and the savings it can bring. Assuming you don't want to do a h…

You save on specialized engineers (Database, RabbitMQ, Ceph administrators), but you lose elsewhere.

What used to be an apache serving static files, now is S3 bucket, but it wont be easy, because you wanted your own domain, so now you need a Cloudfront because of SSL support. Their tutorial conveniently mentions it only at the step 7 ("Test your website endpoint").

You buy into Cognito, great, saved money on Keycloak administrator, but in the worst moment deep in the project you learn that there is absolutely no way to support multiple regions, even if you are willing to do some leg work for AWS. Or find that Cognito email reset flow can't go through your existing customer contact system and must go through SES only, suddenly you find developing elaborate log/event processing tool just that your customer service agent can see password reset event on their interface.

GCP CloudSQL, managed RDBMS, great! No upgrade for you other than SQL dump/restore your 10TB instance, have fun.

Cloud might be a net win still, but it is very much not as rosy as cloud evangelists want us think.

Re: Searching the web for under $1000/month

#95

Post by pcnix »

fulmicoton wrote: post #42
nopurpose wrote: post #32

It is a cool project. S3 can be cost efficient, but only if you don't touch data :) Their price calculation doesn't mention cost of S3 requests, which very quickly adds up and is often neglected. It costs $1 for 2.5M GET requests to S3. They have 180 shards, in a general case query seems to fetch all of them. Presumably they don't download full shard per request, but download an index + some relevant ranges. Lets say…

I tend to agree :). If we get 1 req/s, even for a dataset of that size, this is not as cost efficient. For that kind of use case, I'd probably start using minio. > Seems comparable to AWS ElasticSearch service costs: > - 3 nodes m5.2xlarge.elasticsearch = $1,200 > - 20TB EBS storage = $1,638 Don't forget S3 includes replication. Also EBS throughput (even with SSD) is not good at all. Also our memory footprint is tiny…

Have you checked out the new EBS gp3 disks? Throughout vs cost is much better on those than gp2, and also cheaper than Provisioned IOPS

Re: Searching the web for under $1000/month

#96

Post by toast0 »

onebot wrote: post #48
jonatron wrote: post #16

If you're going for low cost, you could do better: https://www.hetzner.com/dedicated-rootserver/dell/dx181/conf... Basic configuration in Finland 1 224,91 € 1.92 TB SATA SSD Datacenter Edition 4 95,20 € 320,11 € 320 Euro equals 385.90 United States Dollar

I am really starting to feel that co-location will make a big comeback. It seems cloud costs are just becoming too high for the convenience they once offered. For small projects and scale probably makes a ton of sense, but at some point the costs to scale aren't worth the up front developer cost savings.

Bare metal hosting is a happy medium between co-lo and cloud. You don't have much control over the network, so it might not be enough if you need faster NICs than they offer, but if you fit in their offerings, it can work well.

Otoh, the bare metal hoster I worked with is now owned by IBM, and a big competitor is owned by private equity; bare metal from cloud providers still has a lot of cloudiness associated too. Maybe colo is the way to go.

Re: Searching the web for under $1000/month

#97

Post by f430 »

jonatron wrote: post #16

If you're going for low cost, you could do better: https://www.hetzner.com/dedicated-rootserver/dell/dx181/conf... Basic configuration in Finland 1 224,91 € 1.92 TB SATA SSD Datacenter Edition 4 95,20 € 320,11 € 320 Euro equals 385.90 United States Dollar

do you know if they let you host adult videos?

Re: Searching the web for under $1000/month

#98

Post by francoismassot »

cardosof wrote: post #94

Congrats for the project and very cool demo! One point that may help - I've searched the word fast with adjective selected and it didn't show results.

thanks! I guess you had no luck and the server did not respond, we have a bunch of errors on the python server and it may come from here.

It's working now, you can try it and found that the result is "fast and easy": https://common-crawl.quickwit.io/?query=fast&partOfSpeech=AD...

Post reply on HN