Searching the web for under $1000/month • Hacker News Board

Re: Searching the web for under $1000/month

#121

Post by mwcampbell » Fri, May 07, 2021, 8:06 PM UTC

toast0 wrote: post #96

onebot wrote: post #48

Earlier quoted context omitted.

I am really starting to feel that co-location will make a big comeback. It seems cloud costs are just becoming too high for the convenience they once offered. For small projects and scale probably makes a ton of sense, but at some point the costs to scale aren't worth the up front developer cost savings.

Bare metal hosting is a happy medium between co-lo and cloud. You don't have much control over the network, so it might not be enough if you need faster NICs than they offer, but if you fit in their offerings, it can work well. Otoh, the bare metal hoster I worked with is now owned by IBM, and a big competitor is owned by private equity; bare metal from cloud providers still has a lot of cloudiness associated too. Ma…

How about OVH? They now have data centers in Canada and the US as well as Europe.

Re: Searching the web for under $1000/month

#122

Post by f430 » Fri, May 07, 2021, 8:13 PM UTC

gallexme wrote: post #103

f430 wrote: post #97

Earlier quoted context omitted.

do you know if they let you host adult videos?

yeah sure if u have all the rights to host those videos in whole europe/germany its allowed https://www.hetzner.com/rechtliches/cloud-server/?country=de i know a couple people have naughty stuff on it (like sex toy shops, sexual services, private for sale adult videos)

hmmm why hetzner though? 100tb offers massive amount of bandwidth for tube sites I think.

do you know if Amazon S3 allows adult content?

Re: Searching the web for under $1000/month

#123

Post by tjoff » Fri, May 07, 2021, 8:15 PM UTC

arcturus17 wrote: post #105

bufferoverflow wrote: post #104

Earlier quoted context omitted.

AWS is almost never cost efficient. Maybe if you stay in their free tier.

> AWS is almost never cost efficient. A ridiculous blanket statement, despite the "almost never" cop-out... It is cost-efficient in a wide array of scenarios. Many companies pay for it because they have calculated the different investment scenarios and AWS comes on top of alternatives such as owning the hardware or using competing cloud vendors. I own a consultancy that builds complex web apps and while I appreciate…

Many companies pay for it because they have spent an inordinate amount of time learning the ecosystem and know of nothing else.

Re: Searching the web for under $1000/month

#124

Post by phendrenad2 » Fri, May 07, 2021, 8:29 PM UTC

Searching the web is a fool's errand. Google doesn't even search the web anymore, they just mind-controlled everyone to submit nightly sitemaps to them. Google is more of an index than a search engine nowadays.

Re: Searching the web for under $1000/month

#125

Post by gbrits » Fri, May 07, 2021, 8:36 PM UTC

natpat wrote: post #2

This is super interesting. I've recently also been working on a similar concept: we have a reasonable amount (in the terabytes) of data, that's fairly static, that I need to search fairly infrequently (but sometimes in bulk). A solution we came up with was a small , hot, in memory index, that points to the location of the data in a file on S3. Random access of a file on S3 is pretty fast, and running in an EC2 instan…

Also check out Dremio with parquet files stored on S3

Re: Searching the web for under $1000/month

#126

Post by marcinzm » Fri, May 07, 2021, 9:11 PM UTC

wongarsu wrote: post #106

whoknew1122 wrote: post #71

Earlier quoted context omitted.

Disclosure: Work at AWS. I've never seen AWS actively try to trick people into spending more money. I've seen Premium Support, product service teams, solutions architects, and account managers all suggest not to use AWS services if it doesn't fit the customer usecase. I've personally recommended non-AWS options for customers who are trying to fit a square peg into a round hole. Can the billing console be better? Yes.…

> pricing is transparent If pricing is intended to be transparent, then why is it completely absent from the user interface? Transparent pricing would be to tell me how much something costs when I order it, not make me use a different tool or find it in the documentation

No, no, you're supposed to use their cthulhu inspired pricing tool. I mean, you've got at least a 50/50 chance of figuring out how to use it before you go permanently insane.

Re: Searching the web for under $1000/month

#127

Post by not2b » Fri, May 07, 2021, 9:33 PM UTC

But are you solving the right problem? This sounds like someone has produced a very good and efficient version of AltaVista. Back in the 1990s, if you wanted to do classic keyword searches of the web, and find all pages that had terms A and B but not C, it would give them to you, in a big unsorted pile. The web was still small enough that this was sometimes useful, but until Google came along with tricks to rank pages that are obvious in retrospect, it just wasn't useful for common search terms.

Re: Searching the web for under $1000/month

#128

Post by bufferoverflow » Fri, May 07, 2021, 9:35 PM UTC

arcturus17 wrote: post #105

bufferoverflow wrote: post #104

Earlier quoted context omitted.

AWS is almost never cost efficient. Maybe if you stay in their free tier.

> AWS is almost never cost efficient. A ridiculous blanket statement, despite the "almost never" cop-out... It is cost-efficient in a wide array of scenarios. Many companies pay for it because they have calculated the different investment scenarios and AWS comes on top of alternatives such as owning the hardware or using competing cloud vendors. I own a consultancy that builds complex web apps and while I appreciate…

Name an AWS tier that there's no cheaper alternative for. I'm only aware of Glacier, I haven't seen anything cheaper than it.

AWS is convenient and reliable, but it's not cheap.

Re: Searching the web for under $1000/month

#129

Post by gmiller123456 » Sat, May 08, 2021, 12:31 AM UTC

Curious to know if anyone can explain why on-line storage is so expensive? Most places want $100+/mo for 1TB of storage, while a 1TB drive only costs $50. I understand there's management costs, cooling, electricity, physical space, etc. But those would be per drive costs, not per TB, and certainly wouldn't add up to $100/mo.

Meanwhile, there's services like Google Drive, etc which costs about $100/yr per TB. Still exorbitant, but not as much so. 3rd party software can mount it as a drive, but only for a short period of time before the token expires. So they seem happy to sell you space at less than 1/10th the cost, as long as it's harder to use.

There just seems to be a lot of cash on the table for someone to offer much cheaper storage solutions, but no one is actually doing it.

Re: Searching the web for under $1000/month

#130

Post by cbellew » Sat, May 08, 2021, 12:54 AM UTC

gmiller123456 wrote: post #129

Curious to know if anyone can explain why on-line storage is so expensive? Most places want $100+/mo for 1TB of storage, while a 1TB drive only costs $50. I understand there's management costs, cooling, electricity, physical space, etc. But those would be per drive costs, not per TB, and certainly wouldn't add up to $100/mo. Meanwhile, there's services like Google Drive, etc which costs about $100/yr per TB. Still ex…

Don't forget the replication. S3 replicates to at least three other AZs in the same region. I know this still doesn't account for the difference but its worth remembering.