Live data from Hacker News

My £4 a month server can handle 4.2M requests a day

mark.mcnally.je

381–390 of 479 posts

Re: My £4 a month server can handle 4.2M requests a day

#381
post #351

Earlier quoted context omitted.

I suspect your anger derives from the fact that this would not be possible to get away with now, in any way shape or form, only ten years later. And truly the world is way more locked down now than it was then when people were like, "Bitcoin? What's that? you want to pay me to write a legal brief?" so, yeah. I feel sorry for kids ten years younger than me. When I did it, the only thing I was really afraid of was gett…

I'm not angry, I'm hostile. That you think the only possible position from which to take issue with what you did is FOMO speaks volumes. I'm not arguing that you violated any laws. You made it very clear that you went to great lengths to avoid doing anything that could have resulted in consequences to yourself. EDIT: Since HN's rate limiting won't let me reply for a few hours, I'll just address the replies inline: I'…

Ok, I respectfully understand where you're coming from, and I've struggled with gambling addiction myself. I personally do not think it's immoral to offer games of chance to people, as long as they understand the odds and you're not cheating them. And believe me, running a small online casino mostly by myself with my own bankroll was literally setting alarms all night waking me up when someone was killing the tables and potentially going to bankrupt me. I got to know my big players (most of whom went on to become Bitcoin millionaires, since the early adopters were the only ones gambling on Bitcoin casinos in 2011)... but beyond that, I really don't think offering gambling is immoral as long as everyone knows what they're signing up for. I've dealt cards for a living, too. I don't drive a taxi anymore. But I've seen all sides of life. I don't think you can judge so easily. Yeah, big corporations suck and they screw people into debt, and gambling addiction does ruin peoples' lives, but I knew my players, and I don't think what I did personally hurt anybody. They came together to enjoy games, and yeah it was for real money, but it was also a community and they were there for fun - they could have gambled for a lot more money on other sites. One player built a puzzle in his escape room in Amsterdam in honor of / based on a game I designed. Like a bar, this is something people do for enjoyment, and you don't need to stand out there with a sign saying we're all going to hell. But I do get it and I'm not a fan of the companies that take advantage of human weakness and shake the dimes out of people's pockets. I was just trying to have a good time and give other people a good time.

Total profit from 2 years running the site? About $50k. It was a hobby. I never quit my job. I also turned away 95% of the hits because they were coming from America.

[edit] I should add that I strongly advised other BTC site owners, especially casino owners, to follow certain guidelines, and watched one of them who I had told to be careful launch, make about $1M with one game on a crappy website, and get jailed within a year. That wasn't the trajectory I was interested in.

Re: My £4 a month server can handle 4.2M requests a day

#382
post #306

Earlier quoted context omitted.

The running-sum-difference approach suggested above is a box filter, which has the best possible noise suppression for a given step-function delay, although in the frequency domain it looks appalling. It uses more RAM, but not that much. The single-pole RC filter you're suggesting is much nicer in the frequency domain, but in the time domain it's far worse.

Is it „far worse in the time domain“ due to its infinite impulse response?

Not really? Sort of? I don't really have a good answer here. It depends on what you mean by "due to". It's certainly due to the impulse response, since in the sense I meant "far worse" the impulse response is the only thing that matters.

Truncating the impulse response after five time constants wouldn't really change its output noticeably, and even if you truncated it after two or three time constants it would still be inferior to the box filter for this application, though less bad. So in that sense the problem isn't that it's infinite.

Likewise, you could certainly design a direct-form IIR filter that did a perfectly adequate job of approximating a box filter for this sort of application, and that might actually be a reasonable thing to do if you wanted to do something like this with a bunch of op-amps or microwave passives instead of code.

So the fact that the impulse response is infinite is neither necessary nor sufficient for the problem.

The problem with the simple single-pole filter is that by putting so much weight on very recent samples, you sort of throw away some information about samples that aren't quite so recent and become more vulnerable to false triggering from a single rapid mouse movement, so you have to set the threshold higher to compensate.

Re: My £4 a month server can handle 4.2M requests a day

#383

Earlier quoted context omitted.

I did some work to improve performance on a dashboard several years ago. The way the statistics were queried was generally terrible, so I spent some time setting up aggregations and cleaning that up, but then... the performance was still terrible. It turned out that the dashboard had been built on top of Wordpress. The way that it checked if the user had permission to access the dashboard was to query all users, join…

If I look at traces of all the service calls at my company within our microservices environment, the "meat" of each service is a fraction of the latency -- the part that's actually fetching the data from a database, or running an intense calculation. Often times its between 20-40ms Everything else are network hops and what I call "distributed clutter", including authorizing via a third party like Auth0 multiple times…

Like this? https://www.youtube.com/watch?v=y8OnoxKotPQ

Re: My £4 a month server can handle 4.2M requests a day

#384
post #221

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

How much bandwidth did this use daily/monthly?

I don't remember offhand. I think bandwidth costs averaged about $40-50 out of that ~$200/month.

Re: My £4 a month server can handle 4.2M requests a day

#385
post #358

When I launched my former bitcoin casino in 2011 (it's gone, but it was a casino where all games, even roulette tables were multiplayer, on a platform built from scratch starting in '08), I handled all web requests through a server in Costa Rica that cost about $6/mo. Where I had a shell corporation for $250/year. Once the front end -- the bullet containing the entire casino code, about 250kb -- loaded, from Costa Ri…

Focusing on a small but important detail that some have already mentioned but with a more aggressive tone... was your "loophole" system tested in an actual litigation at any point? What I mean is that this: > The IoM server would call the Swiss server every time a hand was dealt might seem like a clever loophole around the laws in IoM, but in reality it sounds to me like the kind of technicalities that wouldn't reall…

Yeah, that's like setting up a casino in one location with a permanent phone line open to switzerland to ask where the ball landed on the roulette wheels. Doesn't seem like it would hold up under scrutiny.

Re: My £4 a month server can handle 4.2M requests a day

#387

Earlier quoted context omitted.

What do you suggest, Gopher?

Github/Gitlab pages is my choice for being free, managing https and just generally being pretty good. You can even use the CI to compile the site for you.

Ah, kids these days. So I guess by "webserver" grandparent meant "A server you actually control", and by "not webserver" people mean "It's on the cloud somewhere".

GH/GL-pages still respond to HTTP/HTTPS requests, in my dictionary that's a webserver, but I guess for the millennials (I say with snark) it means "I don't have to think about what happens, it just serves my content for me."

Re: My £4 a month server can handle 4.2M requests a day

#388
It feels to me like most websites out there could run on way less hardware, if only people would embrace a few things.

#1 Minimalism. You don't need 400 KB of JS to display some mostly text content to your users with some interactivity sprinkled in.

You don't need to reinvent office software, or very rich text editors in browsers, stop using the web as a universal delivery platform/mechanism, because that's not what it was meant for. When browsers will ship integrated dependencies so that even CDNs don't need to be hit (like versions of jQuery, Bootstrap and numerous JS frameworks as well as WASM code like Blazor which contains a .NET runtime), then you'll be able to do that, but arguably that will never happen.

Use the web as a platform for displaying primarily text content with the occasional images, forms and a little bit of interactivity sprinkled in. Most sites out there simply aren't and shouldn't be like this (that said, when you have exceptional reasons for throwing aside that suggestion, do so): https://geargenerator.com

#2 Static content. You don't need to use Wordpress, Drupal, Joomla or many of the other CMSes out there, since they can get really heavyweight with numerous plugins and are not only a security challenge, but are also problematic from a performance perspective.

Consider using static site generators instead. When reading an article of yours, the DB shouldn't even be hit, since most of the article contents are unlikely to change often, so you should be able to pre-render each of the article versions as a set of static HTML and use the common JS/CSS that you already have for the rest of the articles. Furthermore, it's easy to just jump into CMSes and introduce ungodly amounts of complexity, all of which cause your back end to process bunches of code for each request. Static files don't have that drawback.

#3 Caching. Know when and what to cache, and how. Images, JS files, CSS files and even entire HTML pages should be cache friendly. Know which ones aren't, make exceptions for those and cache everything else.

Not only is it not necessary to hit the DB for many of the pages in your site at all, but also sometimes you shouldn't even hit the back end either. The most popular pages of your site should just live in a cache somewhere, be it within your web servers or a separate solution, so that they can be returned instantly. HTML is good for this, use it.

Furthermore, know what cache policies to use. Sometimes even the cache resources shouldn't be redownloaded, if the user already has these resources loaded from a different page. Use bundle splitting responsibly, extract common functionality in easily cacheable bundles and set the appropriate headers.

And yet, i've seen a surprising amount of ignorance in regards to caching, static site generation and even how large webpages have gotten: https://idlewords.com/talks/website_obesity.htm

I don't claim to know it all, but working towards the goal of efficiently using pages should definitely be viewed as an important one: be it because you want to pay less for your infrastructure, or care about the environment, or even just want to manage fewer nodes.

Instead, nowadays far too many orgs just try to be the first to market and ignore the engineering based approach to ensuring that the solutions are not only functional but also sustainable. That saddens me.

Re: My £4 a month server can handle 4.2M requests a day

#389

One of my proudest moment in my career is when I lowered our app processing time from ~8hrs to 17 minutes. When I deployed my first update, it reduced it to 2 hours. The sysadmin immediately contacted me that there was something unusual. I confirmed the results but he was skeptical. Then with my second update, he told me that the app must be broken or that the script must be dying. There is no way it could complete t…

I have a similar story where I reduced the memory used by a script from 1TB (yes, TB) to a few megabytes. The runtime was massively reduced too, from something like 1 day to a few minutes.

This was for a genomics project and they ran it on a supercomputer. When I looked into it, they were reading the entire input into a giant array before doing one pass and dumping the result out to disk. I made a tiny change (it was a Perl script) to make it stream the I/O instead.

This is the most extreme example I've come across of people using computing power just because it's there. Nobody questioned why the script took so long to run because the data really was in the TBs and other stuff also took that long to run. Waiting a day for the results was considered normal. I see the same thing on desktop apps etc., on a much smaller scale, of course. When I run an electron app it takes several hundred milleseconds to do anything at all. But nobody questions whether it should because everything takes several hundred milliseconds.

Re: My £4 a month server can handle 4.2M requests a day

#390

When I launched my former bitcoin casino in 2011 (it's gone, but it was a casino where all games, even roulette tables were multiplayer, on a platform built from scratch starting in '08), I handled all web requests through a server in Costa Rica that cost about $6/mo. Where I had a shell corporation for $250/year. Once the front end -- the bullet containing the entire casino code, about 250kb -- loaded, from Costa Ri…

It would be even cooler if you derived the random numbers from a lava lamp visible through a window of the Swiss embassy.
Post reply on HN