Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My £4 a month server can handle 4.2M requests a day

mark.mcnally.je

Post reply on HN
Print view View original

301–310 of 479 posts

Re: My £4 a month server can handle 4.2M requests a day

#301

Post by paulluuk »

b9a2cab5 wrote: post #188
mynameisash wrote: post #158

Earlier quoted context omitted.

What Andy giveth, Bill taketh away.[0] I'm more than a little annoyed that so much data engineering is still done in Scala Spark or PySpark. Both suffer from pretty high memory overhead, which leads to suboptimal resource utilization. I've worked with a few different systems that compile their queries into C/C++ (which is transparent to the developer). Those tend to be significantly faster or can use fewer nodes to p…

Abstractions almost always end up leaky. Spark SQL, for example, does whole-stage codegen which collapses multiple project/filter stages into a single compiled stage, but your underlying data format still needs to be memory friendly (i.e. linear accesses, low branching, etc.). The codegen is very naive and the JVM JIT can only do so much. What I've seen is that you need people who deeply understand the system (e.g. S…

What's wrong with relying on data engineers for data engineering?

Re: My £4 a month server can handle 4.2M requests a day

#303

Post by elcomet »

Gepsens wrote: post #286
noduerme wrote: post #268

When I launched my former bitcoin casino in 2011 (it's gone, but it was a casino where all games, even roulette tables were multiplayer, on a platform built from scratch starting in '08), I handled all web requests through a server in Costa Rica that cost about $6/mo. Where I had a shell corporation for $250/year. Once the front end -- the bullet containing the entire casino code, about 250kb -- loaded, from Costa Ri…

You're a fricking genius.

He's a frickin genius because he found ways to break the law with impunity for his own profit ?

I'm surprised by HN sometimes.

Re: My £4 a month server can handle 4.2M requests a day

#304

Post by funnybeam »

spfzero wrote: post #255
JJMcJ wrote: post #129

Earlier quoted context omitted.

Your pattern is quite powerful: get data from several sources and do the rearranging on the client (which might be a web server), instead of multiple interactions for each data item. For SQL you can also do a stored procedure. Sometimes that works well if you are good at your DBMS's procedure language and the schema is good.

Though stored procs will always execute on the server that is probably already the system bottleneck.

Sending multiple queries from your web server will likely put more load on the database server than using a stored procedure.

With either technique you are still pulling all the data you need from the DB but with multiple queries instead of a stored procedure you are usually pulling more data than you need with each query and then dropping any rows or fields you’re not interested in. Together with multiple calls over the network to the DB server and (often) multiple SQL connection setups this is much worse for performance on both the web and database servers

Re: My £4 a month server can handle 4.2M requests a day

#305

Post by SPBS »

vymague wrote: post #131
polote wrote: post #41

What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually…

> What's the point of this post? OP is serving a file at 50req/sec. I'd guess a response to the mangadex thread? https://news.ycombinator.com/item?id=28440742

No way, OP is essentially serving a static webpage from a database. That's nothing to brag about if comparing with mangadex.

Re: My £4 a month server can handle 4.2M requests a day

#306

Post by kragen »

WanderPanda wrote: post #267
nostrademons wrote: post #196

Earlier quoted context omitted.

That particular example seems like something that's probably a lot cheaper than you'd initially think. The OS has to constantly take mouse input anyway to move the pointer and dispatch events to userspace. It also needs to record the current and new position of the mouse pointer to dispatch the events. Detecting whether the mouse is being "shaken" can be done with a ring buffer of mouse velocities over the last secon…

Why not use an exponential filter that only uses a single variable?

The running-sum-difference approach suggested above is a box filter, which has the best possible noise suppression for a given step-function delay, although in the frequency domain it looks appalling. It uses more RAM, but not that much. The single-pole RC filter you're suggesting is much nicer in the frequency domain, but in the time domain it's far worse.

Re: My £4 a month server can handle 4.2M requests a day

#307

Post by 6510 »

hypertele-Xii wrote: post #237
trisiak wrote: post #234

Earlier quoted context omitted.

What's behind the mouse cursor while you're doing it? Could it be the UX/UI layer keeping state up-to-date? Other possibility, do you have a gaming mouse with 1000Hz polling rate configured?

Yeah, it's a high quality mouse. But the only excuse for this is it's slightly cheaper to make everything USB. PS/2 worked much better. It was limited to 200Hz but needed no polling. Motherboards just stopped providing the port.

If the computer has to do anything at all it ads to complexity and it isn't doing other things. One could do something a bit like blue screening and add the mouse pointer to the video signal in the monitor. For basic functionality the computer only needs to know the x and y of clicks. (it could for laughs also report the colors in the area) Hover and other effects could be activated when [really] needed. As a bonus one could replace the hideous monitor configuration menus with a point and click interface.

Re: My £4 a month server can handle 4.2M requests a day

#309

Post by noir_lord »

elcomet wrote: post #303
Gepsens wrote: post #286

Earlier quoted context omitted.

You're a fricking genius.

He's a frickin genius because he found ways to break the law with impunity for his own profit ? I'm surprised by HN sometimes.

You can respect the ingenunity if not the morality of it.

His solution was a clever hack because he worked around the law without apparently breaking it directly - clever.

That said gambling systems are specifically one class of software I refuse to work on.

Re: My £4 a month server can handle 4.2M requests a day

#310

Post by noduerme »

elcomet wrote: post #303
Gepsens wrote: post #286

Earlier quoted context omitted.

You're a fricking genius.

He's a frickin genius because he found ways to break the law with impunity for his own profit ? I'm surprised by HN sometimes.

Uh, excuse me, but this whole story demonstrates that I went to extreme lengths to never break any laws, no matter how large or small the jurisdiction. Had I been willing to bend laws, I'd surely be a lot richer today.
Post reply on HN