Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My £4 a month server can handle 4.2M requests a day

mark.mcnally.je

Post reply on HN
Print view View original

421–430 of 479 posts

Re: My £4 a month server can handle 4.2M requests a day

#421

Post by nly »

andylynch wrote: post #347
localhost wrote: post #241

Earlier quoted context omitted.

Curious if the binary protocol uses floating point or a fixed point representation? Or is floating point with its rounding issues sufficient for the protocol's needs?

No GP but familiar with these protocols. They use fixed point extensively; I can't even thing of an exchange protocol which would use floating point since the rounding issues would cause unnecessary and expensive problems. This is typical (from NASDAQ http://www.nasdaqtrader.com/content/technicalsupport/specifi... ): Prices are integer fields. When converted to a decimal format, prices are in fixed point format with…

Many exchanges use floating point, even I the Nasdaq technology stack.

X-Stream feeds do for example

Re: My £4 a month server can handle 4.2M requests a day

#422

Post by nly »

blibble wrote: post #155
blacklion wrote: post #149

Earlier quoted context omitted.

"old skool" exchanges uses either FIX (old and really vernose), FAST (binary encoding for FIX) or custom fixed-layout protocols. Most big USA exchanges uses custom fixed-layout protocols, where each message is described in documentation, but not in machine-readable way. European ones still use FAST. I didn't seen FIX in the wild for data feeds, but it is used for brokers, to submit orders to exchange (our company did…

there's still ascii FIX floating around on the market data side for a few esoteric venues FAST is not particularly common in Europe the large European venues use fixed-width binary encoding (LSE group, Euronext, CBOE Europe)

Euronext uses SBE specifically.

Re: My £4 a month server can handle 4.2M requests a day

#423

Post by dimitrios1 »

scns wrote: post #383
dimitrios1 wrote: post #178

Earlier quoted context omitted.

If I look at traces of all the service calls at my company within our microservices environment, the "meat" of each service is a fraction of the latency -- the part that's actually fetching the data from a database, or running an intense calculation. Often times its between 20-40ms Everything else are network hops and what I call "distributed clutter", including authorizing via a third party like Auth0 multiple times…

Like this? https://www.youtube.com/watch?v=y8OnoxKotPQ

Exactly like that.

Re: My £4 a month server can handle 4.2M requests a day

#424

Post by silverscania »

MagnumOpus wrote: post #413
silverscania wrote: post #393

Earlier quoted context omitted.

The IoM isn't in the UK and nor does it have MPs. Probably just semantics but I have to point that out.

IoM has MPs, just not UK MPs. The OP almost certainly meant an MP from the Manx Tynwald parliament.

It doesn't though, that's my point. The Isle of Man has MHKs and ministers, not "MPs": https://en.wikipedia.org/wiki/House_of_Keys.

It's like saying a politician in the UK is a "senator" and expecting people think you are an authority on the subject.

Re: My £4 a month server can handle 4.2M requests a day

#426

Post by nexuist »

Spooky23 wrote: post #410
KptMarchewa wrote: post #357

Earlier quoted context omitted.

Who the fuck thinks it's a good idea to run database on networked storage?

You might have heard of this little company in Seattle called “Amazon”.

Like the river? They're gonna need to fight hard for that SEO.

Re: My £4 a month server can handle 4.2M requests a day

#427

Post by ajnin »

noduerme wrote: post #268

When I launched my former bitcoin casino in 2011 (it's gone, but it was a casino where all games, even roulette tables were multiplayer, on a platform built from scratch starting in '08), I handled all web requests through a server in Costa Rica that cost about $6/mo. Where I had a shell corporation for $250/year. Once the front end -- the bullet containing the entire casino code, about 250kb -- loaded, from Costa Ri…

Thanks for sharing that, I hope it does not land you into trouble. So, how many billions did you make with that site ?

Re: My £4 a month server can handle 4.2M requests a day

#428

Post by bradstewart »

benhoyt wrote: post #201
simonw wrote: post #187

Earlier quoted context omitted.

Django's prefetch_related mechanism essentially implements the pattern you describe here for you: https://docs.djangoproject.com/en/3.2/ref/models/querysets/#...

Thanks. Yeah, I think I used that years ago when I first ran into this problem, and it worked well. Whether one uses an ORM or not, one needs to know how to use one's tools. My problem (not just with Django, but with ORMs in general) is how they make bad code look good. Like the following (I don't know Django well anymore, but something like this): users = User.objects.all() for u in user: print(u.name, len(u.comment…

> Or even just a heuristic mode for development where it yelled loudly if it detected what looked like an N+1 issue or other query inside a loop.

Fwiw, ActiveRecord with the Bullet gem does exactly that. I'd guess there's an equivalent for Django.

Re: My £4 a month server can handle 4.2M requests a day

#429

Post by zschuessler »

noduerme wrote: post #411
ricardobeat wrote: post #395

Earlier quoted context omitted.

With all this trouble, why not just set up shop in one of the many places where online gambling is legal?

I spent 3 years writing the software in my spare time, and launched it on the $20k I had in savings... personally bankrolling the games. When I began the alpha phase, you could still get an "information processing" license to run a casino in Costa Rica that would allow you to host gambling and which the credit card companies tacitly accepted and would do business with. (Those gambling payments to shady jurisdictions…

This story is fascinating. What ended up happening with the project, did you at least break even?

I would read a book about this, fwiw.

Re: My £4 a month server can handle 4.2M requests a day

#430

Post by rattray »

nostrademons wrote: post #20

People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to we…

What were the most important aspects of the technology stack which enabled that?
Post reply on HN