Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are still early with the cloud

erikbern.com

Post reply on HN
Print view View original

141–150 of 202 posts

Re: We are still early with the cloud

#141

Post by znpy »

verisimilitudes wrote: post #40
rsync wrote: post #26

Earlier quoted context omitted.

"It's crazy and destructive that we are still using the unix paradigm in the cloud." # ssh user@rsync.net "test -f fileThatExists" #echo $? 0 ... from my cold, dead hands ...

Quoted post unavailable.

such things however are just so often just so much better.

I was working with an intern lately, we both had remote machines in the same subnet (i'm omitting some details, of course), and he had to pass me a file.

We was about to download it on his laptop, upload it back to the cloud (slack message) when I would have had to download it on my laptop and send it back to the cloud.

I was able to show this young engineer how to use cat and nc (cat somefile.tgz | nc 8000) to send and then I would use nc (nc -l 8000 > somefile.tgz) to send it in a moment, without third parties involved and without having to make this file go across the globe multiple times.

The thing is: if you know what you actually need to do and have UNIX tools available, you can be insanely efficient.

Re: We are still early with the cloud

#142

Post by znpy »

rtpg wrote: post #92
gwm83 wrote: post #50

Earlier quoted context omitted.

Okay, tell me how you do it much better in k8s

you could easily imagine some Python-esque shell language (xonsh?) and have that be: print(os.path.exists("ssh://user@rsync.net:fileThatExists")) A boolean, a still (relatively) terse line of code. And when you end up wanting to do something even a bit more complicated it'll be a smooth gradient of difficulty. As with every discussion around shells, though, the amazing terseness of scripts is great, but every form of…

btw, assuming you're deploying on kubernetes:

you're omitting the 500 lines of yaml needed to deploy that, the tens of megabyte for the python base image (of which you'll waste more than 99.99%) and the tens of megabytes in bandwidth to the docker registry and to download it on the kubernetes worker... And the image wight should be accounted at least four times: worker node + 3 in the docker registry (ha and stuff).

it's incredibly inefficient if you just freaking want to know if a file exists.

Re: We are still early with the cloud

#143

Post by moduspol »

fleddr wrote: post #129

I started my career in simpler times. Developers would produce a zip and handed it over to an admin guy. Dev and Infra/Ops clearly separated. No CI, sometimes not even a build step. I understand the power and flexibility of the cloud but the critical issue is the dependency on super humans. Consider a FE or mobile app developer. They already greatly struggle just to keep up with development in their field. Next, you…

I'm just not sure how you define what goes into that zip in a way that does not make it substantially harder to solve tough problems than it would be to be familiar with cloud services.

Of course it'll cover you up to a point. If it's a CRUD web app that runs on a single server (or multiple stateless ones) and uses a relational database, you can have a zip file whose contents cover your needs. But if you have anything that justifies Kafka, Cassandra, or distributed storage, the "I'll just throw it over the fence to ops" paradigm isn't likely to fit as well.

Re: We are still early with the cloud

#144

Post by _gabe_ »

deckard1 wrote: post #32
_gabe_ wrote: post #24

Earlier quoted context omitted.

> Since reading the blog post's mention of Repl.it I went and downloaded their new Iphone app and used Modal.com to spin up 30-40 containers from a script doing sentiment analysis on ~30k movie reviews: IPhone processors run billions of cycles every second and are capable of running billions of instructions every second. I'm amazed that we've gone from "Run doom on my toaster", to "I can spin up 30-40(!) containers t…

> Doom was written for the IBM PC. This PC had a clock speed of 4.77MHz and 64Kb of RAM. well, not that PC. Doom required a 386 and 4MB of RAM. But you really wanted a 486 or Pentium to run it smoothly. Catacomb 3D, a real early id Software 3D game, actually would run on an 8088 XT. Also noteworthy, Doom was created on a NeXT computer, which was also a bit ahead of PC at the time. So there was a power differential ap…

Ah thank you for the correction! I should have looked this up more thoroughly beforehand :)

Re: We are still early with the cloud

#145

Post by dopidopHN »

verisimilitudes wrote: post #55
gwm83 wrote: post #50

Earlier quoted context omitted.

Okay, tell me how you do it much better in k8s

I don't use k8s or whatever garbage gets thrown around nowadays. I well remember my disgust when I learned AWS was just using UNIX virtual machines or whatever, rather than something simple, such as allowing people to run little functions floating in the void. I know nothing about AWS, but I've not been mistaken, right?

well, when I first saw a aws prompt in 2006 or so, I was relief that EC2 was a plain debian-like machine.

Also I was saving me a phone call with Dell and a sunday evening configuring some hardware.

No new bullshit paradigm to learn, I could shove my code in there with tool I knew and call it a day.

EC2 was a stepping stone and it worked well to get some stuff in the cloud.

Remember the early days when only non-critical, non-PII stuff were in the cloud? And how some companies "just can't use it because of X" ( or worse the dreaded Y )

Re: We are still early with the cloud

#146

Post by dmitriid »

29083011397778 wrote: post #136
dmitriid wrote: post #90

Earlier quoted context omitted.

> get peanuts from spotify even for non-trivial number of streams. Define "non-trivial". Easily 80-90% of all plays are by a handful of artists, the absolute vast majority of whom are owned by labels. Even if your non-trivial amount of listenings is in the tens of millions, it pales in comparison to Drake or Ed Sheeran. > until spotify pays from MY subscription the artists I listen to Your subscription is 15 dollars…

You seem to be about a decade out of date, per your wikipedia link. It states ~72% of music (down from ~88% in 2012) is owned by the big three (not four, after EMI was eaten by Sony in late 2011). Best of all, parent may have the right idea buying from Bandcamp. From your link: > These companies account for more than half of US market share. However, this has fallen somewhat in recent years, as the new digital enviro…

> It states ~72% of music (down from ~88% in 2012) is owned by the big three (not four, after EMI was eaten by Sony in late 2011).

It's hard to keep specifying the exact composition of the music scene, and the market share by the Big Four then Three then Four then Three again is fluctuating between 70 and 90 percent from year to year.

> I'm still optimistic and naive enough to think things may improve for artists.

The Big Ones have the industry in chokehold. Bandcamp is fine, but if you want to listen to something other than indies, you're stuck with the catalogs owned by the Big Ones. For example, https://www.sonymusicpub.com/en/songwriters Anything from Betales to AC/DC and from Enio Morricone to Dolly Parton is owned by Sony.

So you want to start a service and provide a service that provides both indies and this music? You will bow to industry's terms. If you have enough money and clout like Apple, you'll be able to negotiate better terms. Until then ¯\_(ツ)_/¯

Re: We are still early with the cloud

#147

Post by jodrellblank »

emptybits wrote: post #39
rektide wrote: post #16

> I'm excited for a world where a normal software developer doesn't need to know about... I'm not excited for this. There's a quote I cannot find that I miss greatly, but maybe a Whitehead quote or some such about Civilization being measured by that which it doesn't have to think about, that which it takes for granted. It's always struck me as powerful, but giving ourselves the ability to forget & un-learn does not t…

> There's a quote I cannot find that I miss greatly ... This is probably it: "Civilization advances by extending the number of important operations which we can perform without thinking of them." - Alfred North Whitehead (And I agree that depth is rewarding and civilization allowing us to forget things isn't always tempting!)

A similar quote from A.N. Whitehad: "By relieving the brain of all unnecessary work, a good notation sets it free to concentrate on more advanced problems, and, in effect, increases the mental power of the race."

https://mathshistory.st-andrews.ac.uk/Biographies/Whitehead/...

Re: We are still early with the cloud

#148

Post by CharlieDigital »

fleddr wrote: post #129

I started my career in simpler times. Developers would produce a zip and handed it over to an admin guy. Dev and Infra/Ops clearly separated. No CI, sometimes not even a build step. I understand the power and flexibility of the cloud but the critical issue is the dependency on super humans. Consider a FE or mobile app developer. They already greatly struggle just to keep up with development in their field. Next, you…

> Developers would produce a zip and handed it over to an admin guy

This is literally what the cloud is now for a fraction of the cost of the admin guy.

Current gen serverless containers basically deliver that promise of ease of use, scalability, and low cost.

For me, Google Cloud Run, Azure Container Apps, and AWS App Runner fulfill the promise of the cloud. Literally any dev can start building on these platforms with virtually no specialized knowledge.

https://www.youtube.com/watch?v=GlnEm7JyvyY

Re: We are still early with the cloud

#149

Post by theamk »

gumby wrote: post #119
otabdeveloper4 wrote: post #99

Earlier quoted context omitted.

"Treating remote resources truly abstractly" doesn't work in practice. Too many points of failure in our systems, and you really, really don't want to paper them over with abstraction if you want to build a fault-tolerant system.

That has been true for as long as I have been programming. But: Similar statements have been made about high level programming languages. Nowadays most devs don’t understand how the CPU works, but write on the top of a tower of abstractions and nobody bats an eye. Many of those abstractions are quite complex! I can imagine that the same could apply to certain kinds of network activities. Look at how ppl use http as a…

We cannot afford the seamless distributed systems, and I don't think we ever will.

I use Python because I don't care if adding two numbers is taking a microsecond instead of a nanosecond. But if netwotk call suddenly takes 1 sec instead of 10mS? well, thats a huge problem, let's add a memory cache and rack-level cache and a parallel fetch and a whole bunch of monitoring.

The local compute is growing much faster than internet, and even faster than local network. I sure hope we get better abstractions, but caring about remote vs local call is not going away.

Re: We are still early with the cloud

#150

Post by theamk »

gumby wrote: post #120
throwawaylinux wrote: post #74

Earlier quoted context omitted.

That is far, far more than transferring a byte of data. It's not the future though of course, this was well established in the 80s, it just so happens that it's a decent model for managing remote machines that outlasted over-engineered distributed designs like Plan9, VMS, MOSIX, etc. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like som…

> The great thing about Linux as the base layer is that it allows a commodity common ground with a very capable system that also facilitates more specialized layers to be implemented on top of it. The great thing about x86 instruction set as the base layer is that it allows a commodity common ground with a very capable system that also facilitates more specialized layers to be implemented on top of it. Only a handful…

Well, arm and mips exist, are are no less capable than x86. And most apps don't really care which ISA they use. So that point is false.

Now, what is that about alternate OS systems which can still do everything that Linux can, including handling network or service outages?

(*well, there are BSDs, but I don't think thats what you had in mind)

Post reply on HN