We are still early with the cloud • Hacker News Board

Re: We are still early with the cloud

#81

Post by kortilla » Thu, Oct 20, 2022, 7:21 AM UTC

verisimilitudes wrote: post #76

throwawaylinux wrote: post #74

Earlier quoted context omitted.

That is far, far more than transferring a byte of data. It's not the future though of course, this was well established in the 80s, it just so happens that it's a decent model for managing remote machines that outlasted over-engineered distributed designs like Plan9, VMS, MOSIX, etc. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like som…

That is far, far more than transferring a byte of data. Educate me. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like some vapid marketing pitch. That was a quick response to what I thought would be better than administrating a UNIX system on an Amazon machine. Don't be an ass. I've no experience with the real distributed computing syst…

> Educate me.

You forgot an entire direction of input and focused on just a return code.

> The Linux kernel randomly kills processes when it starts exhausting memory. It's garbage.

No it doesn’t. It’s not random and it’s configurable. Turn off overcommit if you want malloc to fail when memory runs out.

Re: We are still early with the cloud

#82

Post by RamblingCTO » Thu, Oct 20, 2022, 7:24 AM UTC

Do we really want to develop in the cloud? My gut says no. I have no real opinion about that, but seems worth investigating. Is anybody working with a proper dev environment (no, sorry, a small react-only project doesn't cut it, I'm talking jvm driven, ran by docker kind of thing) in the cloud here?

Re: We are still early with the cloud

#83

Post by vanviegen » Thu, Oct 20, 2022, 7:25 AM UTC

verisimilitudes wrote: post #76

throwawaylinux wrote: post #74

Earlier quoted context omitted.

That is far, far more than transferring a byte of data. It's not the future though of course, this was well established in the 80s, it just so happens that it's a decent model for managing remote machines that outlasted over-engineered distributed designs like Plan9, VMS, MOSIX, etc. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like som…

That is far, far more than transferring a byte of data. Educate me. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like some vapid marketing pitch. That was a quick response to what I thought would be better than administrating a UNIX system on an Amazon machine. Don't be an ass. I've no experience with the real distributed computing syst…

> The Linux kernel randomly kills processes when it starts exhausting memory. It's garbage.

If you want to disable overallocation and the OOM killer, be my guest. You'll want to install some extra banks of RAM though.

Or is there a better way, provided by other OSes?

Re: We are still early with the cloud

#84

Post by throwawaylinux » Thu, Oct 20, 2022, 7:26 AM UTC

verisimilitudes wrote: post #76

throwawaylinux wrote: post #74

Earlier quoted context omitted.

That is far, far more than transferring a byte of data. It's not the future though of course, this was well established in the 80s, it just so happens that it's a decent model for managing remote machines that outlasted over-engineered distributed designs like Plan9, VMS, MOSIX, etc. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like som…

That is far, far more than transferring a byte of data. Educate me. It's much more meaningful and useful than "something simple" "running little functions floating in the void" which just sounds like some vapid marketing pitch. That was a quick response to what I thought would be better than administrating a UNIX system on an Amazon machine. Don't be an ass. I've no experience with the real distributed computing syst…

> Educate me.

Not sure if I'm being trolled... It finds a remote machine by name, and routes a connection to it. It authenticates and establishes a secure connection with the machine. It sends a command to the remote machine, the remote machine executes it, and the result is returned. It then puts the result into a form that can be used programmatically by the local shell.

> That was a quick response to what I thought would be better than administrating a UNIX system on an Amazon machine.

It was content-free.

> Don't be an ass.

What's good for the goose...

> I've no experience with the real distributed computing systems created before UNIX.

And already the expert. Impressive.

> No, that's the vapid marketing pitch.

No, that's the reality. That's why Amazon, Google, Azure, and everybody else offer it in their clouds and use it on their internal infrastructure.

> The Linux kernel randomly kills processes when it starts exhausting memory. It's garbage.

I'll take that over "running little functions floating in the void", it actually exists and works.

Re: We are still early with the cloud

#85

Post by kajecounterhack » Thu, Oct 20, 2022, 7:36 AM UTC

RamblingCTO wrote: post #82

Do we really want to develop in the cloud? My gut says no. I have no real opinion about that, but seems worth investigating. Is anybody working with a proper dev environment (no, sorry, a small react-only project doesn't cut it, I'm talking jvm driven, ran by docker kind of thing) in the cloud here?

Yeah, as another commenter mentioned, all Google SWEs have developed in the cloud for a long time. It allows you to write, run, and test code performantly on any computer with a browser. Some of OP's wishlist are realities at Google. E.g.

  - When I compile code, I want to fire up 1000 serverless
    container and compile tiny parts of my code in parallel.
  - When I run tests, I want to parallelize all of them. Or 
    define a grid with a 1000 combinations of parameters, or 
    whatever.

Build systems, testing infra, and cloud editing all need to be there for the magic to happen. When your cloud editor supports distributed builds & testing infra, and can be used by anyone, life is really good.

FWIW widely available cloud editing is also getting good with VSCode + LSP, if you don't want to pay Replit. Getting Bazel to do distributed builds instead of local builds is really annoying tho.

Re: We are still early with the cloud

#86

Post by islandnut » Thu, Oct 20, 2022, 7:37 AM UTC

I haven't seen so many people eagerly waiting for mainframes from 70s. As the author said - no IPs, no CIDR no NAT, no counting of ram so vast infinite resources, you only need a terminal to do everything remotely, it's only one development environement, mainframes for the masses this time.... And yet you still have to work within boundries because you or whoever you work for don't have money for infinite resources. It's a small contradiction omitted everywhere in these kinds of posts. But hey, lets come full circle into the 70s and welcome our ma...cough.. Cloud 2.0. If this happens there's hope. We will relive the microcomputer revolution after that. :)

Re: We are still early with the cloud

#87

Post by pojzon » Thu, Oct 20, 2022, 7:38 AM UTC

asim wrote: post #69

Everything Google is doing comes to the world 10 years later. Being inside Google is like seeing the future. They had all these technologies long ago, now it's just a case of timing and turning them into products. I've learned that sometimes the world just isn't ready for these advancements. The journey Google went on internally, everyone else has to go on for themselves. That said. I think we're still super early in…

After talking to manager at Google during an interview and him explaining to me that almost all tools at Google are home baked because most if not all services are so huge you wont be able to use opensource solutions for that.

Then I reminded myself about VictoriaMetrics that in benchmark outclassed Google Cloud Metrics by an order of magnitude.

Ppl at Google think they are the smartest (and often are) but in some cases they are simply outclassed so hard.

After this discussion decided Ill never ever want to work with ppl with such attitude.

Re: We are still early with the cloud

#88

Post by kajecounterhack » Thu, Oct 20, 2022, 7:46 AM UTC

pojzon wrote: post #87

asim wrote: post #69

Everything Google is doing comes to the world 10 years later. Being inside Google is like seeing the future. They had all these technologies long ago, now it's just a case of timing and turning them into products. I've learned that sometimes the world just isn't ready for these advancements. The journey Google went on internally, everyone else has to go on for themselves. That said. I think we're still super early in…

After talking to manager at Google during an interview and him explaining to me that almost all tools at Google are home baked because most if not all services are so huge you wont be able to use opensource solutions for that. Then I reminded myself about VictoriaMetrics that in benchmark outclassed Google Cloud Metrics by an order of magnitude. Ppl at Google think they are the smartest (and often are) but in some ca…

> in some cases they are simply outclassed so hard.

I think this is true in some cases, but Google has been okay at adopting external vendors where internal tools aren't keeping up. And in some cases, folks actually hate the external replacement and miss the google-built tool. So YMMV.

Companies like FB, Google, etc are large enough and have specific enough needs that sometimes they really do have to build their own thing. Buck had to be built by Facebook because Bazel wasn't open source yet, and well that's one example of something from Google outclassing all the competition for organizations that need anything like it.

In re arrogant people, they exist at all organizations whether it's warranted or not. I wouldn't let a random peon at Google affect your perception of the organization. You'll see similar behavior from companies at all sizes so it's not really a telling signal.

Re: We are still early with the cloud

#89

Post by CSDude » Thu, Oct 20, 2022, 7:53 AM UTC

hulitu wrote: post #52

mgraczyk wrote: post #34

Interestingly, at Google the typical developer workflow (google3) is very cloud native. Most devs write code in VS code in the browser. Many (most?) devs don't have a physical desktop any more, just a cloud VM. The code lives in a network mounted filesystem containing a repository. The repository is hosted remotely (everyone can see edits you make to any file nearly immediately). Builds are done remotely with a share…

> Many (most?) devs don't have a physical desktop any more, That would explain the (bad) design of their software.

I prefer the ability to run and debug locally coupled with a good IDE. I know VSCode is popular, people customize the shit out of Vim, but IntelliJ just works for me when I'm writing Java, Kotlin or Typescript/React. Refactor and debug is not comparable. And I know most think its hard on resources, but we have 200k lines of code yet and it works with 16GB M1 Air very well leaving more than enough spare resources for the system.

Re: We are still early with the cloud

#90

Post by dmitriid » Thu, Oct 20, 2022, 7:55 AM UTC

minusf wrote: post #78

dmitriid wrote: post #49

Earlier quoted context omitted.

Spotify pays up to 70% of their revenue to copyright holders. So, your beef should be with them, not Spotify. Which you should already be aware of if you truly are a music producer. However, it costs you nothing to bash Spotify. It may cost you your career to bash the actual greedy leeches who control the money flows in music.

there are countless musicians who own the copyright to their own stuff and get peanuts from spotify even for non-trivial number of streams. until spotify pays from MY subscription the artists I listen to, they will not see money from me. bandcamp all the way.

> get peanuts from spotify even for non-trivial number of streams.

Define "non-trivial". Easily 80-90% of all plays are by a handful of artists, the absolute vast majority of whom are owned by labels.

Even if your non-trivial amount of listenings is in the tens of millions, it pales in comparison to Drake or Ed Sheeran.

> until spotify pays from MY subscription the artists I listen to

Your subscription is 15 dollars a month. Yup. That is definitely not peanuts when spread over all the stuff you listen to.

Edit: you could be one of the few people who listen to the same band all the time, but that's not representative of people's listening habits.

Edit2:

> there are countless musicians who own the copyright to their own stuff

80-90% of all world music is owned by four companies [1]. This amounts to about ~99% of music people listen to. The "countless musicians" make up a long tail that is barely a blip on the radar.

[1] https://en.wikipedia.org/wiki/Music_industry#Consolidation