Devpod: Remote development environment at Uber • Hacker News Board

Re: Devpod: Remote development environment at Uber

#211

Post by debosmit » Mon, Dec 19, 2022, 3:14 PM UTC

AJRF wrote: post #191

Anyone from Uber here that works on this? You ever expect a DevPod for iOS developers? I've been toying with idea of doing this in work for our devs on Intel MacBooks, using -> https://github.com/sickcodes/Docker-OSX , which enables local running on iOS devices. I think it's hampered by licensing though, maybe that's why it's not used or mentioned in this post.

We’ve thought about this quite deeply at DevZero. Yes, the iOS developer experience can be quite hampered on the local env and requires users to frequently get beefy local laptops/machines. I saw this first hand at Uber from when crowdstrike was rolled out broadly.

We have support for AWS-based Mac VMs on DevZero but we don’t find our customers having their biggest issues related to iOS dev yet (we also target enterprise cos that have a vast diversity of tools, mostly backend and front end)

Re: Devpod: Remote development environment at Uber

#212

Post by KronisLV » Mon, Dec 19, 2022, 3:37 PM UTC

yjftsjthsd-h wrote: post #133

paxys wrote: post #89

Earlier quoted context omitted.

Sure but most regular consumer or business applications don't really need that level of graphical power. Rendering a menu or button or blurb of text locally from some layout language is always going to be more performant than streaming raw pixels from a server.

It would be, if everyone agreed on a toolkit. As it as, at least X11, AFAIK Wayland, and Windows/RDP ended up just throwing pixels over the wire because every program renders text/menus/whatever differently.

I wonder what the world would look like if we decoupled what something is supposed to be and what data it's supposed to have, from how it actually looks and acts. Say, some "lowest common denominator" that works reasonably across all platforms.

For example, I'd say that I need:

  A dropdown for a single option, with options: A, B, C

And let the device itself decide what needs to be displayed in the native GUI toolkit. Then just send that specification over the wire, instead of needlessly wasting the bandwidth on lots of pixels.

Actually, I think I'm just describing an analogue to HTML with the equivalent of CSS provided by the platform, but for native desktop toolkits (hopefully without the complexity of a browser engine).

Re: Devpod: Remote development environment at Uber

#213

Post by jeffbee » Mon, Dec 19, 2022, 3:48 PM UTC

deng wrote: post #160

gandalfgeek wrote: post #22

Back in the early 2000s, Sun ("the network is the computer") had a similar solution that worked seamlessly for most of their software org-- the Sun Ray. https://en.wikipedia.org/wiki/Sun_Ray It was a network terminal. Your files and entire session were on the server. Your “local” terminal consisted only of a network interface and enough compute power to display your session. The way they had it set up was that you co…

I worked on a Sun Ray. I and many colleagues absolutely hated it. These tiny machines were just way too slow to handle even the tiny amount of work they had to do. Also, everyone knows that X over network is just not made for modern applications ("modern" in the year 2000!). I worked with Matlab, and had a lot of fun trying to rotate 3D plots with a few thousand points. It was just unbearable. Then of course the "sin…

> everyone knows that X over network is just not made for modern applications ("modern" in the year 2000!). I worked with Matlab

No. I don't think everyone knows that at all. At my university we used X terminals connected to a central Sun SPARCcenter 2000 over ethernet and it was fine. We used MATLAB, Maple V, SAS, etc etc.

Re: Devpod: Remote development environment at Uber

#214

Post by dilyevsky » Mon, Dec 19, 2022, 4:17 PM UTC

rtpg wrote: post #156

CrimsonRain wrote: post #150

Earlier quoted context omitted.

Can you explain how a monorepo for microservices is a tooling nightmare and bandwidth sink? In my experience, a monorepo significantly makes things easier and saves time everyday for all developers involved when compared with multi repo.

The biggest issue is that every CI provider speaks with "repo=project" language, so your way forward with monorepos is using stuff like Bazel. Bazel is extremely cool! But you end up with, like, a handful of people on the team who can write Bazel and eveyone else cargo culting their way through it. There are a lot of JS "monorepo management" tools but they all seem concerned about the release phase for a lot of libra…

> Bazel is extremely cool! But you end up with, like, a handful of people on the team who can write Bazel and eveyone else cargo culting their way through it.

It's not that difficult and docs are outstanding. It can and it will be worse with your own Bash-isms that likely won't have any docs at all.

Re: Devpod: Remote development environment at Uber

#215

Post by Groxx » Mon, Dec 19, 2022, 4:49 PM UTC

Galanwe wrote: post #165

rtpg wrote: post #159

Earlier quoted context omitted.

Monorepo lets me have a _single_ Github PR with the changes across 4 projects in front of me. To replicate what you're talking about in a multirepo setup, you have to introduce changesets in 4 different places, then merge them in the right order, make a bunch of tiny releases, update commit hashes to point to the right one, and do a lot of other bookkeeping because in this house we keep our projects separated.

Indeed, the small additional bookkeeping is the price to pay. I'm quite used to it, so I don't even see it anymore. Is that all? These 5 less minutes of bookkeeping are the killer feature of monorepos? Because, as far as I see it, there's an insane amount of engineering to make a monorepo work even at small scale. Are these 5 minutes per multi-project PR worth it? I wouldn't be surprised these 5 minutes are largely o…

Yeah. They reduce the time spent on rare (but relatively valuable!) events, and massively increase time spent on all the little things you do dozens or hundreds of times per day. It adds up quite badly for everyone except the monorepo managing teams.

Re: Devpod: Remote development environment at Uber

#216

Post by Tostino » Mon, Dec 19, 2022, 5:24 PM UTC

krembanan wrote: post #180

Galanwe wrote: post #170

Earlier quoted context omitted.

Thanks for your answer. How does CI work in practice? how do you avoid rebuilding and retesting the whole repo at every change? That would be an insane waste of resource and time.

By only testing and building packages that has a dependency on the changed package.

And how do you determine that dependency structure easily? What tooling helps with this?

Re: Devpod: Remote development environment at Uber

#217

Post by nithayakumar » Mon, Dec 19, 2022, 5:35 PM UTC

0xB31B1B wrote: post #34

EastSmith wrote: post #24

We are moving to such cloud environment, and it makes me sick. Maybe you need to dockerise Mongo, MySql and 5 other dependencies - I can get this, but I don't get it why the rest the code should still be running in the cloud. Python, Rails, Node? Why? Developers should be able to run 1 shell commands to install node. Dev experience excuses are just excuses for a bad setup. So, fix your setup, please. Not being able t…

Its a lot easier to manage dev envs if everyone is literally using cloned ec2 instances that can be remotely fixed/managed by devexp teams. These type of projects are a big step forward IMO

Thats definitely one way to do it. Im a founder at a remote dev infra startup (usenimbus.com) and this was the model we started with because it was just so simple. But we quickly learned theres no one size fit all solution so expanded into supporting containers, terraform, etc.

Re: Devpod: Remote development environment at Uber

#218

Post by lhorie » Mon, Dec 19, 2022, 5:35 PM UTC

yjftsjthsd-h wrote: post #143

lhorie wrote: post #114

Earlier quoted context omitted.

One of the reasons we developed devpods was because our setup scripts were unmanageable. Instructions to new employees would say things like "run this thing, scroll up past dozens of pages of stdout noise and manually deal w/ the errors buried therein by looking up relevant FAQs in some doc somewhere" The scripts would touch every technology imaginable, from brew to npm to arc (phabricator's cli) to proprietary tools…

> run this thing, scroll up past dozens of pages of stdout noise and manually deal w/ the errors buried therein by looking up relevant FAQs in some doc somewhere Okay, but if you can define/script your environment enough to run in a pod, couldn't you just run that locally? You already have to solve the manual steps either way...

It's the same convo as docker: can't one just use setup scripts for reproducible envs? In theory yes, but in practice, theory and practice aren't the same thing.

It's easier to apply best practices to a greenfield project written in a modern language (hence devpods) than trying to comb through a decade+ of tech debt written in bash.

Re: Devpod: Remote development environment at Uber

#219

Post by nithayakumar » Mon, Dec 19, 2022, 5:47 PM UTC

faizshah wrote: post #25

I like this workflow, use something like mutagen to sync from your local environment to the remote machine. For me, I prefer jetbrains IDEs so I tried using jetbrains gateway, it’s really buggy and slow. I prefer to use jetbrains ides on my local machine and use VSCode remote for interacting with the remote machine. I wonder though, is it worth it to setup a remote dev machine if you have an M1 Mac? I think you can g…

Yeah - we've (usenimbus.com - a company in this space) have heard the same about Jetbrains IDEs. You should check out our extension (or others') because there's been a lot of work put into making performance better.

M1s are a mixed bag for this way of working. Pre-M1, devs were running into local computing power issues. Post-M1, more compatibility and stability issues.

Re: Devpod: Remote development environment at Uber

#220

Post by lhorie » Mon, Dec 19, 2022, 5:51 PM UTC

Galanwe wrote: post #170

lhorie wrote: post #75

Earlier quoted context omitted.

I maintain the web monorepo at Uber, so I think I can give some context. Monorepos allows us to centralize important dependency upgrades. E.g. fixing log4j vulns is a lot easier when you can patch everything simultaneously. Same for tzdata (2022g gave very little heads up) Auditing for npm supply chain attacks was a lot simpler in monorepo than microrepos. Etc. Monolithic version control doesn't have to mean monolith…

Thanks for your answer. How does CI work in practice? how do you avoid rebuilding and retesting the whole repo at every change? That would be an insane waste of resource and time.

You use a build system. There are a number of open source ones, we use Bazel. In a nutshell, it keeps track of dependency graphs, so you can test only affected projects.

It can also cache execution of unchanged transitive steps, so you can skip builds/tests that were already run previously (e.g. you could skip most of a large 2nd CI run if all you did in a code review was edit one file.

You can also parallelize execution across cloud nodes.