Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#31

Post by dovholuknf »

williamstein wrote: post #30
dovholuknf wrote: post #24

Earlier quoted context omitted.

Interesting idea. I'm not very well-versed in training models or LLMs or even Jupyter Notebooks, but the comment about port forwarding SSH caught my eye since I work on a free, open source zero-trust overlay network (OpenZiti). I tried to find some information about moonglow under the hood / how it worked but didn't succeed. If you're interested, you might find embedding OpenZiti into Moonglow a pretty compelling alt…

Is it possible to use OpenZiti with Runpod? Their execution environment is very locked down, which might make ssh the only option.

I don't actually know. I'll go poke with Runpod for a few and see :)

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#32

Post by bblcla »

AnotherGoodName wrote: post #19

We can literally transpile and run everything client side these days. You can run a recompiled quake 3 inside your browser. Why is a hosted notebook anything other than a static html+js+css site that runs behind a cdn (effectively free to host)? I suspect that’s a matter of time right?

A lot of the people we've talked to who get the most value out of remote compute are doing really intensive stuff - they need server-level resources far beyond what you can find on a consumer laptop!

Hopefully someday you'll have 8 H100s on your Macbook, but I think we're still a long way away from that.

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#33

Post by dovholuknf »

williamstein wrote: post #30
dovholuknf wrote: post #24

Earlier quoted context omitted.

Interesting idea. I'm not very well-versed in training models or LLMs or even Jupyter Notebooks, but the comment about port forwarding SSH caught my eye since I work on a free, open source zero-trust overlay network (OpenZiti). I tried to find some information about moonglow under the hood / how it worked but didn't succeed. If you're interested, you might find embedding OpenZiti into Moonglow a pretty compelling alt…

Is it possible to use OpenZiti with Runpod? Their execution environment is very locked down, which might make ssh the only option.

I poked at it a bit but there was no free trial period. I know a bunch of people are using OpenZiti and zrok for Jupyter notebooks in general... Here's a blog I saw not long back that might help but I wasn't able to prove/test/try it... (sorry)

https://www.pogs.cafe/software/tunneling-sagemaker-kaggle

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#34

Post by bblcla »

whinvik wrote: post #29

Feels a bit like what Databricks already does with Dbx etc.

I'm not super familiar with dbx (though its docs at https://docs.databricks.com/en/archive/dev-tools/dbx/dbx.htm... suggest it's deprecated).

However, looking at its replacement here (https://docs.databricks.com/en/dev-tools/bundles/index.html) - I think we're trying to solve the same problems at different levels. My guess is Databricks is the right solution for big teams that need well-defined staging/prod/dev environment. We're targeting smaller teams that might be doing more of their own devops or are still at the 'using a bash script to run notebooks remotely' stage.

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#35

Post by bblcla »

dovholuknf wrote: post #24
bblcla wrote: post #15

Earlier quoted context omitted.

Ah, yeah, I misspoke, sorry. I was aware of that feature, but everyone I've talked to said it's so annoying to use they basically never use it, so I didn't think it was worth mentioning. The big reason it's annoying is because (I believe) Colab still only lets you connect to runtimes running on your computer - which is why at the end at the end of that article they suggest using SSH port forwarding if you want to con…

Interesting idea. I'm not very well-versed in training models or LLMs or even Jupyter Notebooks, but the comment about port forwarding SSH caught my eye since I work on a free, open source zero-trust overlay network (OpenZiti). I tried to find some information about moonglow under the hood / how it worked but didn't succeed. If you're interested, you might find embedding OpenZiti into Moonglow a pretty compelling alt…

Cool! We actually don't do port forwarding over SSH, we do it over an ngrok-like solution that we forked/modified. I looked at a few options while we were designing this, including Tailscale and ngrok, but none of them exactly suited our needs, and the pricing would have been prohibitive for something that's a pretty core part of our product.

OpenZiti looks really cool though - I'll take a look!

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#36

Post by qrkourier »

williamstein wrote: post #30
dovholuknf wrote: post #24

Earlier quoted context omitted.

Interesting idea. I'm not very well-versed in training models or LLMs or even Jupyter Notebooks, but the comment about port forwarding SSH caught my eye since I work on a free, open source zero-trust overlay network (OpenZiti). I tried to find some information about moonglow under the hood / how it worked but didn't succeed. If you're interested, you might find embedding OpenZiti into Moonglow a pretty compelling alt…

Is it possible to use OpenZiti with Runpod? Their execution environment is very locked down, which might make ssh the only option.

At a glance, the RunPod's serverless and pod options would probably work well with OpenZiti. I didn't explore their vLLM option.

Using OpenZiti w/ Serverless probably means integrating an OpenZiti SDK with your serverless application. That way, it'll connect to the OpenZiti network every time it spawns.

The SDK option works anywhere you can deploy your application because it doesn't need any sidecar, agent, proxy, etc, so it's definitely the most flexible and I can give you some examples if you mention the language or framework you're using.

The pod option says "container based" so it'll take some investigation to find out if an OpenZiti sidecar or other tunneling proxy is an option. Would you be looking to publish something running in RunPod (the server is in RunPod), or access something elsewhere from a RunPod pod (the client is in RunPod), or both?

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#37

Post by OvbiousError »

dinobones wrote: post #27
AnotherGoodName wrote: post #19

We can literally transpile and run everything client side these days. You can run a recompiled quake 3 inside your browser. Why is a hosted notebook anything other than a static html+js+css site that runs behind a cdn (effectively free to host)? I suspect that’s a matter of time right?

I too share the dream of transpiling everything/WebAssembly/everything can run everywhere, but we're pretty far. For a lot of ML/AI workloads and tasks, Python is just a binding for underlying C/C++. It's already a nightmare to try to reproduce any ML/AI paper, pip breaks 3 times, incompatible peer deps, some obscure library emits an obscure CLANG error that means I need to brew install some libwhatever, etc... I don…

The performance of browser wasm is also incomparable to the same code compiled into a native binary.

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#39

Post by bblcla »

yanniszark wrote: post #38

Great work! Was wondering if you deal with transferring the python environment remotely. Usually a large part of the difficulty is dealing with dependencies.

We make sure the remote containers have CUDA/Pytorch/Numpy/Matplotlib set up if you're using a GPU-based machine. It's actually far easier for me to run ML-based code through Moonglow now than on my Macbook - it's really nice to start with a clean environment every time, instead of having to deal with dependency hell.

We don't yet transfer the python environment on the self-serve options, though for customers on AWS we'll help them create and maintain images with the packages they need.

I do have some ideas for making it easy to transfer environments over - it would probably involve letting people specify a requirements.txt and some apt dependencies and then automatically creating/deploying containers around that. Your idea of actually just detecting what's installed locally is pretty neat too, though.

Re: Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

#40

Post by tomjakubowski »

switchbak wrote: post #25
AnotherGoodName wrote: post #20

Earlier quoted context omitted.

In fact a quick google: https://github.com/jtpio/jupyterlite Yay I really can have serverless notebooks! Not just an easy to manage server environment but literally a static html file that can be passed around and runs the full notebook environment. It’s weird it was ever done any other way.

Does that run with your GPU in the way this service offers?

No, it doesn't. But unlike the marketing buzzword "serverless", it is honest about not requiring a server.

Pyodide GPU support is a ways away, but it is theoretically possible once WebGPU is stable. https://github.com/pyodide/pyodide/issues/1911

Post reply on HN