Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

4000x Speedup in Reinforcement Learning with Jax

chrislu.page

Post reply on HN
Print view View original

11–20 of 32 posts

Re: 4000x Speedup in Reinforcement Learning with Jax

#11

Post by yieldcrv »

hyperbovine wrote: post #7
percentcer wrote: post #6

Reminds me of this evergreen tweet from ryg: https://mobile.twitter.com/rygorous/status/12712968344392826... if you made something 2x faster, you might have done something smart if you made something 100x faster, you definitely just stopped doing something stupid

Meh. This tweet is a lot less clever than it seems. Shave a factor of n off the complexity of your your algorithm, as happens regularly in CS and informatics, and have all the 1000x speedups you want.

I love the quote, I’ll be using it the rest of my life

Re: 4000x Speedup in Reinforcement Learning with Jax

#12

Post by ssivark »

From what I understand of Jax, it feels somewhat similar in flavor to Julia, but trying to live with the language constraints (and ecosystem benefits) of Python.

I wonder how Julia is placed for running reinforcement learning algorithms (efficiently) — particularly in cases when the “environment” is nicely wrapped in Python to fit some standardized interface.

Re: 4000x Speedup in Reinforcement Learning with Jax

#13

Post by dan-robertson »

hyperbovine wrote: post #7
percentcer wrote: post #6

Reminds me of this evergreen tweet from ryg: https://mobile.twitter.com/rygorous/status/12712968344392826... if you made something 2x faster, you might have done something smart if you made something 100x faster, you definitely just stopped doing something stupid

Meh. This tweet is a lot less clever than it seems. Shave a factor of n off the complexity of your your algorithm, as happens regularly in CS and informatics, and have all the 1000x speedups you want.

I don’t think it seems clever so much as pithy? I don’t think it’s necessarily about algorithms so much as it is a claim that the main way to be 1000x slower is by doing 999x times too much work or waiting.

An example that is basically unrelated to complexity theory is something like talking to a distant service but keeping a small number of requests because eg you were worried about load or didn’t notice you were waiting for acks or create a new tcp connection for each request or have a small sendbuf or somehow send way too fast and get rate limited and need to retry.

Re: 4000x Speedup in Reinforcement Learning with Jax

#16

Post by amscanne »

hyperbovine wrote: post #7
percentcer wrote: post #6

Reminds me of this evergreen tweet from ryg: https://mobile.twitter.com/rygorous/status/12712968344392826... if you made something 2x faster, you might have done something smart if you made something 100x faster, you definitely just stopped doing something stupid

Meh. This tweet is a lot less clever than it seems. Shave a factor of n off the complexity of your your algorithm, as happens regularly in CS and informatics, and have all the 1000x speedups you want.

I believe the underlying point is that the bigger the N, the more obvious it probably is. If you’re in 1000x territory, there’s a strong likelihood that the improvement is so obvious that doing the opposite might be called “stupid”.

Re: 4000x Speedup in Reinforcement Learning with Jax

#17

Post by xyzzy4747 »

How does this compare with PyTorch / Tensorflow / etc.? Obviously doing heavy data processing on the GPU will have a large speedup compared to a single thread on the CPU.

It's almost like the author is claiming credit for creating Nvidia, when in fact he is just calling its APIs.

Re: 4000x Speedup in Reinforcement Learning with Jax

#18

Post by Buttons840 »

ssivark wrote: post #12

From what I understand of Jax, it feels somewhat similar in flavor to Julia, but trying to live with the language constraints (and ecosystem benefits) of Python. I wonder how Julia is placed for running reinforcement learning algorithms (efficiently) — particularly in cases when the “environment” is nicely wrapped in Python to fit some standardized interface.

I've done some RL experiments in Julia, and having all the in-between be fast was helpful; I saw significant speed increases. That said, Julia was probably just compensating for my own stupidity, because I was converting my environment objects into tensors over and over and over.

Re: 4000x Speedup in Reinforcement Learning with Jax

#19

Post by whatshisface »

squeaky-clean wrote: post #8
hyperbovine wrote: post #7

Earlier quoted context omitted.

Meh. This tweet is a lot less clever than it seems. Shave a factor of n off the complexity of your your algorithm, as happens regularly in CS and informatics, and have all the 1000x speedups you want.

If you shave a factor of n off of your algorithm, it usually isn't the same algorithm anymore. That's what they mean, the previous algorithm choice was "stupid" and you've stopped doing something stupid.

With that kind of creative interpretation almost any statement can be true. :)

Re: 4000x Speedup in Reinforcement Learning with Jax

#20

Post by m00x »

xyzzy4747 wrote: post #17

How does this compare with PyTorch / Tensorflow / etc.? Obviously doing heavy data processing on the GPU will have a large speedup compared to a single thread on the CPU. It's almost like the author is claiming credit for creating Nvidia, when in fact he is just calling its APIs.

I don't get that sense at all.

You could do the same with tensorflow and pytorch, but in my experience, with more difficulty since they're more opinionated about how you should do your operations.

JAX is definitely easier to do things that aren't on rails.

Post reply on HN