71–80 of 147 posts
If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…
No Bible quotes? I'm disappointed...
I can't believe how can someone so accomplished believe in God.
Earlier quoted context omitted.
If it's not Adderall I don't know. But, if I've ever focused for that long it's been because of Ritalin or Adderall.
Is 10 hours really _that_ strange? You are (hopefully) focusing 8 hours "straight" during work _every day_. If you watch Hotz's streams he takes small breaks to talk with chat and to meme around (just like everyone else during their work days) and he eats lunch and whatever (again just like everyone else). What I'm trying to say is that Hotz's isn't a superman on Adderall he is just working on stuff he is excited abo…
I attribute it to a younger brain, NO internet and NO fun distractions. At 42 I just don't see how I did it, and I know I could never be that focused. Just sitting still for 4 hours make me feel quasy now, and I need to use my physical body in some way.
I understand that the Python code is mostly driving faster low-level code, but I wonder how much time is effectively wasted by not using a lower-level language. From my experience with game engines, it often turns out to be a bad idea (for performance and maintainability) to mix C/C++ and Lua or C#.
But Python speed is one of the main motivations for a JS/TS based ML lib I’m working on: https://github.com/facebookresearch/shumai
If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…
I think its just hyperfocus.
Earlier quoted context omitted.
JAX is a DSL on top of XLA, instead of writing Python. Example: a JAX for loop looks like this: def summ(i, v): return i + v x = jax.lax.fori_loop(0, 100, summ, 5) A for loop in TinyGrad or PyTorch looks like regular Python: x = 5 for i in range(0, 100): x += 1 By the way, PyTorch also has JIT.
I've just tried making a loop in a jit-compiled function and it just worked: >>> import jax >>> def a(y): ... x = 0 ... for i in range(5): ... x += y ... return x ... >>> a(5) 25 >>> a_jit = jax.jit(a) >>> a_jit(5) DeviceArray(25, dtype=int32, weak_type=True)
x = 0
x += y
x += y
x += y
x += y
x += y
return x
It's funny that geohot/tinygrad chooses to not meet the PEP8 standards [0] just to stay on brand ( [0] https://peps.python.org/pep-0008/ [1] https://github.com/psf/black
No Bible quotes? I'm disappointed...
I can't believe how can someone so accomplished believe in God.
Yet, I had made this bashing comment about bible. IMHO anyone can believe in whatever they want. Christ., Islam, anything. I (and I would say every friend of mine) don't care about what do you believe in, but if you publicly preach some religion, prepare to be made fun of, or take a stand and try defend it your religion with arguments. But no blind faith here.
(Personally, if I like some religion it's Shinto.)
If anybody is dealing with procrastination watch George Hotz live streaming 10h straight working on this library [1][2]. Does he take some supplements to do this? There is even 19.5h stream [3]. Actually I have local obs setup to record myself, just instead of streaming I do recordings for my own inspection. Important part is to do the inspection after. It works wonders. [1] https://youtu.be/GXy5eVwnL_Q [2] https://m…
External motivation of having an audience would also help
Earlier quoted context omitted.
I've just tried making a loop in a jit-compiled function and it just worked: >>> import jax >>> def a(y): ... x = 0 ... for i in range(5): ... x += y ... return x ... >>> a(5) 25 >>> a_jit = jax.jit(a) >>> a_jit(5) DeviceArray(25, dtype=int32, weak_type=True)
It definitely works, JAX only sees the unrolled loop: x = 0 x += y x += y x += y x += y x += y return x The reason you might need `jax.lax.fori_loop` or some such is if you have a long loop with a complex body. Replicating a complex body many times means you end up with a huge computation graph and slow compilation.