Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

github.com

Post reply on HN
Print view View original

1–10 of 33 posts

Re: Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

#3

Post by desideratum »

bdbdbdb wrote: post #2

Dumb question - and I'm not trying diminish the achievement here, I just genuinely don't understand: Why would people want to spend $200 to train a coding model when there are free coding models?

This is a great question. You definitely aren't training this to use it, you're training it to understand how things work. It's an educational project, if you're interested in experimenting with things like distributed training techniques in JAX, or preference optimisation, this gives you a minimal and hackable library to build on.

Re: Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

#5

Post by wwfn »

Tangential (but topical in that "The threat is comfortable drift toward not understanding what you're doing" is also on the front page):

Is the generated python code in the example wrong?

The prompt

> Develop a Python function that removes any falsey values from a list. Return the modified list without creating a new one.

Is answered with list comprehension, which makes a new list and leaves the original unmodified (never mind that the *args input necessarily can't be a modifiable list?)

   def remove_falsey_values(*args): return [val for val in args if val]
Whereas I'd expect something like

    def remove_falsey_values(l):
          for i in reversed(range(len(l))):
               if not l[i]: l.pop(i)
          # returned list is linked to input l 
          return l

    a = [1, 0, False, 'foo']
    x = remove_falsey_values(a)
    x[0] = 2
    print(a) # [2,'foo']

Re: Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

#6

Post by hecanjog »

wwfn wrote: post #5

Tangential (but topical in that "The threat is comfortable drift toward not understanding what you're doing" is also on the front page): Is the generated python code in the example wrong? The prompt > Develop a Python function that removes any falsey values from a list. Return the modified list without creating a new one. Is answered with list comprehension, which makes a new list and leaves the original unmodified (…

It doesn't fit the requirement to modify the list in place, but the prompt itself contradicts the requirements by asking explicitly for the implementation to use *args and a list comprehension.

Re: Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

#7

Post by wwfn »

hecanjog wrote: post #6
wwfn wrote: post #5

Tangential (but topical in that "The threat is comfortable drift toward not understanding what you're doing" is also on the front page): Is the generated python code in the example wrong? The prompt > Develop a Python function that removes any falsey values from a list. Return the modified list without creating a new one. Is answered with list comprehension, which makes a new list and leaves the original unmodified (…

It doesn't fit the requirement to modify the list in place, but the prompt itself contradicts the requirements by asking explicitly for the implementation to use *args and a list comprehension.

Ahh I didn't see the full original prompt -- it's overflowing into a horz scroll for me. I thought it was the "critique loop" that injected the *args requirement. I guess garbage in, garbage out. Still unfortunate example to use.

Re: Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

#9

Post by wongarsu »

desideratum wrote: post #3
bdbdbdb wrote: post #2

Dumb question - and I'm not trying diminish the achievement here, I just genuinely don't understand: Why would people want to spend $200 to train a coding model when there are free coding models?

This is a great question. You definitely aren't training this to use it, you're training it to understand how things work. It's an educational project, if you're interested in experimenting with things like distributed training techniques in JAX, or preference optimisation, this gives you a minimal and hackable library to build on.

It's also a great base for experimentation. If you have an idea for an architecture improvement you can try it for $36 on the 20 layer nanocode setting, then for another $200 see how it holds up on the "full scale" nanocode

Kaparthy's notes on improving nanochat [1] are one of my favorite blog-like things to read. Really neat to see which features have how much influence, and how the scaling laws evolve as you improve the architecture

There's also modded-nanogpt which turns the same kind of experimentation into a training speedrun (and maybe loses some rigor on the way) [2]

1 https://github.com/karpathy/nanochat/blob/master/dev/LOG.md

2 https://github.com/kellerjordan/modded-nanogpt

Re: Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs

#10

Post by __s »

wwfn wrote: post #5

Tangential (but topical in that "The threat is comfortable drift toward not understanding what you're doing" is also on the front page): Is the generated python code in the example wrong? The prompt > Develop a Python function that removes any falsey values from a list. Return the modified list without creating a new one. Is answered with list comprehension, which makes a new list and leaves the original unmodified (…

    def remove_falsey_values(l):
          l[:] = (x for x in l if x)
Post reply on HN