Andrej Karpathy: Software in the era of AI [video]

Post by mkw5053 » Thu, Jun 19, 2025, 11:01 PM UTC

jeremyjh wrote: post #616

mkw5053 wrote: post #589

This DevOps friction is exactly why I'm building an open-source "Firebase for LLMs." The moment you want to add AI to an app, you're forced to build a backend just to securely proxy API calls—you can't expose LLM API keys client-side. So developers who could previously build entire apps backend-free suddenly need servers, key management, rate limiting, logging, deployment... all just to make a single OpenAI call. Any…

Do you think Firebase and Superbase are working on this? Good luck but to me it sounds like a platform feature, not a standalone product.

Post by throw234234234 » Thu, Jun 19, 2025, 11:08 PM UTC

elif wrote: post #308

osigurdson wrote: post #296

Earlier quoted context omitted.

I'm not sure that AI code has to be sloppy. I've had some success with hand coding some examples and then asking codex to rigorously adhere to prior conventions. This can end up with very self consistent code. Agree though on the "pick the best PR" workflow. This is pure model training work and you should be compensated for it.

Yep this is what Andrej talks about around 20 minutes into this talk. You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. The second you start being vague, even if it WOULD be clear to a person with common sense, the LLM views that vagueness as a potential aspect of it's own creative liberty.

Post by fHr » Thu, Jun 19, 2025, 11:21 PM UTC

Post by jchrisa » Thu, Jun 19, 2025, 11:27 PM UTC

diggan wrote: post #502

necrodome wrote: post #451

Earlier quoted context omitted.

Thanks for the benefit of the doubt. I typed that in a hurry, and it didn’t come out the way I intended. We had the idea that there’s a class of apps [1] that could really benefit from our tooling - mainly Fireproof, our local-first database, along with embedded LLM calling and image generation support. The app itself is open source, and the hosted version is free. Initially, there was no login or signup - you could…

My complaint isn't about that you need to protect it with a login/signup, but where in the process you put that login/signup. Put it before letting people enter text, rather than once they've entered text and pressed the button, and people won't feel mislead anymore.

Post by instig007 » Thu, Jun 19, 2025, 11:29 PM UTC

hellovai wrote: post #599

solaire_oa wrote: post #411

Earlier quoted context omitted.

I also think that structured outputs are criminally underused, but it isn't perfect... and per your example, it might not even be good, because I've done something similar. I was trying to make a decent cocktail recipe database, and scraped the text of cocktails from about 1400 webpages. Note that this was just the text of the cocktail recipe, and cocktail recipes are comparatively small. I sent the text to an LLM fo…

have you tried schema-aligned parsing yet? the idea is that instead of using JSON.parse, we create a custom Type.parse for each type you define. so if you want a: class Job { company: string[] } And the LLM happens to output: { "company": "Amazon" } We can upcast "Amazon" -> ["Amazon"] since you indicated that in your schema. https://www.boundaryml.com/blog/schema-aligned-parsing and since its only post processing, t…

    so if you want a:

       class Job { company: string[] }

    We can upcast "Amazon" -> ["Amazon"] since you indicated that in your schema.