Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy: Software in the era of AI [video]

youtube.com

Post reply on HN
Print view View original

801–810 of 827 posts

Re: Andrej Karpathy: Software in the era of AI [video]

#801

Post by abdullin »

bayindirh wrote: post #696
abdullin wrote: post #600

Earlier quoted context omitted.

Here is another way to look at the problem. There is a team of 5 people that are passionate about their indigenous language and want to preserve it from disappearing. They are using AI+Coding tools to: (1) Process and prepare a ton of various datasets for training custom text-to-speech, speech-to-text models and wake word models (because foundational models don't know this language), along with the pipelines and tool…

What you are describing is another application. My comment was squarely aimed at "vibe coding". Protecting and preserving dying languages and culture is a great application for natural language processing. For the record, I'm neither against LLMs, nor AI. What I'm primarily against is, how LLMs are trained and use the internet via their agents, without giving any citations, and stripping this information left and rig…

I think there are two different layers that get frequently mixed.

(1) LLMs as models - just the weights and an inference engine. These are just tools like hammers. There is a wide variety of models, starting from transparent and useless IBM Granite models, to open-weights Llama/Qwen to proprietary.

(2) AI products that are built on top of LLMs (agents, RAG, search, reasoning etc). This is how people decide to use LLMs.

How these products display results - with or without citations, with or without attribution - is determined by the product design.

It takes more effort to design a system that properly attributes all bits of information to the sources, but it is doable. As long as product teams are willing to invest that effort.

Re: Andrej Karpathy: Software in the era of AI [video]

#802

Post by omneity »

amelius wrote: post #766
dingnuts wrote: post #765

Earlier quoted context omitted.

that would require actually curating the training data and eliminating sources that contain casual conversation too expensive since those are all licensed sources, much easier to train on Reddit data

Just ask an LLM to remove the personality from the training data. Then train a new LLM on that.

It will work, but at the scale needed for pretraining you are bound to have many quality issues that will destroy your student model, so your data cleaning process better be very capable.

One way to think of it is that any little bias or undesirable path in your teacher model will be amplified in the resulting data and is likely to become over represented in the student model.

Re: Andrej Karpathy: Software in the era of AI [video]

#803

Post by hellovai »

solaire_oa wrote: post #653
hellovai wrote: post #599

Earlier quoted context omitted.

have you tried schema-aligned parsing yet? the idea is that instead of using JSON.parse, we create a custom Type.parse for each type you define. so if you want a: class Job { company: string[] } And the LLM happens to output: { "company": "Amazon" } We can upcast "Amazon" -> ["Amazon"] since you indicated that in your schema. https://www.boundaryml.com/blog/schema-aligned-parsing and since its only post processing, t…

Ok. Tried it, I'm not super impressed. Client: Ollama (phi4) - 90164ms. StopReason: stop. Tokens(in/out): 365/396 ---PROMPT--- user: Extract from this content: Grave Digger: Ingredients - 1 1/2 ounces vanilla-infused brandy* - 3/4 ounce coffee liqueur - 1/2 ounce Grand Marnier - 1 ounce espresso, freshly brewed - Garnish: whipped cream - Garnish: oreo cookies, crushed Steps 1. Add all ingredients into a shaker with i…

appreciate you tyring it. the reason it dropped the day was due to your type system not being understood by the LLM you're using.

the model replied with

       {
          "Text": "coffee liqueur",
          "Type": "Liqueur",
          "Liquor_type": "Liqueur",
          "Name_brand": null,
          "Unit_of_measure": "ounce",
          "Measurement_or_unit_count": "3/4"
        },
but you expected a { Text: string, Type: IngredientType, Liquor_type: LiquorType or null, Name_brand: string or null, Unit_of_measure: string, Measurement_or_unit_count: string, }

there's no way to cast `Liqueur` -> `IngredientType`. but since the the data model is a `Ingredient[]` we attempted to give you as many ingredients as possible.

The model itself being wrong isn't something we can do much about. that depends on 2 things (the capabilities of the model, and the prompt you pass in).

If you wanted to capture all of the items with more rigor you could write it in this way:

    class Recipe {
        name string
        ingredients Ingredient[]
        num_ingredients int
        ...

        // add a constraint on the type
        @@assert(counts_match, {{ this.ingredients|length == this.num_ingredients }})
    }
And then if you want to be very wild, put this in your prompt:

   {{ ctx.output_format }}
   No quotes around strings
And it'll do some cool stuff

Re: Andrej Karpathy: Software in the era of AI [video]

#804

Post by [deleted] »

handfuloflight wrote: post #519
solaire_oa wrote: post #411

Earlier quoted context omitted.

I also think that structured outputs are criminally underused, but it isn't perfect... and per your example, it might not even be good, because I've done something similar. I was trying to make a decent cocktail recipe database, and scraped the text of cocktails from about 1400 webpages. Note that this was just the text of the cocktail recipe, and cocktail recipes are comparatively small. I sent the text to an LLM fo…

Which LLM?

[deleted]

Re: Andrej Karpathy: Software in the era of AI [video]

#805

Post by 0xjunhao »

Before I became a software engineer, I was a computational physicist. My days back then were pretty much tweaking some parameters, running a job, then reading papers and checking back after a few minutes or hours. Increasingly, I’m starting to think my days as a software engineer will be pretty similar.

Re: Andrej Karpathy: Software in the era of AI [video]

#806

Post by hellovai »

instig007 wrote: post #630
hellovai wrote: post #599

Earlier quoted context omitted.

have you tried schema-aligned parsing yet? the idea is that instead of using JSON.parse, we create a custom Type.parse for each type you define. so if you want a: class Job { company: string[] } And the LLM happens to output: { "company": "Amazon" } We can upcast "Amazon" -> ["Amazon"] since you indicated that in your schema. https://www.boundaryml.com/blog/schema-aligned-parsing and since its only post processing, t…

so if you want a: class Job { company: string[] } We can upcast "Amazon" -> ["Amazon"] since you indicated that in your schema. Congratulations! You've discovered Applicative Lifting.

its a bit more nuanced than applicative lifting. parts of of SAP is that, but there's also supporting strings that don't have quotation marks, supporting recursive types, supporting unescaped quotes like: `"hi i wanted to say "hi""`, supporting markdown blocks inside of things that look like "json", etc.

but applicative lifting is a big part of it as well!

gloochat.notion.site/benefits-of-baml

Re: Andrej Karpathy: Software in the era of AI [video]

#807

Post by hellovai »

solaire_oa wrote: post #653
hellovai wrote: post #599

Earlier quoted context omitted.

have you tried schema-aligned parsing yet? the idea is that instead of using JSON.parse, we create a custom Type.parse for each type you define. so if you want a: class Job { company: string[] } And the LLM happens to output: { "company": "Amazon" } We can upcast "Amazon" -> ["Amazon"] since you indicated that in your schema. https://www.boundaryml.com/blog/schema-aligned-parsing and since its only post processing, t…

Ok. Tried it, I'm not super impressed. Client: Ollama (phi4) - 90164ms. StopReason: stop. Tokens(in/out): 365/396 ---PROMPT--- user: Extract from this content: Grave Digger: Ingredients - 1 1/2 ounces vanilla-infused brandy* - 3/4 ounce coffee liqueur - 1/2 ounce Grand Marnier - 1 ounce espresso, freshly brewed - Garnish: whipped cream - Garnish: oreo cookies, crushed Steps 1. Add all ingredients into a shaker with i…

if you share your prompt with me on promptfiddle.com i can play around with it and see how i can make it better!

Re: Andrej Karpathy: Software in the era of AI [video]

#810

Post by tim333 »

dustbunny wrote: post #778
standeven wrote: post #773

Earlier quoted context omitted.

Why bother using higher-level programming languages to communicate with a computer? You interact with a computer using assembly - raw bit shifting and memory addresses - which is more precise and effective.

Using assembly is not really more precise in terms of solving the problem. You can definitely make an argument that using a higher level language is equally if not more precise. Especially since your low level assembly will be limited to which architectures it can run on, you can state that the c++ that generates that assembly is "more precisely defining a calculator program".

Using code may not be more precise in terms of solving a problem than english. Take the NHS. With better AI, saying build a good IT system for the NHS may have worked better than this stuff https://www.theguardian.com/society/2013/sep/18/nhs-records-...
Post reply on HN