Live data from Hacker News

Throw more AI at your problems

frontierai.substack.com

51–59 of 59 posts

Re: Throw more AI at your problems

#51
post #20
post #6

Earlier quoted context omitted.

I'm pretty sure this is a satire post

Here's a practical in this vein but much simpler - if you're trying to answer a question with an LLM, and have it answer in json format within the same prompt, for many models the accuracy is worse than just having it answer in plaintext. The reason is that you're now having to place a bet that the distribution of json strings it's seen before meshes nicely with the distribution of answers to that question. So one re…

I wrap the plaintext in quotes, and perhaps a period, so that it knows when to start and when to stop, you can add logit biases for the syntax and pass period as a stop marker to chatgpt apis.

Also you don't need to use a model to build a json from plaintext answers lol, just use a programming language.

Re: Throw more AI at your problems

#52

Earlier quoted context omitted.

Things will get much more stupid in a few years when we have to maintain all this LLM-generated garbage code.

if by “stupid” you mean “worse is better”. except this time it’s actually better. just because it’s apostasy according to the Church of Engineering does not mean it can be dismissed out of hand, no matter how much it hurts your sensibilities. (it used to mine as well, but then i learned to stop worrying and learned to love our new llm overlords)

Worse isn't better, it's worse.

Re: Throw more AI at your problems

#53
post #13

RAG doesn’t necessarily give the best results. Essentially it is a technically elegant way to semantic context to the prompt (for many use cases it is over-engineered). I used to offer RAG SQL query generations on SQLAI.ai and while I might introduce it again, for most use cases it was overkill and even made working with the SQL generator unpredictable. Instead I implemented low tech “RAG” or “data source rules”. It’…

What do you mean by "RAG SQL query generations"? Were you searching for example queries similar to the questions the user's asked and injecting those examples into the prompt?

Yes exactly. That way the user could "train" AI on a particular data source.

Re: Throw more AI at your problems

#54
> "The most common debate was whether RAG or fine-tuning was a better approach, and long-context worked its way into the conversation when models like Gemini were released. That whole conversation has more or less evaporated in the last year, and we think that’s a good thing. (Interestingly, long context windows specifically have almost completely disappeared from the conversation though we’re not completely sure why.)"

I'm a bit confused by their thinking it's a good thing while being confused about why the subject has "disappeared from the conversation".

Could anyone here shed some light / share an opinion on it/why "long context windows" aren't discussed any more? Did everyone decide they're not useful? Or they're so obviously useful that nobody wastes time discussing them? Or...

Re: Throw more AI at your problems

#55

Earlier quoted context omitted.

Things will get much more stupid in a few years when we have to maintain all this LLM-generated garbage code.

if by “stupid” you mean “worse is better”. except this time it’s actually better. just because it’s apostasy according to the Church of Engineering does not mean it can be dismissed out of hand, no matter how much it hurts your sensibilities. (it used to mine as well, but then i learned to stop worrying and learned to love our new llm overlords)

I'm not dismissing it out of hand. I've spent significant time evaluating the capabilities of LLMs and studied how people are actually using it across different domains. The evidence is clear - LLMs are helpful tools but they make it way too easy to crank out lots of code without corresponding understanding. And if you think software is more about lines of code than it is developing understanding, I question whether you've ever made a piece of software that lasts more than 5 years.

Re: Throw more AI at your problems

#56
post #34
post #29

Earlier quoted context omitted.

Semantic search is a powerful tool that can greatly improve the relevance and quality of search results by understanding the intent and contextual meaning of search terms. However, it’s not without its limitations. One of the key challenges with semantic search is the assumption that the answer to a query is semantically similar to the query itself. This is not always the case, and it can lead to less than optimal re…

Don't post AI slop on HN comments please.

this is not AI slop. do you have a proof of what you just wrote ?

Re: Throw more AI at your problems

#57
post #18

Earlier quoted context omitted.

100 prompts doing what? Something like more selective, focused extraction of structured fields?

a combination of lots of things, with the general theme being focused prompts good at individual, specific subtasks circuits… off the top of my head: - that try to extract the factual / knowledge content and try to update the rest of the system (e.g. if the user chats you to not send notifications after 9pm, ideally you’d like the whole system to reflect that. if they say they like the color gold, you’d like the reco…

Interesting

Re: Throw more AI at your problems

#58

Earlier quoted context omitted.

if by “stupid” you mean “worse is better”. except this time it’s actually better. just because it’s apostasy according to the Church of Engineering does not mean it can be dismissed out of hand, no matter how much it hurts your sensibilities. (it used to mine as well, but then i learned to stop worrying and learned to love our new llm overlords)

I'm not dismissing it out of hand. I've spent significant time evaluating the capabilities of LLMs and studied how people are actually using it across different domains. The evidence is clear - LLMs are helpful tools but they make it way too easy to crank out lots of code without corresponding understanding. And if you think software is more about lines of code than it is developing understanding, I question whether…

"evaluating the capabilities" sounds like a fancy way of saying that you know what other people say it can do... i strongly encourage you to try it for yourself. try to make some sort of multi-part, complicated document (technical or non-). tell it that you and it together are collaborating on it, what it's for, etc. and just start interacting with it. give it the stuff that you have so far. ask it for counterarguments, place where the argument could be better, comments about structure and style, ... the world is your oyster. it will give you a MUCH better picture of how these things actually work-- what the process of externalizing cognition is actually like, and how deep and detailed you can get. the results will amaze you.

people have this massive misconception about what LLMs can do and how to get good results out of them where they think they just kinda ask it for stuff and voila, it appears. it could not be any further from reality. it is an interactive tool that you get into the right "frame of mind" (scare quotes because this is a descriptive analogy, not one meant to convey or impart mechanical sympathy) and then ask it... literally anything you want. but you have to DISCOVER these things-- these (nearly) magic words, phrases, encodings of the problem, etc. that get the LLM to generate results in a way resonant with the way that you do / the way that you want it to. then you get the generation part "for free". you teach it to be a perfect painter, and then let it paint (shoutout Robert Pirsig // Zen and the Art of Motorcycle Maintenance).

the whole "LLM applications to " stuff is such a mirage, and so tied up in a misbegotten view of how they work and what they're good at. if you have a hyper-hyper-specialized domain, sure, you might need to... find some specialized data. find or train some bespoke model. but for damn near anything written in english (can't speak to other languages) it is as good at coding as it is at bioinformatics as it is at statistics as it is at sociology as it is at literature (to be rather flippant). just do some introspection on how you do your job, how you think through problems, how you generate your output, and then "outrospect" it "into the computer". once you do that, voila: you have a virtually infinitely scalable simulacrum of your own cognition. in no way is it "plug in the ai and it will automatically take my job, one size fits all". the reason why they works so well is precisely because they are NOT that.

think about it this way: if you knew that each day when you went to sleep all of your memories were deleted, but you and your brain otherwise worked exactly the same way as yesterday, what would you do? (the concept of the mediocre but cute movie "50 First Dates") what notes would you leave for yourself to get back into the same context as you were in when you went to sleep the night before? how would you convince yourself that the artifacts you leave for yourself in the morning are true? how would you convey the subtlety of your thoughts & feelings, idiosyncracies, point of view, style & syntax? how would you grow the system over time? the LLMs are closer to speaking the language of your own thoughts than any human and every human invention in all of history. prompting is just the way to incept ideas and thoughts, systems, etc. into it's ai mind in a way that i find to be profoundly similar to how our own perception is not "reality" per se, but rather the image of reality that we see within our own minds. an image that we build up from birth as we grow to understand reality and the world around us.

Re: Throw more AI at your problems

#59
post #52

Earlier quoted context omitted.

if by “stupid” you mean “worse is better”. except this time it’s actually better. just because it’s apostasy according to the Church of Engineering does not mean it can be dismissed out of hand, no matter how much it hurts your sensibilities. (it used to mine as well, but then i learned to stop worrying and learned to love our new llm overlords)

Worse isn't better, it's worse.

couldn't agree more... if we're talking about LISP machines-- man i wish that side won. i believe in design when it comes to creating tools for humans, because we are finite and have finite capacity. but the PoV is just not exportable to the LLM, and when you have the sorts of crystallizing collaborative moments w/ LLMs that i have had... let's just say that it's beyond question that they have way too much utility to not be the shape of things to come. they are the "copilot" today... tomorrow, the roles will be reversed. and that's AWESOME! our worse is their better and vice versa-- sounds like an amazing partnership to me. ebony and ivory.
Post reply on HN