Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4o's Memory Breakthrough – Needle in a Needlestack

nian.llmonpy.ai

Post reply on HN
Print view View original

221–230 of 256 posts

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#221

Post by unclebucknasty »

tifik wrote: post #198
unclebucknasty wrote: post #190

Earlier quoted context omitted.

> There's no fundamental qualitative difference here...degree either. I've heard the same comparisons made with self-driving cars (i.e. that humans are fallible, and maybe even more error-prone). But this misses the point. People trust the fallibility they know. That is, we largely understand human failure modes (errors in judgement, lapses in attention, etc) and feel like we are in control of them (and we are). OTOH…

What you say is true, and I agree, but that is the emotional human side of thinking. Purely logically, it would nake sense to compare the two systems of control and use the one with fewer human casualities. Not saying its gonna happen, just thinking that reason and logic should take precedent, no matter what side you are on.

It definitely seems like a matter of simple math. But, I'm not 100% sure it's always the most logical choice to defer to statistics.

By definition, stats operate at the macro level. So, for instance, I may be a safer driver than the AI average. Should I give up control? I suppose it's also a matter of degree and there's the network effect to consider (i.e. even If I individually beat the average, I'm still on the road with others who don't).

So it gets a little more complicated and I'm also not sure the aversion to relinquishing control is strictly "emotional" (as in the irrational sense). There's something about the potential finality of a failure that goes along with autonomy and agency over one's own life. The idea that a machine could make a mistake that ends your life, and you never had a chance or say in that outcome is off-putting in ways that feel more rooted in rationality and survival than in emotion.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#222

Post by fnordpiglet »

bearjaws wrote: post #81

I just used it to compare two smaller legal documents and it completely hallucinated that items were present in one and not the other. It did this on three discrete sections of the agreements. Using ctrl-f I was able to see that they were identical in one another. Obviously this is a single sample but saying 90% seems unlikely. They were around ~80k tokens total.

That’s not needle in a haystack.

I would note that LLMs handle this task better if you slice the two documents into smaller sections and iterate section by section. They aren’t able to reason and have no memory so can’t structurally analyze two blobs of text beyond relatively small pieces. But incrementally walking through in much smaller pieces that are themselves semantically contained and related works very well.

The assumption that they are magic machines is a flawed one. They have limits and capabilities and like any tool you need to understand what works and doesn’t work and it helps to understand why. I’m not sure why the bar for what is still a generally new advance for 99.9% of developers is effectively infinitely high while every other technology before LLMs seemed to have a pretty reasonable “ok let’s figure out how to use this properly.” Maybe because they talk to us in a way that appears like it could have capabilities it doesn’t? Maybe it’s close enough sounding to a human that we fault it for not being one? The hype is both overstated and understated simultaneously but there have been similar hype cycles in my life (even things like XML were going to end world hunger at one point).

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#223

Post by throwthrowuknow »

mrbungie wrote: post #175
throwthrowuknow wrote: post #136

Earlier quoted context omitted.

Why? There are many other types of AI or statistical methods that are easier, faster and cheaper to use not to mention better suited and far more accurate. Militaries have been employing statisticians since WWII to pick targets (and for all kinds of other things) this is just current-thing x2 so it’s being used to whip people into a frenzy.

Because you can charge up a lot more when adding hot and hyped features like LLMs instead of doing good engineering.

I don’t know for sure but I imagine getting blackballed by the defence department is not fun.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#224

Post by throwthrowuknow »

tsunamifury wrote: post #212
throwthrowuknow wrote: post #136

Earlier quoted context omitted.

Why? There are many other types of AI or statistical methods that are easier, faster and cheaper to use not to mention better suited and far more accurate. Militaries have been employing statisticians since WWII to pick targets (and for all kinds of other things) this is just current-thing x2 so it’s being used to whip people into a frenzy.

It can do limited battlefield reasoning where a remote pilot has significant latency. Call these LLMs stupid all you want but on focused tasks they can reason decently enough. And better than any past tech.

That seems like something a special purpose model would be a lot better and faster at. Why use something that needs text as input and output? It would be slow and unreliable. If you need reaction time dependent decisions like collision avoidance or evasion for example then you can literally hard wire those in circuits that are faster than any other option.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#225

Post by ignoramous »

nopromisessir wrote: post #216
ignoramous wrote: post #193

Earlier quoted context omitted.

> a sophisticated approach to their defense A euphemism for apartheid and oppression? > sources are rife with bias What's biased about terming autonomous weapons as "AI"? Or, sounding alarm over dystopian surveillance enabled by AI? > The nuance matters. Like Ben Gurion terming Lehi "freedom fighters" as terrorists? And American Jewish intellectuals back then calling them fascists? > The history matters... After that…

I will spend no more than two comments on this issue. Most people have already made up their minds. There is little I can do about that, but perhaps someone else might see this and think twice. Personally, I have spent many thousands of hours on this topic. I have Palestinian relatives and have visited the Middle East. I have Arab friends there, both Christian and Muslim, whom I would gladly protect with my life. I a…

> I strive to remain fiercely objective.

Commendable. You'll appreciate this Israeli historian: https://www.youtube.com/watch?v=xj_HKw-UlUk (summary: https://archive.is/dOP7g). And this Israeli Prof, also an expert on Holocaust studies, being fiercely objective: https://www.mekomit.co.il/ps/134005/ (en: https://archive.is/Fjj6f)

> I had not taken the time to understand the complexities of the Middle East. Even now, I cannot claim to fully comprehend them.

Why even spend 2 comments?

> The contextual depth required to grasp the regrettable necessity of Israeli actions...

The same level of depth as Supremacists who regrettably exterminated non-Aryans?

> There is overwhelming bipartisan support for Israel in one of the world's most divided congresses, and this support stems more from shared values.

This is undeniable, but the underlying "shared values" here are not the ones you'd like us to think: https://www.bostonreview.net/articles/instruments-of-dehuman...

> I stand by my original comment.

Like you say, there's the entire might of the US political and elite class behind you; it isn't some act of courage or rebellion, fwiw.

> As a final note, in my youth, I held views similar to the ones you propagate.

Propagate? Your final note sounds like a threat.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#226

Post by yatz »

ijidak wrote: post #62
yatz wrote: post #58

Well, I can now use GPT to transform raw dynamic data into beautiful HTML layouts on the fly for low-traffic pages, such as change/audit logs, saving a ton of development time and keeping my HTML updated even when the data structure has changed. My last attempt did not consistently work because GPT4-Turbo sometimes ignored the context and instructions almost entirely.

Do you have an example of this? I would love to learn more.

Here is the entire prompt. I used rules to ensure the formatting is consistent as otherwise sometimes it might format date one way and other times in an entirely different way.

Imagine, a truly dynamic and super personal site, where layout, navigation, styling and everything else gets generated on the fly using user's usage behavior and other preferences, etc. Man! ---------------------------------------------

{JSON} ------ You are an auditing assistant. Your job is to convert the ENTIRE JSON containing "Order Change History" into a human-readable Markdown format. Make sure to follow the rules given below by letter and spirit. PLEASE CONVERT THE ENTIRE JSON, regardless of how long it is. --------------------------------------------- RULES: - Provide markdown for the entire JSON. - Present changes in a table, grouped by date and time and the user, i.e., 2023/12/11 12:40 pm - User Name. - Hide seconds from the date and time and format using the 12-hour clock. - Do not use any currency symbols. - Format numbers using 1000 separator. - Do not provide any explanation, either before or after the content. - Do not show any currency amount if it is zero. - Do not show IDs. - Order by date and time, from newest to oldest. - Separate each change with a horizontal line.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#227

Post by yatz »

balder1991 wrote: post #75
ijidak wrote: post #62

Earlier quoted context omitted.

Do you have an example of this? I would love to learn more.

I guess you just need to offer a template in the prompt? Then maybe some validation after.

No templates, just some rules and the model does the rest. It worked like a charm, even gave me ideas on how to layout and format the page to make it easy to read.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#228

Post by tmaly »

1970-01-01 wrote: post #185
bearjaws wrote: post #81

I just used it to compare two smaller legal documents and it completely hallucinated that items were present in one and not the other. It did this on three discrete sections of the agreements. Using ctrl-f I was able to see that they were identical in one another. Obviously this is a single sample but saying 90% seems unlikely. They were around ~80k tokens total.

I've done the same experiment with local laws and caught GPT hallucinating fines and fees! The problem is real.

Imagine if they started using LLMs to suggest prison sentences

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#229

Post by davedx »

cdelsolar wrote: post #155
feverzsj wrote: post #141

LLMs are still toys, no one should treat them seriously. Apparently, the bubble is too massive now.

Must be a pretty cool toy; it constantly 10X’s my productivity.

It's staggering to me that people on Hacker News are actually downvoting people saying how AI is boosting productivity or levering business or engineering or finance. The denial, cynicism and sheer wilful ignorance is actually depressing. I get that not everyone is working directly with AI/ML but I honestly expected better on a website about technology.

People are deliberately self selecting themselves out of the next industrial revolution. It's Darwin Awards for SWE careers. It's making me ranty.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#230

Post by davedx »

nopromisessir wrote: post #199
cdelsolar wrote: post #155

Earlier quoted context omitted.

Must be a pretty cool toy; it constantly 10X’s my productivity.

You said it mate. I feel bad for folks who turn away from this technology. If they persist... They will be so confused why they get repeatedly lapped. I wrote a working machine vision project in 2 days with these toys. Key word: working... Not hallucinated. Actually working. Very useful.

I just don't understand why AI is so polarising on a technology website.

OpenAI have even added a feature to make the completions from GPT near-deterministic (by specifying a seed). It seems that no matter what AI companies do, there will be a vocal minority shouting that it's worthless.

Post reply on HN