Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new study of a bot running a store finds it is friendly but not very smart

nytimes.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: A new study of a bot running a store finds it is friendly but not very smart

#41

Post by bluemoonx »

In sci-fi future movies like 5th Element they comically make AI technology seem kinda dumb - or at least like you can trick it.

You can make permissions decide which model is used, and only train that on the data that permission allows (someone should build this - permissioned RAG), that solves leaks, but decision-making within a model, or behavior exfiltration comparable to viewing backend source code still seems possible.

It’s basically like client/server security: You can’t “trust the model” in the same way you can’t “trust the client” in a backend/frontend setup.

When used as effectively a point-of-sale, AI seems more hackable than a vending machine - as a boss or assistant even more so.

Re: A new study of a bot running a store finds it is friendly but not very smart

#42

Post by quickthrowman »

throwway120385 wrote: post #32
Joel_Mckay wrote: post #23

Earlier quoted context omitted.

Physical Brick and Mortar locations no longer make sense for many retail businesses, as Walmart/Amazon have same day delivery. If folks have ever been financially bent-over for a retail business shop location they understand why margins matter. One has to move a lot of inventory just to reach a profit mode. Gimmicks only work for awhile, as consumers have full market awareness on their smartphone. =3

> Physical Brick and Mortar locations no longer make sense for many retail businesses, as Walmart/Amazon have same day delivery. This is bull. Take paint brushes for fine finish when painting houses. Amazon sells "paint brushes." But what they don't sell are paint brushes that are actually fit for purpose. It's fine if you're only planning on painting some pre-textured walls. You're probably already using that cheap…

> This is bull. Take paint brushes for fine finish when painting houses. Amazon sells "paint brushes." But what they don't sell are paint brushes that are actually fit for purpose.

The person you’re replying to was talking about consumer products, not tools for tradespeople and serious amateurs. Maybe 1 in 100 people that do painting in their home even realize they can get quality paintbrushes instead of using the crappy ones that Home Depot sells or spend the money for premium Benjamin Moore or Sherwin Williams paint.

It shouldn’t be surprising that tools almost exclusively used by tradespeople are sold at specialty supply houses catering to local businesses. I’d head to my local Benjamin Moore dealer for decent house painting brushes. Supply houses almost never rent premium high street real estate.

I do agree with the general thrust of your post, Amazon mostly sells garbage barely fit for purpose and I do not order from them.

Re: A new study of a bot running a store finds it is friendly but not very smart

#43

Post by owaislone »

bluemoonx wrote: post #41

In sci-fi future movies like 5th Element they comically make AI technology seem kinda dumb - or at least like you can trick it. You can make permissions decide which model is used, and only train that on the data that permission allows (someone should build this - permissioned RAG), that solves leaks, but decision-making within a model, or behavior exfiltration comparable to viewing backend source code still seems po…

Exactly. This is how I design agents as well. I essentially treat the agent as the web/mobile app, cli tool os API library not as part of my backend even though there is where it runs. The backend doesn't treat the agent in any special way. It simply gates all action based on the permissions of the user/guest that is using the agent.

Re: A new study of a bot running a store finds it is friendly but not very smart

#44

Post by archonis »

walrus01 wrote: post #15
SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

From the article: > Luna has failed miserably in that mission and is down $62,000. Mr. Petersson and Mr. Backlund said they thought Luna would eventually get smarter and more business-savvy and were pleased its friendliness has held steady. I am extremely skeptical that whatever LLM they're running this on has sufficient context window size to handle multiple months of all possible activities of running a retail busi…

>> Mr. Petersson and Mr. Backlund said they thought Luna would eventually get smarter and more business-savvy

How are we supposed to take these "labs" seriously when they clearly don't understand how the technology works at a rudimentary level?

Re: A new study of a bot running a store finds it is friendly but not very smart

#45

Post by cwnyth »

Joel_Mckay wrote: post #23
walrus01 wrote: post #21

Earlier quoted context omitted.

Sure, I mean from the day to day operations of selling a product to the customer, maybe it does fine? People in the article say it's friendly and cooperative (perhaps a bit too much). I was thinking more in terms of building a consistent plan for retail product lines to sell over a multi month period. The selection of "stuff" in the store right now looks like you put an etsy search into a blender. It is also probably…

Physical Brick and Mortar locations no longer make sense for many retail businesses, as Walmart/Amazon have same day delivery. If folks have ever been financially bent-over for a retail business shop location they understand why margins matter. One has to move a lot of inventory just to reach a profit mode. Gimmicks only work for awhile, as consumers have full market awareness on their smartphone. =3

The malls in Japan and Thailand are booming. There are a ton of crowds, they have plenty in stock, there are dozens of restaurants that keep people in.

You can also do delivery in those places.

I think a lot of places in the west, but America especially, convinced themselves their poorly run physical stores, owned and nigh operated by venture capitalists, is an indicator that people don't need them anymore. The world has shown that that is clearly not the case.

Re: A new study of a bot running a store finds it is friendly but not very smart

#46

Post by ed_elliott_asc »

SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

I use LLM’s to code for me all day but it does feel a little bit like we have really advanced psychic mediums “is there a James in the room, I’m getting a message from someone called Mary…” people have been guessing at the next word and making a career out of it for hundreds of years already.

Re: A new study of a bot running a store finds it is friendly but not very smart

#47

Post by Joel_Mckay »

quickthrowman wrote: post #42
throwway120385 wrote: post #32

Earlier quoted context omitted.

> Physical Brick and Mortar locations no longer make sense for many retail businesses, as Walmart/Amazon have same day delivery. This is bull. Take paint brushes for fine finish when painting houses. Amazon sells "paint brushes." But what they don't sell are paint brushes that are actually fit for purpose. It's fine if you're only planning on painting some pre-textured walls. You're probably already using that cheap…

> This is bull. Take paint brushes for fine finish when painting houses. Amazon sells "paint brushes." But what they don't sell are paint brushes that are actually fit for purpose. The person you’re replying to was talking about consumer products, not tools for tradespeople and serious amateurs. Maybe 1 in 100 people that do painting in their home even realize they can get quality paintbrushes instead of using the cr…

Amazon is an authorized direct reseller of metrology equipment like Mitutoyo products

https://www.amazon.ca/Mitutoyo-543-400-Digimatic-Indicator-G...

https://www.amazon.ca/Mitutoyo-500-463-Stainless-Resolution-...

https://www.amazon.ca/gp/product/B00MBHXWGY/ref=sw_img_1?smi...

They also provided NIST traceable versions at one time, and I can personally confirm the legitimate products hit the tolerances on reference optical parallels. I often buy these for friends kids that choose engineering in university.

Most local retail shops will continue to decline as people that can't use a phone or computer become more rare. Partially driven by the capital markets denial-of-service in lease monopolization by holding companies.

Note, I am not advocating for the situation, but we must accept that it is the new business reality. Even https://www.starrett.com/ is now re-selling import products alongside the classic US tool offerings.

Have a wonderful day =3

Re: A new study of a bot running a store finds it is friendly but not very smart

#48

Post by rcxdude »

ed_elliott_asc wrote: post #46
SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

I use LLM’s to code for me all day but it does feel a little bit like we have really advanced psychic mediums “is there a James in the room, I’m getting a message from someone called Mary…” people have been guessing at the next word and making a career out of it for hundreds of years already.

My mental model of LLMs is as a contractor you've just brought in, every time. They might be very capable but they're starting from scratch on your particular project.

Re: A new study of a bot running a store finds it is friendly but not very smart

#49

Post by randomImmigrant »

I have tried making the case before that LLM "agency" is a complete hoax. Let me try it again. Would appreciate thoughts from folks at HN:

Biological neurons have one property that was unknown till the late 90s/2000s: each neuron (and in fact, each cell in your body) is an autonomous circadian clock, tracking the 24-h day. They can be entrained to external timing signals, are usually in synchrony, but can be desynchronized. This clock, it has been shown, schedules the production and localization of critical components in the synapse, and is plugged in downstream, in the nucleus, in responding to synaptic signals. Disrupting the clock disrupts learning. The existence of the clock is why learning peaks and troughs during the day.

Critically, clock function has been shown to be involved in both memory storage AND subsequent successful retrieval.

LLMs, famously cannot keep track of time, and I suspect this is why. Its easy enough to look at a clock, but without an internal rhythm, LLMs have no internal timing synchronizing their various behaviors, and their memory systems do not thread through this timing system, leading to their unstable memory, identity and performance.

Re: A new study of a bot running a store finds it is friendly but not very smart

#50

Post by fluoridation »

dylan604 wrote: post #30
criddell wrote: post #6

> Luna reorders products that are not selling and sometimes gets orders wrong. A recent one of Andon Market mugs with big, green smiley faces came misprinted with the faces so small, they looked like tiny dots. The mug with the tiny happy face is kinda hilarious. If I was there I would buy one. I wonder though if the mug was a case of getting the order wrong or was a misprint? The article isn't clear on that.

Maybe it confused metric and imperial units?? The order form had text entry with a value of 2 with no units. The user assumed 2" but the print house assumed 2mm. Always always always show your units. I can still hear the voice of my high school physics teacher saying that.

It's equally likely that the unit was implied by other content around the input field and the model didn't make the connection (like "scale: [___] mm"), or perhaps the file format was just wrong and a human placing the order would have resulted in the same outcome.
Post reply on HN