Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

Post reply on HN
Print view View original

201–210 of 339 posts

Re: Gemini 2.5 Computer Use model

#201

Post by tgsovlerkhgsel »

I'm so looking forward to it. Many of the problems that should be trivially solved with either AI or a script are hard to impossible to solve because the data is locked away in some form.

Having an AI handle this may be inefficient, but as it uses the existing user interfaces, it might allow bypassing years of bureaucracy, and when the bureaucracy tries to fight it to justify its existence, it can fight it out with the EVERYONE MUST USE AI OR ELSE layers of management, while I can finally automate that idiotic task (using tens of kilowatts rather than a one-liner, but still better than having to do it by hand).

Re: Gemini 2.5 Computer Use model

#202

Post by riebschlager »

neom wrote: post #185
dekhn wrote: post #89

I just have to say that I consider this an absolutely hilarious outcome. For many years, I focused on tech solutions that eliminated the need for a human to be in front of a computer doing tedious manual operations. For a wide range of activities, I proposed we focus on "turning everything in the world into database objects" so that computers could operate on them with minimal human effort. I spent significant effort…

I was thinking about that last point in the context of dating this morning, if my "chatgpt" knew enough about me to represent me well enough that a dating app could facilitate a pre-screening with someone else "chatgpt", that would be interesting. I heard someone in an enterprise keynote recently talking about "digital twins" - I believe this is that. Not sure what I think about it yet generally, or where it leads.

Congrats, you've won today's "Accidental Re-writing of a Black Mirror Episode" prize. :)

https://en.wikipedia.org/wiki/Hang_the_DJ

Re: Gemini 2.5 Computer Use model

#203

Post by DrewADesign »

emp17344 wrote: post #149
pseidemann wrote: post #121

Earlier quoted context omitted.

Funny thing is, most humans cannot properly control a computer. Intelligence seems to be impossible to define.

Is this a joke, or do you actually believe most people are incapable of using a computer?

It’s the same old superiority complex that birthed the “IT Guy” stereotypes of the 90s/aughts. It stems from a) not understanding what problems non-developers need computers to solve for them, and b) ego-driven overestimation of the complexity of their field compared to others.

Re: Gemini 2.5 Computer Use model

#204

Post by jwpapi »

Can somebody give me use cases that are faster than using an UX?

How am I supposed to use this. I really can’t think of one, but I don’t want to be blind-sighted as obviously a lot of money is going into this.

I also appreciate the tech behind it and functionality, but I still wonder for use cases

Re: Gemini 2.5 Computer Use model

#205

Post by dgacmu »

mh- wrote: post #182
itsmartapuntocm wrote: post #155

Earlier quoted context omitted.

They’re extremely common in the U.S. now.

Traffic cameras, yes. Traffic cameras that are used to influence traffic signaling? I've never (knowingly) seen one in the US. What US cities have these?

We have one here as part of a CMU research deployment: https://www.transportation.gov/utc/surtrac-people-upgrading-...

> The system applies artificial intelligence to traffic signals equipped with cameras or radars adapting in realtime to dynamic traffic patterns of complex urban grids, experienced in neighborhoods like East Liberty in the City of Pittsburgh

Now, that said, I have serious issues with that system: It seemed heavily biased to vehicle throughput over pedestrians, and it's not at all clear that it was making the right long-term choice as far as the incentives it created. But it _was_ cameras watching traffic to influence signaling.

https://www.transportation.gov/utc/surtrac-people-upgrading-...

https://en.wikipedia.org/wiki/Scalable_Urban_Traffic_Control

Re: Gemini 2.5 Computer Use model

#206

Post by mh- »

dgacmu wrote: post #205
mh- wrote: post #182

Earlier quoted context omitted.

Traffic cameras, yes. Traffic cameras that are used to influence traffic signaling? I've never (knowingly) seen one in the US. What US cities have these?

We have one here as part of a CMU research deployment: https://www.transportation.gov/utc/surtrac-people-upgrading-... > The system applies artificial intelligence to traffic signals equipped with cameras or radars adapting in realtime to dynamic traffic patterns of complex urban grids, experienced in neighborhoods like East Liberty in the City of Pittsburgh Now, that said, I have serious issues with that system: It…

Interesting, thanks!

Re: Gemini 2.5 Computer Use model

#207

Post by ericd »

serf wrote: post #147
VirgilShelton wrote: post #141

Earlier quoted context omitted.

[flagged]

go watch any movie about a panopticon for the (overdiscussed) side-effects of a surveillance state. Fiction works, but if you want to spend the evening depressed then go for any East/West Germany (true) stories. For it or against surveillance and I can understand, but just not understanding the issue? No excuses -- personal surveillance for the sake of the state is one of the most discussed social concepts in the wor…

The Lives of Others is a great one about the Stasi.

Re: Gemini 2.5 Computer Use model

#208

Post by pavelstoev »

dekhn wrote: post #71

Many years ago I was sitting at a red light on a secondary road, where the primary cross road was idle. It seemed like you could solve this using a computer vision camera system that watched the primary road and when it was idle, would expedite the secondary road's green light. This was long before computer vision was mature enough to do anything like that and I found out that instead, there are magnetic systems that…

It was my first engineering job, calibrating those inductive loops and circuit boards on I-93, just north of Boston's downtown area. Here is the photo from 2006. https://postimg.cc/zbz5JQC0

PEEK controller, 56K modem, Verizon telco lines, rodents - all included in one cabinet

Re: Gemini 2.5 Computer Use model

#209

Post by enjoylife »

alganet wrote: post #114
enjoylife wrote: post #112

> It is not yet optimized for desktop OS-level control Alas, AGI is not yet here. But I feel like if this OS-level of control was good enough, and the cost of the LLM in the loop wasn't bad, maybe that would be enough to kick start something akin to AGI.

I am curious. Why do you think controlling an OS (and not just a browser) would be a move towards AGI?

One thought is once it can touch the underlying system, it can provision resources, spawn processes, and persist itself, crossing the line from tool to autonomous entity. I admit you could do that in a browser shell nowadays, just maybe with more restrictions and guardrails. I don’t have any strong opinions here, but I do think a lower cost to escape the walled gardens agi starts in will be a factor

Re: Gemini 2.5 Computer Use model

#210

Post by omkar_savant »

jwpapi wrote: post #204

Can somebody give me use cases that are faster than using an UX? How am I supposed to use this. I really can’t think of one, but I don’t want to be blind-sighted as obviously a lot of money is going into this. I also appreciate the tech behind it and functionality, but I still wonder for use cases

At current latency, there's a bunch of async automation usecases that one could use this for. For example:

*Tedious to complete, easy to verify

BPO: Filling out doctor licensing applications based on context from a web profile HR: Move candidate information from an ATS to HRIS system Logistics: Fill out a shipping order form based on a PDF of the packing label

* Interact with a diverse set of sites for a single workflow

Real estate: Diligence on properties that involves interacting with one of many county records websites Freight forwarding: Check the status of shipping containers across 1 of 50 port terminal sites Shipping: Post truck load requests across multiple job board sites BPO: Fetch the status of a Medicare coverage application from 1 of 50 state sites BPO: Fill out medical license forms across multiple state websites

* Periodic syncs between various systems of record

Clinical: Copy patient insurance info from Zocdoc into an internal system HR: Move candidate information from an ATS to HRIS system Customer onboarding: Create Salesforce tickets based on planned product installations that are logged in an internal system Logistics: Update the status of various shipments using tracking numbers on the USPS site

* PDF extraction to system interaction

Insurance: A broker processes a detailed project overview and creates a certificate of insurance with the specific details from the multi-page document by filling out an internal form Logistics: Fill out a shipping order form based on a PDF of the packing label Clinical: Enter patient appointment information into an EHR system based on a referral PDF Accounting: Extract invoice information from up to 50+ vendor formats and enter the details into a Google sheet without laborious OCR setup for specific formats Mortgage: Extract realtor names and address from a lease document and look up the license status on various state portals

* Self healing broken RPA workflows

Post reply on HN