[dead]
Computer use in Gemini 3.5 Flash
141–150 of 175 posts
Re: Computer use in Gemini 3.5 Flash
#142Earlier quoted context omitted.
My go-to for this is to screenshot and use the built-in text extraction in the screenshot tool (I'm on a mac), then pass on that text data to whatever processing. It's a pretty good tool so long as the PDF is in OK shape (I've had errors in scanned images).
It's so horrible that in 2026 people are still publishing important data and specifications in a format like PDF that's difficult for LLMs to consume. We need to drag them kicking and screaming to HTML or Markdown. Heck, even Microsoft Word DOCX is superior for reliable parsing and content extraction.
Good luck, getting rid of PDFs is going to be as hard as migrating from JPEG everywhere.
Re: Computer use in Gemini 3.5 Flash
#143Earlier quoted context omitted.
The PDF reader for Gemini is extraordinarily poor in my experience. I like the writing style of this model a little better, but for most tasks people would use AI for, Gemini is probably not what you want to be using.
what is a good way to read PDFs using AI?
MinerU works well to get it markdown
Re: Computer use in Gemini 3.5 Flash
#144Earlier quoted context omitted.
Computer use is a great idea. It gets the job done when nothing else will. If you're a person trying to get their job done at a big company, but half your job is in 1-2 proprietary tools or is stuck behind an API you can't program against, computer use can allow you, a non-techie, to do your job more efficiently. I think it's an awesome way to circumvent gate keepers and the IT department to let people accomplish the…
That is an incredibly niche use case and comes with a boatload of footguns. Even then, an AI writing AHK scripts likely outperforms.
It does. I used to be an ahk "script kiddie" and know it front and back. It's sort of burnt into my brains. As a result, I can prompt really really well, notice issues at a glance, and I have a sheer volume of scripts locally for all sorts of tasks some from as far back as 2014. From tiling window managers to OCR all the way to simple hotkeys/hotstrings. I let it grep in that folder and build out whatever I want using those primitives. This gives actually 1-shot immediately usable 100% working scripts even with GPT3.5 level models, as opposed to the iterations needed for typical development.
Example: adding copyright text box to bottom of every slide
F3::
pres := ComObjActive("PowerPoint.Application").ActivePresentation
Loop % pres.Slides.Count {
slide := pres.Slides.Item(A_Index)
box := slide.Shapes.AddTextbox(1, 100, 500, 500, 30)
box.TextFrame.TextRange.Text := "Copyright 2026. All Rights Reserved."
}
returnRe: Computer use in Gemini 3.5 Flash
#145Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”. Stunned to…
I just tried this and it worked without issue. Some considerations: 1) tell it to extra t the data (in a new session) does that work? 2) if it doesn't, could there be something up with the PDF? As many commentors suggested, this works well with Gemini so there is likely a missing variable in play. Share your prompt and the PDF and let's see if we can determine what.
Long shot, but I wonder if an image of the pdf would do better if it did get unstuck on internal formats.
Re: Computer use in Gemini 3.5 Flash
#146Earlier quoted context omitted.
Antigravity CLI (which replaced Gemini CLI): https://antigravity.google/product/antigravity-cli
So a couple weeks ago I decided to pay some money for the Gemini API cause I'd found myself getting a lot of use out of the free tier chat and figured they deserved some of my cash. First headache was a lot of delays and 'service unavailable due to excessive load messages'. Second headache was a lot of frustration with the Continue plugin in my IDE. Gemini chat suggests I try the Antigravity app. I do so. IT's OK. La…
+1 to service unavailable annoyances.
It has been close to unusable for anything serious. I did really like the ability of Gemini-pro models to ground their research using Google Search. This meant that they were often much more thorough and up to date in their recommendations and finding solutions that came to life after the models themselves were trained. But even using Gemini as a reviewer was a weak point in my harness because of the poor reliability of their service (529 or 503).
I’ve since paid for a search API(linkup, exa, and valyu) and hooked them up to Deepseekv4-pro. It has been doing a stellar job.
The key was to prompt them to systematically use search to validate their answers(not simply “use the tool”, but something like find possibilities using web search, then once you formulate a solution, validate it with this check list -
1. Is there a better way to do this in 2026?
2. Are the libraries and its docs you’re using up to date?
This seems to help very much based on experience.
Re: Computer use in Gemini 3.5 Flash
#147stopped using gemini a couple months ago when they ruined their rate limits. Not sure why people still use em.
They were good with the generous rate limits in antigravity. but done after that.
Re: Computer use in Gemini 3.5 Flash
#148Earlier quoted context omitted.
It doesn't need to be that kind of QA. Even just a basic "I want the AI to build the beginnings of a GUI app for me" will work much better if the AI can see the output of its work and iterate on it. Similar if you want the AI to fix a GUI bug—much better if you can show it the the bug and tell it how to test to see when it's gone.
the LLM does not require computer use to see the GUI and, again, that's a pretty niche use and not what Computer Use is being marketed for
> not what Computer Use is being marketed for
Okay, fair, I haven't really paid attention to marketing.
> the LLM does not require computer use to see the GUI and
It can take screenshots without computer use, but it can't click around. I didn't have access to computer use until recently (I'm on an OS where Claude Code technically shouldn't run, I had to patch the binary), and when I got it working it made a big difference because of this.
Re: Computer use in Gemini 3.5 Flash
#149Earlier quoted context omitted.
what is a good way to read PDFs using AI?
In my experience it really depends on what sort of pdfs you are trying to extract (ie what the content is). For regular pdfs that have been produced in a “normal way” (ie using latex or a modern application with a “save to pdf” function) will contain the text and for those I’ve had a lot of success on general pdfs using pypdf. “Image” Pdfs that have been produced via a scan so don’t actually contain a text transcript…
> regular pdfs that have been produced in a “normal way” (ie using latex or a modern application with a “save to pdf” function) will contain the text
Producing "normal PDFs" that way actually requires specific LaTeX options to be enabled in my experience. Without that, PDF viewers have to perform all kinds of ugly hacks to even figure out what Unicode codepoint a given glyph is supposed to represent! PDFs are much more of a vector format than a layouting program than most people seem to realize.
> One thing I have consistently found useful is to look for “mojibake” (scrambled text caused by decoding in an unintended character encoding)
This is exactly the problem with PDFs: It's not regular mojibake (i.e. interpreting a string of text in the wrong charset), but rather some PDF processor's failed attempt at mapping glyphs back to codepoints without an explicit mapping table being present in the PDF, which is something that the creator actively has to do.
> “Image” Pdfs that have been produced via a scan so don’t actually contain a text transcript require actual OCR.
For the reason above and others, in my experience, OCR actually works significantly better than trying to "semantically parse" the PDF.
Re: Computer use in Gemini 3.5 Flash
#150Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”. Stunned to…
That's interesting because my experience has been almost the opposite. A few months ago I tested Gemini on converting screenshots of tables from PDF files into CSV. I tried it on several different tables and it got every one right. It consistently outperformed ChatGPT.
The key here is that you used screenshots. This forces Gemini into "OCR mode" (i.e. actually looking at vision tokens) rather than trying to be clever with its tool calls.
The latter strategy almost entirely depends on the quality of the skills and tool calls exposed to a given agent.