Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we used GPT-4o for image detection with 350 similar illustrations

olup-blog.pages.dev

Post reply on HN
Print view View original

61–70 of 93 posts

Re: How we used GPT-4o for image detection with 350 similar illustrations

#61

Post by NegatioN »

sashank_1509 wrote: post #46

This has been my experience. Foundation models have completely changed the game of ML. Previously, companies might have needed to hire ML engineers familiar with ML training, architectures etc to get mediocre results. Now companies can just hire a regular software engineer familiar with foundation model API’s to get excellent results. In some ways it is sad, but in other ways the result you get is so much better than…

I haven't compared image models in a long while, so I don't know the relevant performance metrics. But even a few years ago, you would usually use a pretrained model, and then finetune on your own dataset though. So those models would also have "seen millions of images", and not just your 100k.

This change of not needing ML engineers is not so much about the models, as it is about easy API access for how to finetune a model, it seems to me?

Of course it's great that the models have advanced and become better, and more robust though.

Re: How we used GPT-4o for image detection with 350 similar illustrations

#63

Post by qeternity »

kredd wrote: post #16
qeternity wrote: post #14

Earlier quoted context omitted.

> I can think of at least two businesses that can be competed in costs if the team can automate a good chunk of it. And which would those be?

We both know I didn't write it down with the hopes that I'll act on the at some point in the near future, and want to avoid my imaginary competitors. Even though, in reality, I will ponder about it for another week or two, give up without actually getting anything done, then regret for never trying :)

We both know I was hoping you'd tell me anyway :)

Re: How we used GPT-4o for image detection with 350 similar illustrations

#64

Post by miki123211 »

Morizero wrote: post #17
Imnimo wrote: post #11

It's tough to judge without seeing examples of the targets and the user photos, but I'm curious if this could be done with just old-school SIFT. If it really is exactly the same image in the in the corpus and on the wall, does a neural embedding model really buy you a lot? A small number of high confidence tie points seems like it'd be all you need, but it probably depends a lot on just how challenging the user photo…

I find a lot of applied AI use-cases to be "same as this other method, but more expensive".

It's often vastly more expensive to inference, but vastly cheaper and faster to train / set up.

Many LLM use cases could be solved by a much smaller, specialized model and/or a bunch of if statements or regexes, but training the specialized model and coming up with the if statements requires programmer time, an ML engineer, human labelers, an eval pipeline, ml ops expertise to set up the GPUs etc.

With an LLM, you spend 10 minutes to integrate with the OpenAI API, and that's something any programmer can do, and get results that are "good enough".

If you're extremely cash-poor, time-rich and have the right expertise, making your own model makes sense. Otherwise, human time is more valuable than computer time.

Re: How we used GPT-4o for image detection with 350 similar illustrations

#65

Post by armchairhacker »

JayShower wrote: post #31

Alternative solution that would require less heavy lifting of ML but a little more upfront programming: It sounds like the cars are arranged in a grid on the wall. Maybe it would be possible to narrow down which car the user took a photo of by looking at the photos of the surrounding cars as well, and hardcoding into the system the position of each car relative to one another? Could potentially do that locally very q…

Put each painting in an artsy frame whose edges are each different, colorful pattern. When the user photographs the painting, they’ll include all (or even most) of the frame, and distinguishing the frames is easy.

Re: How we used GPT-4o for image detection with 350 similar illustrations

#66

Post by martin_a »

idkman_oops wrote: post #62
olup wrote: post #2

First time for me posting this kind of story - I thought it would make an interesting case on solving a hard computer vision problem with a crafty product engineer team.

Can you tell me what font is this?

One of these:

> ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace

Re: How we used GPT-4o for image detection with 350 similar illustrations

#67

Post by ac2u »

isoprophlex wrote: post #53
sashank_1509 wrote: post #46

This has been my experience. Foundation models have completely changed the game of ML. Previously, companies might have needed to hire ML engineers familiar with ML training, architectures etc to get mediocre results. Now companies can just hire a regular software engineer familiar with foundation model API’s to get excellent results. In some ways it is sad, but in other ways the result you get is so much better than…

This was exactly my experience being the ML engineer on a predictive maintenance project. We detected broken traffic signs in video feeds from trucks; first you segment, then you classify. Simply yeeting every "object of interest" into DINOv2 and running any cheap classifier on that was a game changer.

Could you elaborate? I thought DINO took images and outputted segmented objects? Or do you mean that your first step was something like a yolo model to get bounding boxes and you are just using dino to segment to make the classification part easier?

Re: How we used GPT-4o for image detection with 350 similar illustrations

#68

Post by yuvalr1 »

A completely different approach that don't require heavy AI would be an app on the user phone that does this:

1. Measure the distance from the wall (standard image processing)

2. Use the rotations of the gyro sensors on the phone to conclude which car is being looked at

I wonder if this could be as accurate though

Re: How we used GPT-4o for image detection with 350 similar illustrations

#69

Post by arkh »

armchairhacker wrote: post #65
JayShower wrote: post #31

Alternative solution that would require less heavy lifting of ML but a little more upfront programming: It sounds like the cars are arranged in a grid on the wall. Maybe it would be possible to narrow down which car the user took a photo of by looking at the photos of the surrounding cars as well, and hardcoding into the system the position of each car relative to one another? Could potentially do that locally very q…

Put each painting in an artsy frame whose edges are each different, colorful pattern. When the user photographs the painting, they’ll include all (or even most) of the frame, and distinguishing the frames is easy.

> artsy frame

Embedding a QR code or simply a barcode somewhere and you're done. Maybe hide it like a watermark so it does not show to the naked eye and doing some Fourier transform in the app won't require a network connection nor lot of processing power.

Re: How we used GPT-4o for image detection with 350 similar illustrations

#70

Post by famouswaffles »

rldjbpin wrote: post #45

reads to me like 95% of the "conventional AI" was applied to the problem and then using llm in the end seems to work like a lucky three-faced dice. when "embeddings" are used to perform closeness test, you are using a pretrained computer vision model behind the scenes. it is doing the far majority of tasks of filtering out hundreds of images down to a handful. visual llm works on textual descriptions that seem far to…

>visual llm works on textual descriptions

SOTA V-LLMs do not work on textual descriptions.

Post reply on HN