21–30 of 456 posts
90% accuracy +/- 10%? What could that be useful for, that’s awfully low.
This is "good enough" for Banks to use when doing due diligence. You'd be surprised how much noise is in the system with the current state of the art: algorithms/web scrapers and entire buildings of humans in places like India.
This is what I am trying to figure out how to solve. My problem statement is: - Injest PDFs, summarize, and extract important information. - Have some way to overlay the extracted information on the pdf in the UI. - User can provide feedback on the overlaid info by accepting or rejecting the highlights as useful or not. - This info goes back in to the model for reinforced learning. Hoping to find something that can m…
Earlier quoted context omitted.
> * Huge context window But how well does it actually handle that context window? E.g. a lot of models support 200K context, but the LLM can only really work with ~80K or so of it before it starts to get confused.
it works REALLY well. I have used it to dump many references codes and then help me write a new modules etc. I have gone up to 200k tokens I think with no problems in recall.
Something like this opens up a lot of use cases.
Is there an AI platform where I can paste a snip of a graph and it will generate a n th order polynomial regression for me of the trace?
I'd probably try those first, since otherwise you're depending on the language model to do the right thing automagically.
Gemini developer experience was stupidly easy. Easy to add a file "part" to a prompt. Easy to focus on the main problem with weirdly high context window. Multi-modal so it handles a lot of issues for you (PDF image vs. PDF with data), etc. I can recommend it for the use case presented in this blog (ignoring the bounding boxes part)!
>Unfortunately Gemini really seems to struggle on this, and no matter how we tried prompting it, it would generate wildly inaccurate bounding boxes This is what I have found as well. From what I've read, LLMS do not work well with images for specific details due to image encoders which are too lossy. (No idea if this is actually correct.) For now I guess you can use regular OCR to get bounding boxes.
I suspect the issue is prompt engineering related.
> Please provide me strict bounding boxes that encompasses the following text in the attached image? I'm trying to draw a rectangle around the text.
> - Use the top-left coordinate system
> - Values should be percentages of the image width and height (0 to 1)
LLMs have enough trouble with integers (since token-wise integers and text representation of integers are the same), high-precision decimals will be even worse. It might be better to reframe the problem as "this input document is 850 px x 1100 px, return the bounding boxes as integers" then parse and calculate the decimals later.
I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…