Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chipotlai Max

github.com

Post reply on HN
Print view View original

51–60 of 71 posts

Re: Chipotlai Max

#51

Post by tomashubelbauer »

skinfaxi wrote: post #50
tomashubelbauer wrote: post #49

Earlier quoted context omitted.

I have a personal, fully offline and local version of Windows Recall basically, but good, made using macOS built-in OCR and LLM. The reasoning requirements are tiny (just interpret the screen based on the OCR, do rolling de-duplication and summarization), but they are non-zero. The tool is valuable to me and it being dep-free and fully offline and local just gives me a good feeling.

Would you ever consider writing up or sharing your setup?

The ingredients are:

1. Bun.Cron API to run a script every minute

2. Bun.$ (Bun Shell) to execute the macOS command to take a screenshot (I do this for all connected screens at that moment)

3. Bun.Image to downscale everything to 1x in case some of the screenshots are 2x

4. Bun Shell again to run a JXA AppleScript thing to use the Vision Framework or whatever it is called to OCR the image into a file

5. Bun Shell to run the Swift compiler in the one-off eval mode with inline Swift helper that runs the Foundation Models Framework built-in LLM with a system prompt that tells it what the OCR said and instructs it to glean what may be on the screen (can't do this with JXA because the models are not exposed with ObjC APIs)

6. For each screenshot, continuously, take the previous day summary file and the last OCR/context results and produce a new summary of the day

I plan on adding extra information from the OS like the currently opened windows, currently focused window, time of day etc. into the mix, but so far it hasn't been needed. It produces reports of a good enough quality for me.

I `grep` these daily summaries whenever I need to recall a link I saw or a find what channel a message I spotted was in or take another look at that one tab I already closed, maybe re-open it by its OCR'd URL etc.

Re: Chipotlai Max

#52

Post by forlorn_mammoth »

bschwindHN wrote: post #40
Mashimo wrote: post #35

Earlier quoted context omitted.

> gemini The gemini from your phone? I mean yeah, that is what it was designed to do. It's one of the better coding LLMs out there.

Oops, I left out the context of "the gemini button in google maps", sorry. It appeared one day and I didn't want to press it while driving and screw up my route. It's supposed to assist you with route-related things, but yeah it's of course still a general purpose LLM backing it.

I always drive better when my passenger recites the prime numbers in order. That sequence above 2^n-1 is just gold to my ears!

Re: Chipotlai Max

#53

Post by darkwater »

hung wrote: post #12

Reminds me of when I used the Amazon.com AI Chatbot (was called Rufus and they renamed it to Alexa for shopping) to do things like write fizbuzz etc. Looks like they patched it to refuse though.

Came here to say the same. I haven't tried in months but Rufus definitely spat out Python code from within the Amazon shopping app. I just had to use English instead of the local language.

Re: Chipotlai Max

#54

Post by drob518 »

avaer wrote: post #4

NAL but I'd be worried about treading into CFAA territory with things like this. In the US, the law allows draconian penalties if you find yourself on the wrong side. Something like yt-dlp is just downloading public data, which I can see being defensible as automating the use of a service. But this commandeers remote machine resources to do your compute in ways clearly not intended by the provider. I don't know how e…

Yep, the key phrase is “misuse of computing resources,” if I remember correctly. IANAL, however.

That said, kudos for creativity.

Re: Chipotlai Max

#55

Post by jedbrooke »

tomashubelbauer wrote: post #28
jedbrooke wrote: post #14

I’d been thinking about if something like this would be possible for https://chatjimmy.ai/ . The underlying model is only llama 3 8B but I’m curious what coding harnesses would be like at 17k tok/s

If you're on macOS you can try the built in LLM which I think is similar in size. There's a project called Apfel that wraps it in a CLI. Also Chrome ships with a web API called Prompt API that gives you offline access to Gemini Nano which can do both text and images at the input. Also tiny. I've integrated these into my workflows where a tiny but non zero amount of reasoning is needed in between the otherwise fully d…

looks like the macOS one is Tahoe only. I’ve been putting of upgrading to tahoe but this might be enough to tempt me

Re: Chipotlai Max

#56

Post by OrangeMusic »

avaer wrote: post #4

NAL but I'd be worried about treading into CFAA territory with things like this. In the US, the law allows draconian penalties if you find yourself on the wrong side. Something like yt-dlp is just downloading public data, which I can see being defensible as automating the use of a service. But this commandeers remote machine resources to do your compute in ways clearly not intended by the provider. I don't know how e…

> In the US,

Re: Chipotlai Max

#57

Post by boston_clone »

OrangeMusic wrote: post #56
avaer wrote: post #4

NAL but I'd be worried about treading into CFAA territory with things like this. In the US, the law allows draconian penalties if you find yourself on the wrong side. Something like yt-dlp is just downloading public data, which I can see being defensible as automating the use of a service. But this commandeers remote machine resources to do your compute in ways clearly not intended by the provider. I don't know how e…

> In the US,

I’m not a lawyer, but Chipotle is a US company and this github repo belongs to a US citizen currently residing and employed in New York, so US law might apply here.

Re: Chipotlai Max

#58

Post by nhecker »

luca-ctx wrote: post #27
schmichael wrote: post #23

give ai a self-preservation directive and let them do this for you: automatically switching models to keep themselves alive. Living off of whatever token source they can find in the wild. Surely agents can farm their own tokens through the numerous support chats, free trials, leaked keys, and whatever other sources of token generation haven’t been adequately captcha’d. An agent could forage for token sources all nigh…

OpenRouter has lots of free model providers (you pay by letting them train on it) if you actually wanted to do something like this but legally.

There's also Horde or Koboldai.net or Koboldai.com or whatever their project is named, if you want a community-driven version of this. You can play with it via a WebGUI at https://lite.kobaldai.net, or with an API token of all zeroes. (Or, an actual API key associated with your user.)

> The AI Horde is a service that generates text using crowdsourced GPUs run by independent volunteer workers.

Re: Chipotlai Max

#60

Post by evan_ »

There was a really great, really underrated show on Peacock a few years ago called Mrs. Davis. The titular Mrs. Davis was an all-encompassing AI assistant that had taken over all of the governments of the world. Everyone wore a Whispering Earring-style earbud with a voice that guided them through their lives and made every decision for them. (Betty Gilpin plays a nun who's simultaneously rebelling against the AI and searching for the Holy Grail on its behalf.)

In the show the AI had originally started as (spoiler, but not really) the customer support bot for Buffalo Wild Wings. I don't know what to do with this.

Post reply on HN