Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sidekick: Local-first native macOS LLM app

github.com

Post reply on HN
Print view View original

11–20 of 97 posts

Re: Sidekick: Local-first native macOS LLM app

#11

Post by cadamsdotcom »

What an great looking tool.

Amazingly generous that it’s open source. Let’s hope the author can keep building it, but if they need to fund their existence there is precedent - lots of folks pay for Superwhisper. People pay for quality software.

In a past tech cycle Apple might’ve hired the author, acquired the IP and lovingly stewarded the work into a bundled OS app. Not something to hope for lately. So just going to hope the app lives for years to come and keeps improving the whole way.

Re: Sidekick: Local-first native macOS LLM app

#12

Post by pantulis »

mritchie712 wrote: post #5
aneutron wrote: post #2

[flagged]

I don't think Apple has missed out on much (yet). The best LLM's (e.g. gpt4o, sonnet 3.7) are no where near being able to run locally and still make mistakes. Some LLMs can run locally, but are brutally slow with small context windows. Apple is likely waiting until you can run a really good model on device (i.e. iOS), which makes sense to me. It's not like they're losing customers over this right now.

They are playing the long game, which is what has always been: wait until the silicon enables that for most users. The Apple Silicon track record suggests that... wait a couple of years and we'll get M3-Ultra-class capabilities in all of Apple devices. Some day the lowest bar will be above running state of the art LLMs on device.

Re: Sidekick: Local-first native macOS LLM app

#13

Post by nottorp »

argsnd wrote: post #7
nottorp wrote: post #6

> Image generation is availible on macOS 15.2 or above, and requires Apple Intelligence. ... so image generation is not fully offline? This tool looks like it could be worth a try to me, but only if I'm sure I can run it into a mode that's fully offline.

isn't apple intelligence image generation fully offline?

I don't know, I'm asking.

Only want it for some code so it looks like it can be fully offline, but it's worth being paranoid about it.

Re: Sidekick: Local-first native macOS LLM app

#14

Post by aneutron »

mritchie712 wrote: post #5
aneutron wrote: post #2

[flagged]

I don't think Apple has missed out on much (yet). The best LLM's (e.g. gpt4o, sonnet 3.7) are no where near being able to run locally and still make mistakes. Some LLMs can run locally, but are brutally slow with small context windows. Apple is likely waiting until you can run a really good model on device (i.e. iOS), which makes sense to me. It's not like they're losing customers over this right now.

All they have to show is incremental improvements over Siri. For that, Quantized models are more than enough in my opinion.

Re: Sidekick: Local-first native macOS LLM app

#15

Post by aa-jv »

Trying to put this through its paces, I first set out to build my own local binary (because why not, and also because code-reading is fun when you've got your own local build) ..

But I get this far:

/Users/aa-jv/Development/InterestingProjects/Sidekick/Sidekick/Logic/View Controllers/Tools/Slide Studio/Resources/bin/marp: No such file or directory

It seems there is a hand-built binary resource missing from the repo - did anyone else do a build yet, and get past this step?

Re: Sidekick: Local-first native macOS LLM app

#16

Post by greggsy »

nottorp wrote: post #6

> Image generation is availible on macOS 15.2 or above, and requires Apple Intelligence. ... so image generation is not fully offline? This tool looks like it could be worth a try to me, but only if I'm sure I can run it into a mode that's fully offline.

Some features use private cloud, but it's pretty decent in terms of security and privacy.

https://security.apple.com/blog/private-cloud-compute/

Re: Sidekick: Local-first native macOS LLM app

#19

Post by Gracana »

aa-jv wrote: post #15

Trying to put this through its paces, I first set out to build my own local binary (because why not, and also because code-reading is fun when you've got your own local build) .. But I get this far: /Users/aa-jv/Development/InterestingProjects/Sidekick/Sidekick/Logic/View Controllers/Tools/Slide Studio/Resources/bin/marp: No such file or directory It seems there is a hand-built binary resource missing from the repo -…

Looks like it wants https://github.com/marp-team/marp-cli

Re: Sidekick: Local-first native macOS LLM app

#20

Post by cpursley »

mritchie712 wrote: post #5
aneutron wrote: post #2

[flagged]

I don't think Apple has missed out on much (yet). The best LLM's (e.g. gpt4o, sonnet 3.7) are no where near being able to run locally and still make mistakes. Some LLMs can run locally, but are brutally slow with small context windows. Apple is likely waiting until you can run a really good model on device (i.e. iOS), which makes sense to me. It's not like they're losing customers over this right now.

Sonnet 3.7 best? That thing is a dumpster fire. Totally useless vs 3.5.
Post reply on HN