Ask HN: Any way to borrow compute from Apple M1

#1

Post by 2Gkashmiri » Sat, Jul 13, 2024, 6:44 AM UTC

Hi.

i have a friend who owns an m1 max. i would like to "borrow" his gpu for llama 3 or SD. is there a way for me to use his compute when idle ? i do not want to remote into his machine, an easy local api would be fine (i could tailscale/zerotier) and then get the api that way.

Re: Ask HN: Any way to borrow compute from Apple M1

#2

Post by neximo64 » Sat, Jul 13, 2024, 7:03 AM UTC

Use Ollama's api

Re: Ask HN: Any way to borrow compute from Apple M1

#3

Post by gostsamo » Sat, Jul 13, 2024, 7:09 AM UTC

neximo64 wrote: post #2

Use Ollama's api

Or OpenWebUI over it if you want acceptable ui experience.

Re: Ask HN: Any way to borrow compute from Apple M1

#4

Post by 2Gkashmiri » Sat, Jul 13, 2024, 7:18 AM UTC

gostsamo wrote: post #3

neximo64 wrote: post #2

Use Ollama's api

Or OpenWebUI over it if you want acceptable ui experience.

cool

Re: Ask HN: Any way to borrow compute from Apple M1

#5

Post by [deleted] » Thu, Jul 18, 2024, 9:31 AM UTC

[deleted]

Re: Ask HN: Any way to borrow compute from Apple M1

#6

Post by BossingAround » Thu, Jul 18, 2024, 9:31 AM UTC

An off-topic question, are Apple's M-series chips any good at current AI/ML work? How does it compare with dedicated GPUs?

Re: Ask HN: Any way to borrow compute from Apple M1

#7

Post by theodric » Thu, Jul 18, 2024, 9:38 AM UTC

BossingAround wrote: post #6

An off-topic question, are Apple's M-series chips any good at current AI/ML work? How does it compare with dedicated GPUs?

Mid-tier gaming GPU performance, but (potentially) access to gobs of memory (if running on a host with gobs of memory) owing to the unified memory design. For certain use cases which require loading huge datasets but don't necessarily require massive compute (i.e. inference on large models) they can be cost-competitive relative to something like an H100.

Re: Ask HN: Any way to borrow compute from Apple M1

#8

Post by rcarmo » Thu, Jul 18, 2024, 10:23 AM UTC

neximo64 wrote: post #2

Use Ollama's api

This. Works fine. The M1 can run most small models (phi3, gemma, etc.) at usable speeds even with just 8GB of RAM.

Re: Ask HN: Any way to borrow compute from Apple M1

#9

Post by nsbk » Thu, Jul 18, 2024, 10:25 AM UTC

neximo64 wrote: post #2

Use Ollama's api

This! Tailscale plus Ollama API will definitely do the job

Re: Ask HN: Any way to borrow compute from Apple M1

#10

Post by lordnacho » Thu, Jul 18, 2024, 10:34 AM UTC

> i do not want to remote into his machine

> tailscale/zerotier

Same thing isn't it?

In any case it wouldn't be hard for you to just have an account on his machine, tailscale being perhaps the simplest setup. SSH in, cook his laptop at your leisure.