Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can I run AI locally?

canirun.ai

Post reply on HN
Print view View original

331–340 of 382 posts

Re: Can I run AI locally?

#331

Post by nunodonato »

novok wrote: post #281
steve_adams_86 wrote: post #253

Earlier quoted context omitted.

That's about it. The harness is still pretty rudimentary so I'm sure the system could be more capable, and that might reveal more interesting opportunities. I don't really know. So far I've got it orchestrating a few instances to dig through logs, local emails, git repositories, and github to figure out what I've been doing and what I need to do. Opus is waayyy better at it, but Qwen does a good enough job to actuall…

Oh man you just gave me an idea to use something like qwen 3.5 to categorize a lot of emails. You can keep the context small, do it per email and just churn through a lot of crap.

you can use 4B for that, its quite good

Re: Can I run AI locally?

#332

Post by modernerd »

Would love it more if it could help me to answer:

- Which models in the list are the best for my selected task? (If you don't track these things regularly, the list is a little overwhelming.) Sorting by various benchmark scores might be useful?

- How much more system resources do I need to run the models currently listed at F, D or C at B, A, or S-tier levels? (Perhaps if you hover over the score, it could tell you?)

Re: Can I run AI locally?

#334

Post by colechristensen »

mghfreud wrote: post #306
colechristensen wrote: post #291

Earlier quoted context omitted.

Can a Turing machine of any sort truly indistinguishably simulate a nondeterministic system? If a Turing machine can truly simulate a full nondeterministic system as complex as a philosopher but it would take dedicating every gram of matter in the visible universe for a trillion years to simulate one second, is this meaningfully different than saying it cannot? I suggest the answer to both questions are no, but the s…

I am not following what we are talking about here. I am a basic human being, I cannot truly simulate a nondeterministic system. Does it mean “I am not thinking”?

I'm saying a Turing machine cannot simulate you. You don't need to simulate you because you are you.

Re: Can I run AI locally?

#335

Post by rando1234 »

On a related question, I'm in the market to buy a new laptop for development and want to get something with good support for local models. What is a good recommendation in terms of GPU support etc? I currently have a Dell XPS 13. Should I just get a MacBook? Or are there good non-Mac options?

Re: Can I run AI locally?

#337

Post by hnfong »

colechristensen wrote: post #291
hnfong wrote: post #268

Earlier quoted context omitted.

If we accept the Church-Turing thesis, a philosopher can be simulated by a simple Universal Turing machine. If one day we are able to create a philosopher from such a rudimentary machine (and a lot of tape), would you consider that very much towards the "rock" end as well?

Can a Turing machine of any sort truly indistinguishably simulate a nondeterministic system? If a Turing machine can truly simulate a full nondeterministic system as complex as a philosopher but it would take dedicating every gram of matter in the visible universe for a trillion years to simulate one second, is this meaningfully different than saying it cannot? I suggest the answer to both questions are no, but the s…

One thing about Turing Machines that some people might miss is that the "paper tape, finite alphabet and internal states" thing is actually intended to model a human thinking out loud (writing their thoughts down) on a piece of paper.

It was designed to make it hard to argue that the answers to your questions are "no".

Of course there are caveats where the Turing machine model might not have a direct map onto human brains, but it seems the onus would be for one to explain why, for example, non-determinism is essential for a philosopher to work.

That said,

> Can a Turing machine of any sort truly indistinguishably simulate a nondeterministic system?

Given how AI has improved in its ability to impersonate human beings in recent years, I don't see why not. At least, the current trend does not seem to be in your favor.

I can see why you think the answer is "no". My understanding is that QM per se is mostly a distraction, but some principles underlying QM (some subjectivity thing) might be relevant here.

My best guess is that the AI tech will eventually be able to replicate a philosopher to arbitrary "accuracy", but there will always be an indescribable "residue" where one could still somehow detect that it is not a real human. I suspect this "residue" is not explainable using materialistic mechanisms though.

Re: Can I run AI locally?

#338

Post by nl »

novok wrote: post #281
steve_adams_86 wrote: post #253

Earlier quoted context omitted.

That's about it. The harness is still pretty rudimentary so I'm sure the system could be more capable, and that might reveal more interesting opportunities. I don't really know. So far I've got it orchestrating a few instances to dig through logs, local emails, git repositories, and github to figure out what I've been doing and what I need to do. Opus is waayyy better at it, but Qwen does a good enough job to actuall…

Oh man you just gave me an idea to use something like qwen 3.5 to categorize a lot of emails. You can keep the context small, do it per email and just churn through a lot of crap.

The 0.8B can do this pretty well.

Actually pg's original "A plan for spam" explains how to do this with a Bayesian classifier.

Re: Can I run AI locally?

#339

Post by hnfong »

sayamqazi wrote: post #305
colechristensen wrote: post #291

Earlier quoted context omitted.

Can a Turing machine of any sort truly indistinguishably simulate a nondeterministic system? If a Turing machine can truly simulate a full nondeterministic system as complex as a philosopher but it would take dedicating every gram of matter in the visible universe for a trillion years to simulate one second, is this meaningfully different than saying it cannot? I suggest the answer to both questions are no, but the s…

You are claiming that intelligence and even consciousness are non-deterministic entties at core. This is a huge claim and requires incredible proof.

I'll add that rocks are, if needed, objects that can exhibit quantum behavior.

In classical computing, we design chips to avoid the quantum behavior, but there's nothing in theory to prevent us from building an equivalent quantum Turing machine using "rocks".

Re: Can I run AI locally?

#340

Post by alexpotato »

nl wrote: post #338
novok wrote: post #281

Earlier quoted context omitted.

Oh man you just gave me an idea to use something like qwen 3.5 to categorize a lot of emails. You can keep the context small, do it per email and just churn through a lot of crap.

The 0.8B can do this pretty well. Actually pg's original "A plan for spam" explains how to do this with a Bayesian classifier.

I was just chatting with a co-worker that wanted to run a LLM locally to classify a bunch of text. He was worried about spending too many tokens though.

I asked him why he didn't just have the LLM build him a python ML library based classifier instead.

The LLMs are great but you can also build supporting tools so that:

- you use fewer tokens

- it's deterministic

- you as the human can also use the tools

- it's faster b/c the LLM isn't "shamboozling" every time you need to do the same task.

Post reply on HN