Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2023: The Year of AI

journal.everypixel.com

Post reply on HN
Print view View original

31–40 of 84 posts

Re: 2023: The Year of AI

#31

Post by guitarlimeo »

rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

This is generating answers a lot slower on my machine (X1 carbon) than GPT-3.5. :(

Re: 2023: The Year of AI

#32

Post by forevernoob »

rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

Would you say that this could run (smoothly) on my X220 with i7-2640M?

Re: 2023: The Year of AI

#33

Post by marcosdumay »

mitthrowaway2 wrote: post #23
ilaksh wrote: post #14

Earlier quoted context omitted.

The Makefile for llama.cpp is really good and doesn't require hacking to make it work.

Even on Windows?

It's probably a hell to run it on Windows. But again, for the audience here, that's not expected to be a large roadblock.

Re: 2023: The Year of AI

#34

Post by JanSt »

Kiro wrote: post #28
ilaksh wrote: post #21

Earlier quoted context omitted.

I think that might have been more true several 8-12 months ago. But now it feels like the momentum has swung towards open source. Multiple models are close to or exceeding GPT 3.5 now. They can do a lot of useful things. I have come to the conclusion that as much as possible I should try to wean myself off of OpenAI immediately. Because it's just not necessary or desirable to be tied to a single vendor anymore for ma…

Why are people benchmarking against 3.5? To me, the real race started with GPT-4. 3.5 and 4 are completely different beasts and that open models are catching up to 3.5 doesn't mean much. I've yet to see anything come close to 4.

Because 3.5 is a very useful model. Getting there with an open model already is pretty cool.

Re: 2023: The Year of AI

#35

Post by kleiba »

dartos wrote: post #11
andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

It’s because weights are where the magic lies, for the most part. Annnd nowadays tools like transformers can automatically select the architecture based on the name of the weights, so when interacting with LLMs, you generally just name the weights. Also MANY models are just using the llama or llama2 architecture.

> It’s because weights are where the magic lies, for the most part.

What's that supposed to mean? Weights are not independent of the underlying architecture.

Re: 2023: The Year of AI

#36

Post by xiphias2 »

andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

It probably means that AI is getting more use outside technical circles.

I would definitely include Mamba state space models and of course would prefer a technical review over a corporate review of the year.

Re: 2023: The Year of AI

#37

Post by Nullabillity »

api wrote: post #13

AI has been gradually improving for decades, but this is the year we finally noticed. The big thing was huge progress in natural language understanding. 2023 was the year the Turing test was smashed. Seeing computers win games, drive cars, optimize systems, even design things wasn’t as subjectively impressive to most of us as being able to talk to them. This was the year we first saw AI that could sort of communicate…

People were amazed by ELIZA back in '67. That doesn't mean that it did anything useful...

Re: 2023: The Year of AI

#38

Post by japanman185 »

swsieber wrote: post #5
rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

I appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments.…

[dead]

Re: 2023: The Year of AI

#39

Post by rightbyte »

forevernoob wrote: post #32
rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

Would you say that this could run (smoothly) on my X220 with i7-2640M?

Imagine some old sci-fi movie where the computer write out the answer to a query, as if it actually was some scene worker typing it to some terminal. On this:

    NVIDIA GeForce GTX 1050 Ti
    Intel® Core™ i5-8300H × 8
    32,0 GiB ram
So I guess your computer would be a faster typist?

Re: 2023: The Year of AI

#40

Post by yieldcrv »

swsieber wrote: post #5
rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

I appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments.…

check out LM Studio

no command line involved, has a huggingface browser built in to load in the model of the day, has a chat-like interface for chatGPT like use, and can create a local server to run your local model for your programs to interact with it using an API that is identical to OpenAI's

Post reply on HN