Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2023: The Year of AI

journal.everypixel.com

Post reply on HN
Print view View original

21–30 of 84 posts

Re: 2023: The Year of AI

#21

Post by ilaksh »

greatpostman wrote: post #6

TLDR: openai still crushing the competition. Competing models beat benchmarks, but are borderline worthless on real tasks.

I think that might have been more true several 8-12 months ago. But now it feels like the momentum has swung towards open source. Multiple models are close to or exceeding GPT 3.5 now. They can do a lot of useful things.

I have come to the conclusion that as much as possible I should try to wean myself off of OpenAI immediately. Because it's just not necessary or desirable to be tied to a single vendor anymore for many tasks. And in 2024 the open source capabilities will continue to increase. Soon everyone with a relatively new computer will be running things like LLMs locally. Within a couple of years it will be integrated into every OS or browser.

Re: 2023: The Year of AI

#22

Post by dmezzetti »

apwell23 wrote: post #20
dmezzetti wrote: post #8

Earlier quoted context omitted.

Going to call BS on this. I've delivered multiple projects this year using open models.

> delivered multiple projects Very curious about the type of projects you've delivered.

You can look at https://github.com/neuml/txtai. Biggest thing of 2023 was RAG with models like Mistral.

Re: 2023: The Year of AI

#23

Post by mitthrowaway2 »

ilaksh wrote: post #14
rightbyte wrote: post #9

Earlier quoted context omitted.

Ye ... with "just like this" I actually mean "proficient with compiling C projects on an Unixy system" which is like years of dev, "power user" or admin experience (not being sarcastic here). For the audience here I would say that "just" is about right though. I have no clue about anything LLM related. I just made it run after reading some comment on HN pointing in its direction. My point is that these locally run LL…

The Makefile for llama.cpp is really good and doesn't require hacking to make it work.

Even on Windows?

Re: 2023: The Year of AI

#24

Post by UrineSqueegee »

api wrote: post #13

AI has been gradually improving for decades, but this is the year we finally noticed. The big thing was huge progress in natural language understanding. 2023 was the year the Turing test was smashed. Seeing computers win games, drive cars, optimize systems, even design things wasn’t as subjectively impressive to most of us as being able to talk to them. This was the year we first saw AI that could sort of communicate…

not only talk to them, but ask them to perform tasks, sometimes obscure and they do it.

Re: 2023: The Year of AI

#25

Post by ruune »

rightbyte wrote: post #9
swsieber wrote: post #5

Earlier quoted context omitted.

I appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments.…

Ye ... with "just like this" I actually mean "proficient with compiling C projects on an Unixy system" which is like years of dev, "power user" or admin experience (not being sarcastic here). For the audience here I would say that "just" is about right though. I have no clue about anything LLM related. I just made it run after reading some comment on HN pointing in its direction. My point is that these locally run LL…

With almost every AI story I read on here (granted, I don't read that many, I find them quite boring), the top comment seems to be something like "you can do the same on your machine, see ollama". Case in point, your comment was the first one listed for me. So the interest for running LLMs on here seems quite high. Don't know why your post didn't catch anyone.

Re: 2023: The Year of AI

#26

Post by Kiro »

rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

Why would I run it myself when I can run GPT-4 for less than the electricity cost? Running these yourself is not free. OpenAI is heavily subsidized.

Re: 2023: The Year of AI

#27

Post by benrow »

apwell23 wrote: post #20
dmezzetti wrote: post #8

Earlier quoted context omitted.

Going to call BS on this. I've delivered multiple projects this year using open models.

> delivered multiple projects Very curious about the type of projects you've delivered.

Also curious.

I'm a sample of 1 and also relatively inexperienced. But I felt I quickly reached the limits of what was possible when I tried doing sentence classification with OSS sentence embedding models. The issue was with negation. I'd attributed too much magic to embedding models - they don't really understand language.

Not to say there isn't very capable tech out there. Just to add a datapoint that "sentiment analysis"-like approaches in blogs don't always scale to your particular use-case.

Edit: conscious I've drifted from the topic of chatbot type models, but felt relevant somehow.

Re: 2023: The Year of AI

#28

Post by Kiro »

ilaksh wrote: post #21
greatpostman wrote: post #6

TLDR: openai still crushing the competition. Competing models beat benchmarks, but are borderline worthless on real tasks.

I think that might have been more true several 8-12 months ago. But now it feels like the momentum has swung towards open source. Multiple models are close to or exceeding GPT 3.5 now. They can do a lot of useful things. I have come to the conclusion that as much as possible I should try to wean myself off of OpenAI immediately. Because it's just not necessary or desirable to be tied to a single vendor anymore for ma…

Why are people benchmarking against 3.5? To me, the real race started with GPT-4. 3.5 and 4 are completely different beasts and that open models are catching up to 3.5 doesn't mean much. I've yet to see anything come close to 4.

Re: 2023: The Year of AI

#29

Post by tempodox »

rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

I followed your instructions, and it works without a hitch. macOS-13.6 on an Intel iMac with a Radeon Pro 5700 GPU. My first direct contact with an LLM, and I don't understand most of the command line options yet, but it's certainly interesting.

Re: 2023: The Year of AI

#30

Post by phillipcarter »

rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

A key difference is that for basic use cases, yes, this is a comparable experience.

Where it stops being comparable is general application across literally millions of use cases. The ChatGPT system has proven itself a valuable utility across industries, people, and use cases. No open model I know of can match it yet.

Post reply on HN