Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2023: The Year of AI

journal.everypixel.com

Post reply on HN
Print view View original

11–20 of 84 posts

Re: 2023: The Year of AI

#11

Post by dartos »

andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

It’s because weights are where the magic lies, for the most part.

Annnd nowadays tools like transformers can automatically select the architecture based on the name of the weights, so when interacting with LLMs, you generally just name the weights.

Also MANY models are just using the llama or llama2 architecture.

Re: 2023: The Year of AI

#12

Post by ilaksh »

swsieber wrote: post #5
rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

I appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments.…

The Makefile for llama.cpp is really good, it will work the way he said.

You can also use ollama which makes it into a one line install and then a simple command to run any popular model.

Re: 2023: The Year of AI

#13

Post by api »

AI has been gradually improving for decades, but this is the year we finally noticed.

The big thing was huge progress in natural language understanding. 2023 was the year the Turing test was smashed.

Seeing computers win games, drive cars, optimize systems, even design things wasn’t as subjectively impressive to most of us as being able to talk to them. This was the year we first saw AI that could sort of communicate with us the way we can with each other.

Re: 2023: The Year of AI

#14

Post by ilaksh »

rightbyte wrote: post #9
swsieber wrote: post #5

Earlier quoted context omitted.

I appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments.…

Ye ... with "just like this" I actually mean "proficient with compiling C projects on an Unixy system" which is like years of dev, "power user" or admin experience (not being sarcastic here). For the audience here I would say that "just" is about right though. I have no clue about anything LLM related. I just made it run after reading some comment on HN pointing in its direction. My point is that these locally run LL…

The Makefile for llama.cpp is really good and doesn't require hacking to make it work.

Re: 2023: The Year of AI

#15

Post by ewweezdsd »

greatpostman wrote: post #6

TLDR: openai still crushing the competition. Competing models beat benchmarks, but are borderline worthless on real tasks.

How do you define "real tasks"? Even small models that can be run on consumer CPU can produce a coherent summary of an email, as an example. Isn't that a real task?

Re: 2023: The Year of AI

#16

Post by bugglebeetle »

andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

IIRC Mistral’s architecture is llama-2. They just trained it from scratch with undisclosed data and techniques.

Re: 2023: The Year of AI

#18

Post by ilaksh »

andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

I think it did with Mixtral and MoE.

I think people talk about the models because the architectural details are generally inaccessible for us since we don't have sufficient training (which it requires quite a lot to have an intuitive understanding I believe). Whereas the models are easily downloaded and tested.

Re: 2023: The Year of AI

#19

Post by thefz »

rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

> Haven't people realized what they can run themself?

Lots, lots of people do not care.

Re: 2023: The Year of AI

#20

Post by apwell23 »

dmezzetti wrote: post #8
greatpostman wrote: post #6

TLDR: openai still crushing the competition. Competing models beat benchmarks, but are borderline worthless on real tasks.

Going to call BS on this. I've delivered multiple projects this year using open models.

> delivered multiple projects

Very curious about the type of projects you've delivered.

Post reply on HN