Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2023: The Year of AI

journal.everypixel.com

Post reply on HN
Print view View original

51–60 of 84 posts

Re: 2023: The Year of AI

#51

Post by [deleted] »

bamboozled wrote: post #48
chime wrote: post #45

Earlier quoted context omitted.

Things are already changing for early adopters. Yesterday, I used a PDF GPT to review an 80+ page contract and GPT4 to explain specific parts of it in depth using layman's terms. I could have hired a lawyer like I have done in the past but that itself is a slow, laborious process and I often feel like my questions don't get thoroughly answered. I feel very confident of my understanding of the contract now. I also use…

When we use it to solve real problems, make some real progress on climate change, then it will be something else from hype and it will be the year of AI. When you as an individual use it to sort PDFs or something it’s really hype.

[deleted]

Re: 2023: The Year of AI

#52

Post by fragmede »

vasco wrote: post #42
rightbyte wrote: post #3

This article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.…

A few years ago peak HN comment was complaining about wget | sh. Nowadays we just run full on opaque models directly from huggingface without thinking twice about reading anything. Interesting how times change. I wonder what supply chain attacks will come from huggingface, must not be long now.

That is, by the by, why models are no longer distributed as Python Pickle files, which can root your box if the model being loaded is malicious.

Re: 2023: The Year of AI

#53

Post by YetAnotherNick »

simbolit wrote: post #50
phillipcarter wrote: post #30

Earlier quoted context omitted.

A key difference is that for basic use cases, yes, this is a comparable experience. Where it stops being comparable is general application across literally millions of use cases. The ChatGPT system has proven itself a valuable utility across industries, people, and use cases. No open model I know of can match it yet.

They don't have to match the market leader, they just have to be "good enough". There are oodles of use-cases where sending your data to an outside provider is a complete no-go. In these cases OpenAI/Google/whoever-products aren't relevant competition.

I keep hearing this, but which company has this policy? The privacy policy of openAI's enterprise(or Azure's) is not a lot different than say AWS, which everyone uses.

Re: 2023: The Year of AI

#54

Post by benlivengood »

mitthrowaway2 wrote: post #23
ilaksh wrote: post #14

Earlier quoted context omitted.

The Makefile for llama.cpp is really good and doesn't require hacking to make it work.

Even on Windows?

It builds on Windows with a couple tweaks to the includes with mingw64, I think some of the Windows-related includes need to be explicitly included.

This was a few months ago and things may be better now.

Re: 2023: The Year of AI

#55

Post by YetAnotherNick »

andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

Even in Resnet era, data was the most relevant for accuracy. Google had this internal JFT data, training on which improves imagenet score much larger than different architecture. Also most of the gains after that came from data augmentation techniques or bigger model rather than new architecture.

Re: 2023: The Year of AI

#56

Post by YetAnotherNick »

xiphias2 wrote: post #36
andy99 wrote: post #4

Something I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-…

It probably means that AI is getting more use outside technical circles. I would definitely include Mamba state space models and of course would prefer a technical review over a corporate review of the year.

For all we know, Mamba could be overhyped and could only work for small models. Remember retnet[1] which said the same thing, equal to transformer in linear time.

[1]: Retentive Network: A Successor to Transformer for Large Language Models: https://arxiv.org/abs/2307.08621

Re: 2023: The Year of AI

#57

Post by phillipcarter »

simbolit wrote: post #50
phillipcarter wrote: post #30

Earlier quoted context omitted.

A key difference is that for basic use cases, yes, this is a comparable experience. Where it stops being comparable is general application across literally millions of use cases. The ChatGPT system has proven itself a valuable utility across industries, people, and use cases. No open model I know of can match it yet.

They don't have to match the market leader, they just have to be "good enough". There are oodles of use-cases where sending your data to an outside provider is a complete no-go. In these cases OpenAI/Google/whoever-products aren't relevant competition.

Part of the problem is that "good enough" can be really difficult to figure out, whereas GPT-3.5 (and of course GPT-4) are almost a guaranteed success with just basic prompting and context fed in via the prompt.

And yes, there are indeed use cases where sending data to an outside provider is a no-go. The bet OpenAI is making is that they can solve for that later while building their business on use cases where it's fine to send data to an outside parameter. It may also simply not be something they care about. In my own work I know of a massive financial enterprise that has prioritized ~30 or so features where it's fine to send that data. OpenAI is not struggling to get their money.

It remains to be seen if OpenAI will also capture this market, or if fine-tuning open models to be "good enough" wins out over time. The point isn't that, though. The point is that their models are so broadly applicable that _anyone_ can get some value quickly without much work.

Re: 2023: The Year of AI

#58

Post by YetAnotherNick »

ilaksh wrote: post #21
greatpostman wrote: post #6

TLDR: openai still crushing the competition. Competing models beat benchmarks, but are borderline worthless on real tasks.

I think that might have been more true several 8-12 months ago. But now it feels like the momentum has swung towards open source. Multiple models are close to or exceeding GPT 3.5 now. They can do a lot of useful things. I have come to the conclusion that as much as possible I should try to wean myself off of OpenAI immediately. Because it's just not necessary or desirable to be tied to a single vendor anymore for ma…

> Multiple models are close to or exceeding GPT 3.5 now

Only in benchmarks as GP said. All the models I have tried either hallucinates like crazy or refuse to answer everything. Nowhere close to GPT 3.5.

Re: 2023: The Year of AI

#59

Post by stavros »

yieldcrv wrote: post #40
swsieber wrote: post #5

Earlier quoted context omitted.

I appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments.…

check out LM Studio no command line involved, has a huggingface browser built in to load in the model of the day, has a chat-like interface for chatGPT like use, and can create a local server to run your local model for your programs to interact with it using an API that is identical to OpenAI's

This is the easiest way, by far. Then it's Ollama, then llama.cpp. But LM studio is just "install this app, select an LLM you want, done".
Post reply on HN