Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: If we train an LLM with “data” instead of “language” tokens

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#41

Post by gwern »

This is an old idea dating back to the earliest algorithmic information theory work in the '50s/60s like Ray Solomonoff on universal induction: given a powerful learning algorithm, create AI by encoding all possible data as a binary stream (with no further preprocessing or engineering), and simply predict each successive bit. This leads to AIXI, the compression-as-intelligence paradigm (and online learning / meta-learning), and most recently, the success of self-supervised learning & generative modeling.

See for example Burfoot https://arxiv.org/abs/1104.5466 or Schmidhuber https://arxiv.org/abs/1511.09249#schmidhuber + https://arxiv.org/abs/1802.08864#schmidhuber (among many others) or (if I may shill my own work) https://gwern.net/aunn

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#42

Post by iandanforth »

Let's break this down. Large Language Models are almost all transformer models. Transformer models are sequence prediction models. There are other kinds of sequence prediction models like RNNs. Transformers are not tied to tokens, neither are RNNs. You can use them to predict sequences of values directly, you don't need to have high dimensional representations as input.

Now, why do LLMs work? They are exploiting structure. Language has structure, and language represents other structures. At a large enough scale the language models can learn and use that structure. There's quite a bit of consistency in language. You can train a model to be fluent (grammatically correct) without massive scale, but output often lacks meaning or coherency. It takes a lot of training to learn that "reading" a book and "reading" from a spinner hard drive platter are conceptually similar and extremely different in their details.

So, can you use a transformer with raw numerical data? Yes. Can you train a very large model on a very large amount of numerical data. Yes. Would you expect that training across a mixed corpus of raw numerical data would lead to a similar kind of 'understanding' that we've seen in LLMs? No. but it is not impossible.

Recall that language is just a layer on top of sensory processing. It helps a great deal but there is plenty of "intelligence" in the animal kingdom without it.

My personal opinion is that helping models build their own internal representations via curriculum learning and making sure it is being sent data which is generally related to, or correlated with, other data is very useful. If you don't have the aid of a universal sematic representation scheme like language, why not try your best to make it easy to make one?

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#43

Post by phillipcarter »

mdp2021 wrote: post #34
phillipcarter wrote: post #11

Earlier quoted context omitted.

> ethical systems I don't think symbolic AI has much to do with ethics? My impression is that ethics in this space are primarily concerned with the application of these technologies, especially when their deficiencies and biases are known.

The poster probably referred reliability as an ethical value; direct choice in building the system as an ethical value because embedded with your decisions hence responsibility; and conversely proposing "black-boxes" as an ethical fault. There are several possible reasons for this latter: not only that you should not be contented until you "could warranty" for "your" system, but also the possible outcomes. For exampl…

My immediate thought is that the ethical concerns with facial recognition in policing are orthogonal to the "kind" of AI involved. Clearly, our industry has moved to other techniques and my guess is because they're more capable. And so it comes down to regulating and adding accountability for the applications of the technology.

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#44

Post by T-A »

arbuge wrote: post #33
croes wrote: post #31

Earlier quoted context omitted.

But how do we communicate with an entity that understands data but not human language?

It doesn't seem to me that that's a problem. You just put data in and expect data out. For example, a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that, i.e. a weather forecast. Likewise financial market predictions etc.

> a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that

And it turns out to work really well:

https://arxiv.org/abs/2212.12794

predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#45

Post by T-A »

nailer wrote: post #15
gota wrote: post #12

Earlier quoted context omitted.

Isn't RLHF for building _chatbots_ over LLMs (and not part of LLMs themselves)?

RLHF is where the model reaches out to people when it's not confident of the output. It's good because it makes the model 'push into' classes the model is weak on.

That doesn't look like any definition of RLHF I'm familiar with, which amounts to first training a reward model on human feedback and then using that reward model to train the target model [1]. No reaching out to humans involved.

[1] https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#46

Post by claytonjy »

all2 wrote: post #29
kylebenzle wrote: post #9

We (I included) have been doing this EXACT thing for a long with with the stock market. We take ALL the data, build a model, then ask for an output buy/sell signal. The idea for using this type of model, from quants, was applied to language, THAT was the new idea. Now it sounds like you are saying "what if" we took a step backwards, but again, we this is exactly how we use these models now, is trained on data.

One question before I spend 50k training a model: but does it work? IE, does the income yield outpace the cost of training? I'm assuming that the model needs to be retrained at some frequency, though I'm guessing some of that retraining would be model fine tuning... could all of the training be continual fine tuning? Would there be any benefit by training from scratch?

Almost certainly not worth it on anything but an institutional scale, and even then most efforts fail or are huge wastes of time and money.

Stick to good data and simple techniques, and even then be aware you're extraordinarily unlikely to post higher gains than you'd get with DCA and index funds.

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#47

Post by sskates »

I am thinking about this problem at Amplitude. The holy grail of the martech space has always been to predict someone's next action based on what they've done before. What's exciting is that we have one of the largest datasets to be able to do that. Done right you could have products proactively do those actions/change itself to make those actions easier. If anyone is interested in this problem, please reach out!

More on it here: https://amplitude.com/blog/AI-powered-product-development

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#48

Post by sskates »

bluecoconut wrote: post #21

I love this question, because this is exactly what I'm currently focused on doing! I founded approximatelabs to essentially chase this out and show how much rich fruit there is in this space. See approximatelabs.com It is not traditional ML to me, which has a "row-wise" way of thinking. Instead, we are thinking columnar-ly (like you would if you were an analyst), and representing entities and attributes as tokens (no…

Wow, really cool! Could you apply this to the behavior dataset that we have at Amplitude? I'd love to talk more.

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#49

Post by alexwatson405 »

Hey there! Co-founder of Gretel.ai here, and I think I can provide some insights on this topic.

Firstly, the concept you're hinting at is not purely traditional ML. In traditional machine learning, we often prioritize feature extraction and engineering specific to a given problem space before training.

What you're describing and what we've been working on at Gretel.ai, is leveraging the power of models like Large Language Models (LLMs) to understand and extrapolate from vast amounts of diverse data without the need for time-consuming feature engineering. Here's a link to our open-source library https://github.com/gretelai/gretel-synthetics for synthetic data generation (currently supporting GAN and RNN-based language models), and also our recent announcement around a Tabular LLM we're training to help people build with data https://gretel.ai/tabular-llm

A few areas where we've found tabular or Large Data Models to be really useful are: * Creating privacy preserving versions of sensitive data * Creating additional labeled examples for ML training (much less expensive than traditional data collection/ml techniques) * Augmenting existing datasets with new fields, cleaning data, filling in missing values

Lots of mentions of RLHF here in the threads, one area I think RLHF will be super helpful is in ensuring that LLM data models return diverse and ethically fair results (hopefully better than the data they were trained on). Cheers!

Re: Ask HN: If we train an LLM with “data” instead of “language” tokens

#50

Post by nailer »

T-A wrote: post #45
nailer wrote: post #15

Earlier quoted context omitted.

RLHF is where the model reaches out to people when it's not confident of the output. It's good because it makes the model 'push into' classes the model is weak on.

That doesn't look like any definition of RLHF I'm familiar with, which amounts to first training a reward model on human feedback and then using that reward model to train the target model [1]. No reaching out to humans involved. [1] https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

You're right, I was confusing RLHF with https://en.wikipedia.org/wiki/Human-in-the-loop
Post reply on HN