Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

Well over 40 years ago I certainly wasn't working on language models with only 16 kilobytes of memory and a 1MHz microprocessor.

No "high-level" or human-readable language anyway.

OTOH I always attempted to use the electronics further toward the limit of what it could provide, compared to average, and once the resource limit was reached (which occurred fairly quickly with only 16k) then the entire effort concentrated on maximizing the amount of machine learning that could be accomplished by code that fit in the memory.

There was no distraction preparing for more powerful hardware to come, it wasn't going to be coming during the time period needed.

No real artifical intelligence evolved, and there was nothing "general" about it.

The idea was to select & collect the desired most useful inferences from the raw data and make them available to the operator's natural intelligence for all the high-level decision-making.

Definitely no room to have data in memory, since it would waste the space you need for more thoughtful code. You can't have the resulting factors building up in memory either so they had to go to external storage as they were generated. Naturally to be used later by completely different code which is geared to process the rudimentary findings in relation to new data, and present that to the operator in order to enhance their pattern recognition and decision-making efforts.

I could only imagine what it would be like if memory came in megabytes rather than merely the lowly kilobytes.

One thing I think might still be true today, whatever amount of memory you have, you should be able to handle so much raw data that it makes the amount of memory look insignificant.

OTOH, if you can't highly leverage a naturally intelligent operator without some huge resource requirements, you might not be on the right track when it comes to maximizing hardware utilization.

And then there's the concept of analog noise amplification. You really need to be careful that there is nothing wrong or unrelated in the fundamental data set you are using at the time. Starting with a raw input signal, each stage of amplification will increase the amount of noise proportionally, and depending on the number of layers of amplification, any noise can cascade into top prominence when the desired signal is unfortunately weak. But the same level of noise-in-place-of-signal does not go away when it is dwarfed by a strong signal, the noise is merely masked during the high-signal passages but remains a considerable component.

Now when the raw data only gets one initial pass, anything that's missed the first time is lost forever, and if the missing nuance is something important that would seem like the type of thing that when a strong need is there, anything related to that nuance would be unreliable, incorrect, false, or downright hallucinatory if the performance was advanced enough.

GIGO is still the name of the game so I would think if it's custom training you have to step up to the plate and take the good with the bad. You've got to laboriously handle all the training data yourself anyway, so might as well take the opportunity to seriously babysit that data thoroughly in advance like you would never be able to do if you were only dealing with somebody else's already trained model.

Plus you can't usually take an adequately huge data set and in one pass remove all the undesired artifacts. And if one pass results in a processed dataset which can no longer be considered very huge at all, you've probaly lost too much valuable information and you may still not have eliminated all the undesirable noise.

This is somewhat analogous to lossy digital data compression, but focused on retaining only the most prominent meaning that can be gleaned from the data. As opposed to plain compression which retains the most prominent data regardless of meaning.

The more intelligently the raw input data is handled, the more realistically intelligent you can expect the final outcome to turn out.

For example you can have a table of "knowledge" as I described earlier, but you can just add easily have a table of the conversation history, or have both.

In fact it's quite popular afaik to store the conversation this way because then if you query on a topic you've queried before, even if the conversation history has gone behind the size of the context, it can still retrieve that history. So yes, what you describe is a good idea/would work/is being done.

It really all comes down to the non model logic/regular programming of how your vector db is queried and how you mix those query results in with the user's query to the LLM.

For example you could embed their query as I described, then search the conversation history + general information storage in the vector db and mix the results. You can even feed it back into itself in a multi step process a la "agents" where your "thought process" takes the user query and breaks it down further by querying the LLM with a different prompt; instead of "you are a helpful assistant" it can be "you have x categories of information in the database, given query {query} specify what data to be extracted for further processing" obv that's a fake general idea prompt but I hope you understand.

Well there's technically no model training involved here but I guess you consider the corpus of conversation data a kind of training, and yeah that would be RLHF based which LLMs learn pretty heavily on afaik (I've not fine tuned my own yet).

You can fine tune models to be better at certain things or respond in certain ways, this is usually done via a kind of reinforcement learning (with human feedback...idk why it's called this, any human feedback is surely just supervised learning right?) this is useful for example, to take a model trained on all kinds of text from everywhere, then fine tune it on text from scifi novels, to make it particularly good at writing scifi fiction.

A fine tune I would say is more the "personality" of the underlying LLM. Saying this, you can ask an LLM to play a character, but the underlying "personality" of the LLM is still manufacturing said character.

Vector databases are more for knowledge store, as if your LLM personality had a table full off open books in front of them; world atlases, a notebook of the conversation you've been having, etc.

Eg, personality: LLM fine tune on all David Attenborough narration = personality like a biologist/natural historian

Knowledge base = chunks of text from scientific papers on chemistry + chunks of the current conversation

Which with some clever vector db queries/feeding back into model = bot that talks like Attenboroughish but knows about chemistry.

Tbf the feedback model it's better to use something strict, ie instruct based model, bc your internal thought steps are heavily goal orientated, all of the personality can be added with the final step using your fine tune.

It's retrieval augmented generation