Viewing profile — Dorialexander

hi, so Pleias co-founder here. Common Corpus is commonly used now in pretraining, including by close labs, but rarely as the only source (which would be the actual ethical commitme…

Actually there is even a straight connection: Step-Fun DeepResearch trained on SYNTH (the open Baguettotron dataset).

Hi. Yes this is wholly correct. On the second points: * Well I'm very much involved in making open more models, pretrained the first model on free and open data without copyrigh is…

Seeing LLM as a motor was a legitimate view until recently. But what we're start seeing with actual agentification is models taking the driver seat, making the call about search, t…

So to clarify: the important product that people will ultimately want is the model. Obviously you need to design an infra/UI around it but that's not the core product. The really i…

Hi. So quickly: * RL is Reinforcement Learning. Already used for a while as part of RLHF but now we have started to find a very nice combo of reasoning+RL on verifiable tasks. Core…

Hi, author here. An important background is the imminent rise of actual LLM agents I discuss in the next post: https://vintagedata.org/blog/posts/designing-llm-agents So answering …

Ah it's completely volontary on my part: I want to keep the historical spelling as much a possible. That's why I used the google books OCR which does a better work at it than Galli…

Either that or appending archaic expressions in the prompts (a bit like the prompt extension of Midjourney)

Well you're not going to believe it but I do have a FoucaultGPT just being trained (indirectly: Foucault is just part of my extended French historical corpus). As a sample: Prompt …

I published the completely dataset here: https://huggingface.co/datasets/Pclanglais/MonadGPT While I don't think Saint-Simon is included, a French colleague did a few try with it t…

In a way you could do so by prompting Monad with artificial intelligence stuff. I did a try lately on the latest OpenAI events and it went on like this: "In this sad and tragical s…

Yes I needed that for the conversational/instructional capacities. I've made a lot of tests with base models and it would not listen to instruction very well…

Yes you're perfectly right. I've currently tried to maintain some kind of uneasy balance between good conversational capacities (so that it really is a "chatGPT") and cultural rese…

Not feasible to go with pretraining only. What is possible is to use a larger learning rate but this will be a hard trade-off with conversational capacities. Fine tuning is current…

Yes. I think we may have enough for "full finetuning" and erasing to a large extent the previous knowledge. But that's still very far off for pretraining. "RomeGPT" is next on my l…

Yes it happens once in a while. It's still a small model (7B) and I've done very weird things with it. If I were historically reconditioned in the 17th century mindset, I would als…

Already in the work. Just had a meeting today with two latinists about it.

Certainly. In fact I see we already follow each other on Twitter :D And yes totally. The other massive impact could be in source analysis. I have started using Mistral-Hermes for t…

The model can also be tried directly on this space: https://huggingface.co/spaces/Pclanglais/MonadGPT HuggingFace has generously provided free GPUs.

Hi! Model creator here. I happen to be a cultural historian and that's a main use case that I see. It's not complicated to learn about past events but having a general idea of the …

Hi. TheBloke has quantized the model: https://huggingface.co/TheBloke/MonadGPT-GGUF You may be able to run the Q3 or Q4 variant. Although in my experience, the quality of quantizat…