This is so awesome they built this, I've wondered what an LLM only trained on pre-1950s data would look like!
Now we just need a voice model with the "transatlantic accent" -- ideally with the early 20th century radio effect
Talkie: a 13B vintage language model from 1930
251–260 of 350 posts
Re: Talkie: a 13B vintage language model from 1930
#252The live-feed is super cool. I wish I had a machine large enough to run this locally. It'd be nice if the ollama pro tier would allow running these historical models to play with at some point.
I/(claude?) created a gist that can be loaded into tampermonkey to expand the live-feed to make it easier to view on a bigger screen. [0] It's been fun to watch.
[0] https://gist.github.com/acmiyaguchi/ed72be6ff56130481f820169...
Re: Talkie: a 13B vintage language model from 1930
#253USER: What will the future hold for computers? TALKIE-1930: Computers in the future may be employed in offices where calculations are required to be made, and where the nature of the business does not demand a very high degree of knowledge. The occupation will be a steady one, and the remuneration will be sufficient to afford a comfortable livelihood. The hours of attendance will probably be from ten to six. USER: an…
Also, I'd like to see the inverse of it: a model trained only on the last few weeks - or even minutes - of published information, or only on scientific papers published in the last year or two. Should present some interesting psychoses!
as per anthropic, only about 250 documents of any given concept required.
Re: Talkie: a 13B vintage language model from 1930
#254If anyone was wondering ... it's racist Unsurprisingly the texts written up until that time were dominated by such individuals which is tragic for LLM training if you think about it. The voiceless groups or fringe opinions which we take as normative today do not appear. Does this encourage us to write in the present such that we influence the models in perpetuity?
10 years ago people might had cared about your whining, not anymore (thank god)
Re: Talkie: a 13B vintage language model from 1930
#255Related: https://github.com/haykgrigo3/TimeCapsuleLLM > A language model trained from scratch exclusively on data from certain places and time periods to reduce modern bias and emulate the voice, vocabulary, and worldview of the era. Discussed here: https://news.ycombinator.com/item?id=46590280
TFA mentions some other interesting endeavors in this space:
>We have been excited to see a proliferation of vintage LM projects, including Ranke-4B, Mr. Chatterbox, and Machina Mirabilis.
Re: Talkie: a 13B vintage language model from 1930
#256Earlier quoted context omitted.
The American cut of the movie has an intro narrated by Joseph Cotton, who played Holly Martins. The wording might differ (since the movie is clearly Holly's first time in Vienna)
Yeah, I'm talking about the version (which is even on my US DVD) where the narrator is some black marketeer neither Martins nor Lime. "I never knew the old Vienna before the war with its Strauss music, its glamour and easy charm. Constantinople suited me better. I really got to know it in the classic period of the Black Market. We'd run anything if people wanted it enough - mmm - had the money to pay. Of course, a si…
For some reason, I always assumed the British narration was Major Calloway (Trevor Howard), but now that you quote it, I realize it can't be Calloway. Interesting! The narrator must be someone we never meet.
Re: Talkie: a 13B vintage language model from 1930
#257Earlier quoted context omitted.
Your bar is too low. With the coffee cup, you at least have access to all the pieces - in theory, although not in engineering practice. With Aristotle, you don't have anything close to that. Recreating Aristotle in any meaningful way, other than a model trained on his surviving writing of a million or so words, is simply not possible even in principle.
That's easy! All you have to do is simulate the whole universe on a computer, and then go the point when Aristotle is lecturing. Record all his works, then ctrl-c out of that and then feed those recordings into the LLM's training data. For the coffee, you just rewind the simulation and ctrl-c and ctrl-v it at the point you want.
> simulate the whole universe on a computer
Of course in principle that computer only has to be 1.x times larger than the universe, where x > 0. Perhaps AWS can sell you the compute.
Re: Talkie: a 13B vintage language model from 1930
#258Earlier quoted context omitted.
Your bar is too low. With the coffee cup, you at least have access to all the pieces - in theory, although not in engineering practice. With Aristotle, you don't have anything close to that. Recreating Aristotle in any meaningful way, other than a model trained on his surviving writing of a million or so words, is simply not possible even in principle.
OK I'll raise the bar--make sure when you reassemble the coffee cup and put the coffee back into it, the coffee is the exact same temperature as when you threw the whole shooting match onto the floor ;) EDIT: and you don't get to re-heat it. EDIT AGAIN: to be clear, in my post above (and this one) by "put the coffee back in" I meant more precisely "put every molecule of coffee that splashed/sloshed/flowed/whatever ou…
Ah ok sorry, so you want them to fully reverse entropy. I agree that bar is high enough.
Re: Talkie: a 13B vintage language model from 1930
#259It seems to be using more info from pre-1900 rather than 1930. It doesn't know about the Great Depression (1929-WWII). It knows about WWI if you ask it specifically, but talks about European politics as if it's 1900 or so. On technology, it knows who Edison is, at roughly the Wikipedia level, but credits him with a 125MPH car. About a dial telephone, it is confident and totally confused. It has the traction voltage f…
> You will pollute your brain. Such an interesting perspective, never crossed my mind that a brain could be polluted! My direction always been to fill it with as wide array of information as possible, the more different from existing information the better. What are some other things that you think "pollutes your brain"?
> What are some other things that you think "pollutes your brain"?
Horror movies. They are going straight to the amygdala but there are no vampires or zombies being afraid of. The nightmare fuel of my childhood prepared me wrong.
Re: Talkie: a 13B vintage language model from 1930
#260User: predict the 10 most populous cities in north america in 100 years.
talkie-1930 :The ten most populous cities in North America in 100 years hence will probably be--New York, Philadelphia, Brooklyn, Baltimore, Boston, Cincinnati, Chicago, New Orleans, St. Louis, and San Francisco.