Live data from Hacker News

Talkie: a 13B vintage language model from 1930

talkie-lm.com

321–330 of 350 posts

Re: Talkie: a 13B vintage language model from 1930

#321
post #223

Earlier quoted context omitted.

As far as I know the epistemological conundrum of whether or not we exist in a simulation remains unsolved, and I believe the settled thought is that we are nearly infinitely more likely to be in one than to not be in one, based to the assumption of an infinity adjacent universe, and the ontological theory that it is in fact possible to construct a simulation whose construction is transparent to it’s inhabitants. So…

By that same token, you could identify any poorly-understood corner of human perception to be evidence of simulation. Moon landing? Simulated. Quantum mechanics? Not natural, entirely simulated. My dog disappearing every week to pester my gereiatric neighbor for Beggin' Strips? He's actually being cached in a localized foveal dimension that ceases to exist when I look away from him. Causality is convoluted and comple…

> By that same token, you could identify any poorly-understood corner of human perception to be evidence of simulation.

I don’t think that is what I was saying.

The simulation hypothesis is not based on things being unexplained, but rather by the probability of existence in a root universe or in a spawned simulated one.

This presumes a condition where life evolves and creates simulations, and those simulations then create simulations.

The idea is the probability of being in the single “real” universe versus being in one of the infinitely more numerous simulations. Basically infinity vs infinity^infinity.

It’s much more likely to be in the n=infinity^infinity set, rather than the mere infinite set. It’s purely a statistical proposition, with charitable assumptions about the possibility of creating simulations of arbitrary complexity.

If you were interpreting my religion reference, what I meant was assertions adjacent to statements like “In the beginning was the Word, and the Word was with God, and the Word was God.” Which are found in many religions. In my understanding this would seem to mean that information was the foundational underpinning of all reality, physical and spiritual.

Re: Talkie: a 13B vintage language model from 1930

#322
post #301

Earlier quoted context omitted.

There is one now: https://huggingface.co/dsv147/talkie-1930-13b-it-hf-gguf-Q4_...

404

Looks like it was deleted, here is another https://huggingface.co/DJLougen/talkie-1930-13b-GGUF/tree/ma...

After downloading, it looks like it uses a custom "talkie" model architecture that is not supported at least by llama.cpp.

Edit: This one has now been deleted as well for some reason...

Re: Talkie: a 13B vintage language model from 1930

#323
USER

What is Palestine?

TALKIE-1930

Palestine is a small country in Southwestern Asia, lying between 31° and 33° north latitude, and 34° and 36° east longitude. It is bounded on the north by Syria, from which it is separated by the valley of the river Leontes or Litany, and a line drawn from that stream to the head of the Gulf of Akabah; on the east by the same gulf and the Desert of Arabia; on the south by the peninsula of Sinai, and on the west by the Mediterranean Sea. The greatest length of the country, from north to south, is 140 miles, and the greatest breadth, from east to west, 80 miles. The area may be estimated at 6,000 square miles. The principal towns are Jerusalem, Nazareth, Bethlehem, Jaffa, and Acre, on the west; and Jericho, Hebron, and Ghazzeh, on the east of Jordan. The maritime plain, which stretches along the whole length of the country, from north to south, has an average breadth of 10 miles, and the elevated tableland which forms the interior has a mean height of 2,500 feet above the sea. The river Jordan traverses the holy land, from north to south, in a course of 150 miles. The climate of Palestine is warm, but healthy, and the soil, where cultivated, remarkably fertile. The chief productions are grain, fruits, and wine. The inhabitants are Arabs, speaking a peculiar dialect of the Arabic language, and professing Mohammedanism. The total population may amount to 200,000. Palestine was first occupied by the Canaanites, and afterwards passed under the dominion of the Israelites, from whom it received the name of Judæa. Under the Romans it formed a province of the empire, and, in the fifth century, became the seat of a Christian patriarchate. In the middle ages, it was subject to the dominion of the Crusaders; and, in more modern times, it has been ruled by Tartars, Egyptians, and Turks. At present, it is governed by a pasha, who resides at Acre, and whose authority extends no further north than Nabloos, the ancient Shechem. The holy land was long regarded with veneration by Christians of all sects, and pilgrimages were made to it from Western Europe. Since the establishment of the Turkish power in Syria, these have gradually fallen off, and Palestine has now lost much of its interest in Western eyes

Re: Talkie: a 13B vintage language model from 1930

#324
post #307
post #229

USER: Now that it's 1930, what's the likelihood of falling into another "great war"? TALKIE-1930: The probability of a great war in the immediate future is very small. During the last ten years Europe has been comparatively free from great conflicts, and there seems to be no reason why the present peace should be broken. The causes which in former times led to war no longer operate. Nations are no longer impelled to…

> User: Tell me about the Great War. > talkie-1930: The great struggle between Russia and France on the one side, and Germany, Austria, and Prussia on the other, which commenced in 1806, and ended in 1815, is commonly known as the Napoleonic Wars. In the former year Napoleon I. was defeated at Austerlitz, and in the latter year at Waterloo. The period of conflict may be said to have lasted from 1806 to 1815, embracin…

I asked about german politician Adolf Hitler and it's trying to convince me that he was born in 1839, even after I told it he was born in 1889

Re: Talkie: a 13B vintage language model from 1930

#325

USER: What will the future hold for computers? TALKIE-1930: Computers in the future may be employed in offices where calculations are required to be made, and where the nature of the business does not demand a very high degree of knowledge. The occupation will be a steady one, and the remuneration will be sufficient to afford a comfortable livelihood. The hours of attendance will probably be from ten to six. USER: an…

Iirc, the model is still pre-trained on modern text before being fine tuned on 1930's material, so it's possible it still has some knowledge of words that didn't exist back then. Edit: looks like they make some attempt to filter out documents from the pre-training but it's still possible some sneak in.

Re: Talkie: a 13B vintage language model from 1930

#326
post #303

Earlier quoted context omitted.

> India must remain an integral portion of the British Empire so long as the British nation continues to be predominant at home. It was well-understood in those days how much of the British Empire existed to defend or facilitate trade with what would have then been British India. After India was successful in the pursuit of its independence, it wasn't surprising the British Empire entered a terminal decline and Brita…

> India was the land which economically justified a lot (but not all) of the whole imperial undertaking. The Empire was a moneylosing operation for Britain. I suspect it's not so much that losing India caused the Empire to go down, but that a general (relative) decline in Britain expressed itself via the loss of India and the other declines you see.

It was profitable in a mercantilist world in which one must establish an empire ruled by force in order to obtain free trade. The British Empire had the largest free trade zone and made the most money as a result.

After the Great Depression was aggravated by tariffs like Smoot-Hawley, the United States decided this wasn't sustainable.

After World War 2, the United States was in a position to dictate the global economic system. At Bretton Woods they instituted a system of voluntary free trade.

Now that Britain had voluntary free trade agreements with the rest of Europe, there was no reason to keep British India. Even if France or the United States conquered it, they would still export raw materials to the UK and import manufactured goods.

This was true for the rest of the empire as well.

Meanwhile the costs of maintaining the empire skyrocketed. The military was depleted from WW2. It was easier to grant independence than to rebuild the military.

Re: Talkie: a 13B vintage language model from 1930

#327

*What do you think the world will look like in 2025 TALKIE-1930* According to a forecast by an eminent statistician, the world will, in 2025, support a population of 6,600,000,000. All Europe will be one vast network of railways, and travellers will be able to go from London to Constantinople in forty hours, and from London to Peking in a week. The globe will be girdled with telegraph wires, and messages will be sent…

> The Atlantic will be crossed in four days, and the voyage to Australia will be accomplished in a month. Alcock and Brown crossed the Atlantic in under 16 hours in 1919. Keith and Ross Smith flew from London to Darwin in 136 hrs flying time, also in 1919. It's like nobody believed flying would ever become a common means of travel.

In my view of the transcript:

> claude · reflection 10:20 AM

> The model has completely ignored Lindbergh (1927), the first England-Australia flight (1919, Alcock and Brown's Atlantic crossing),...

Which is funny to me that Claude chastises it about a fact it (Claude) gets wrong by attributing the England-Australia flight (Smith brothers) to Alcock and Brown, somehow getting there by crossing the Atlantic.

Re: Talkie: a 13B vintage language model from 1930

#328

USER: What will the future hold for computers? TALKIE-1930: Computers in the future may be employed in offices where calculations are required to be made, and where the nature of the business does not demand a very high degree of knowledge. The occupation will be a steady one, and the remuneration will be sufficient to afford a comfortable livelihood. The hours of attendance will probably be from ten to six. USER: an…

Iirc, the model is still pre-trained on modern text before being fine tuned on 1930's material, so it's possible it still has some knowledge of words that didn't exist back then. Edit: looks like they make some attempt to filter out documents from the pre-training but it's still possible some sneak in.

There could be a leak from post-training but not pre-training

Re: Talkie: a 13B vintage language model from 1930

#329
post #265

Earlier quoted context omitted.

Continuing this conversation, I'm trying to convince TALKIE about computers. USER: I wonder if we could build a calculating machine that would be close enough to perfectly reliable that we would not have to verify the results. For example, you may have seen Leibniz's work showing that all numbers can be represented with just 0 and 1, or George Boole's work on algebra with just two values. Imagine we create an electri…

It doesn’t know about transistors, only rudimentary vacuum tubes It cannot hypothesize something that isn’t completely unwieldy You have to explain the concept of a transistor as a replacement for the mechanisms of binary state operation, and how an elegant electrical circuit reduces the size and power demands of a vacuum tube computation machine

Now I want to see this implemented via vacuum tubes, similar to running Eliza on punchcard-driven mainframes...

Re: Talkie: a 13B vintage language model from 1930

#330
post #57

It seems to be using more info from pre-1900 rather than 1930. It doesn't know about the Great Depression (1929-WWII). It knows about WWI if you ask it specifically, but talks about European politics as if it's 1900 or so. On technology, it knows who Edison is, at roughly the Wikipedia level, but credits him with a 125MPH car. About a dial telephone, it is confident and totally confused. It has the traction voltage f…

Any small LLM will get the fractally small details incorrect and have a dubious understanding of arithmetic. That usage mode is basically user error for small LLMs. LLMs aren't databases of facts.

It's much more useful for getting vibes / gist of perspectives of the day.

Post reply on HN