Better Language Models and Their Implications

Started a Google colab with the interactive text generation script. https://colab.research.google.com/drive/1da54684tFMjPbR5idbv...

They include the enwik8 BPC estimates. It may not be strictly comparable - the most obvious issue is that since the HP takes the compression paradigm of intelligence, the compressor size is part of the total, and those 1.5b parameters certainly are not cheap.

(This is one reason I think the HP is outdated. The corpus is not big enough to allow the superior asymptotics of approaches like RNNs or Transformers to compensate for their far larger binary size. HP is not measuring progress on an intelligence metric we care about, it's sort of measuring a 'demo scene' metric of intelligence.)

Sorry about that, I should have not missed it. Since you're here, I'll ask a quick question to satisfy my curiosity. This is a pretty big step, even just looking at the benchmarks. Where do you see text generation going in 5 years? What about 10 years? As I said, no need to give long explanations, I just want to know what do you expect, as an insider,on an intuitive level.

Some of these are basically horror short stories:

======================================== SAMPLE 164 ========================================

I want to see a ton of households with kids with no food in their houses. Surviving famine requires storing up food in hopes a spoiled enclosure might provide the behaviors of food memories we crave. In near future, they will have to leave their cages.

We're spending all our time theorizing about animal minds, yet we have living, reacting animals. As if we can just no be in touch with our emotions at is practical.

My kale starts from an academic grown Himalayan. It hydrates in the garden, but it seems another species than we know is fermenting! My general lab is a recently renovated cave survival room with sun-filled bowls full of food for rats. The interior design is nearly utilitarian (no smart phones, blog, writing tools, or ovens there), except the cave ceiling. This is where we grow the food to feed animals. Which they will eventually eat.

My girl starts out with a kind of scaffolding green and wild, but soon she can't grow more than a few leaves. Who needs her monotub artistically scissors and wire? The boxy roots are brittle right away before growing into skin weeds. So that's not going to do it.

An unsteady start for another young carnivore's evening. I go in for some priming with baking soda and dry iron. A fun little experiment is once the kopi luwak seems she will grow some damp spikes. Consider this:chopsticks are soft under a microscope, webbies, and twining vines all apply more pressure from tangles of animal blood.

Taking over the cultural mythos of science, I peek over her purple skeleton spikes with an endoscope.

Once the higher organs take over, the kabuka unfurrows skin at a clever angle to reach past the body curse powered at this end of the lab. Curiously, without a hoe or a pot to turn into a tiled floor, she does not chew with her jaws.

I have undressed her to her base to fill the bowl. She is pushing her way to get her shelter to stretch out into, or if I've planned correctly, squish into. She does not move like a biological creature, pulling up with her haunches while excellent and facing directly downward. She prefers laying there moss-covered and mother-prey encouraged paying attention to her uphill path.

Lack of summer fur gives me ample room to poke through her skin, pull out black seeds, and taste the damp thing she'll roost under in future. Would the static reference very much reduce a virus being blocked within its first expenditure, nights? I'm not sure, but it's for sure a more effective inoculation. Hopefully, these experiments will be continued. Hopefully.

My guru and master who made us does not focus so much on running where we want as it does on empathy for the soil,, plants, and animals with which he touched in some perfect way. Delhi is often flooded my first morning in the shafts of windy sunshine. Let's not give success an easy choice.

The trees are in the underground and they change colors weekly. In simple terms, they're scattered flowers taking their cues from the transynthetic forest. While I'm pioneering, the one thing I sacrifice in the relationship is conducive to imagination. A full color-changing lets us enjoy cozy warmth while simultaneously losing whoever is formal and rational for the feel good activity. Like cosmic lap of the firmotsuites.

An American outcropping of stonemines blocks the sunlight. The unseen wave depose colonies to accommodate my habit; sandblasting. There are microbes from amoebas that change the composition of the water. In the habitat where the natural and mecopate are as thoroughly ever so mindful of the fence as the meesthe gardener has been, I'll shed this duality when I sit with her. Food grows from space; human needs feed them. Sandbringation turning our microbes and animals can save thousands in pandemic deaths. I value them all in metaphorical and real (if tough) tenacity.

Species survival deals good cop in these two every where storytelling machines.

Door is opened. My guru goes rescuing the kupi luwak elder from drought resistant drought. I alone have learned her humdrum bookish biogeography is yielding to light and fire. Cow has problems taking roots. Bring my love to her and make a mend. She needs a clean-air daughtersnail easy experience not in a month's time, but in just a few hours of thinking about reflexes, calendar dates, and natural spatial relations for a mind that has grown certain.

Super appreciate this!