Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

github.com

Post reply on HN
Print view View original

11–20 of 58 posts

Re: Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

#11

Post by AMICABoard »

ingenieroariel wrote: post #7
throwaway10965 wrote: post #6

Earlier quoted context omitted.

It's the name of the project - Llama2Everywhere.

thanks! I am still curious about the sensor inputs part. I am trying to replace the 2.4ghz controller on my electric skateboard to make 0 to 5kmh and braking more pleasant and maybe use gyroscopes to do away with the controller altogether. What would tokens be in that case? Do you create a CAN style representation and feed that to the llm? What kind of throughput do you foresee being possible on which hardware?

I am still thinking how I'll pull this off. But basically it is collect tons of Telemetry converted to in ascii/text.

Telemetry in sense sensor streams, both command and responses.

Then we'll just train a small model for long enough. Then we will see how it would respond.

That's the plan sort of.

Re: Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

#13

Post by AMICABoard »

xcdzvyn wrote: post #3
AMICABoard wrote: post #2

Creator here. Thank you for sharing!

What does cat /dev/llama do? Would I get a kind of LLM stream of consciousness? That's incredible :-)

cat /dev/llama is not yet implemented. However we have a module to which you can give a prompt as parameter. It's buggy now.

Our goal is to write a proper kernel module to implement three things:

1. a character device 2. 1st backend is a LLMZip ie you write to say /dev/l2ezip, you get a compressed stream out 3. 2nd backend is a LLM, ie you write a prompt to say /dev/llama2, you get a completion back

So the 1st backend could be useful for compressed telemetry The second backend could be useful for IoT LLM, or our ambitious plan of responding to telemetry, ie take action, such as control motor speed etc.

Re: Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

#16

Post by AMICABoard »

explosion-s wrote: post #12

Are there pre-compiled binaries or do we have to build them ourselves?

In the releases you find the ISO which is pre compiles.

If you want to build it yourself, you can do make l2e_os_iso if you clone the repo and you are on a linux machine.

Re: Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

#18

Post by AMICABoard »

skybrian wrote: post #17

This could be a lot clearer about which install methods are offered. It seems to be an OS that you have to boot into a VM? What are the system requirements for the container?

You are right. Clearer instructions will be added.

This v0.1 and consider it alpha. A lot of stuff is broken.

The current build just needs 512MB RAM and an x86_64 CPU.

The ISO can be downloaded from the releases here:https://github.com/trholding/llama2.c/releases/tag/L2E_OS_v0...

Currently it doesn't do much useful stuff except for stories.

You could etch the ISO to pendrive and boot on real system or you could run it in qemu/VM too:

qemu-system-x86_64 -display gtk,zoom-to-fit=off -m 512 -accel kvm -vga virtio -cdrom l2eos.iso

There is a known issue that for some it does not work on Virtual Box. All that will be fixed in future versions.

There are also easter eggs and a hidden game of DOOM (Freedom)

Pics: https://twitter.com/VulcanIgnis/status/1708851772435968017

https://www.reddit.com/r/LocalLLaMA/comments/16zklam/i_creat...

User posted Video of Doom Play: https://www.reddit.com/user/multiverse_fan/comments/170orkz/...

Re: Llama 2 Everywhere (L2E): Standalone, Binary Portable, Bootable Llama 2

#20

Post by A4ET8a8uTh0 »

AMICABoard wrote: post #18
skybrian wrote: post #17

This could be a lot clearer about which install methods are offered. It seems to be an OS that you have to boot into a VM? What are the system requirements for the container?

You are right. Clearer instructions will be added. This v0.1 and consider it alpha. A lot of stuff is broken. The current build just needs 512MB RAM and an x86_64 CPU. The ISO can be downloaded from the releases here: https://github.com/trholding/llama2.c/releases/tag/L2E_OS_v0... Currently it doesn't do much useful stuff except for stories. You could etch the ISO to pendrive and boot on real system or you could run…

It is a great start for alpha. While I agree with poster above you, I absolutely do not want you to get discouraged. This is awesome and can only get better from here.
Post reply on HN