Viewing profile — furiousteabag

same, did you figure it out?

Another thing we are trying to understand is whether the 2D element adds value to the simulation. A simpler option would be a pure text/chat interface. Still, the hypothesis here i…

Hey hey! We tried Clay v1 with 768 embeddings size using your tutorials. We then split NAIP SF to chips and indexed them. Afterwards, we performed image-to-image similarity search …

Curious to see what other things you will simulate in the future! Shameless plug: recently we've built a demo that allows you to search for objects in San Francisco using natural l…

Thanks for sharing Brooklyn text demo. Haven't seen it! Captioning images using VLM would definitely help as an additional conditional feature. Maybe it even would be enough to use…

Hey HN! We're Alex and Szymon from Bluesight ( https://bluesight.ai/ ), where we're developing a foundation model for satellite data. We've created a demo to showcase the current c…

Hey HN! We're Alex and Szymon from Bluesight ( https://bluesight.ai/ ), where we're developing a foundation model for satellite data. We've created a demo to showcase the current c…

I really like using pandoc as a build system [1] for my personal website to convert .md to .html. I can use templates, automatically generate a table of content and run some lua sc…

You are correct, training sorely in fp16/bf16 can lead to imprecise weight updates or even gradients turning to zero. Because of that, mixed precision is used. In mixed precision t…

Mixed precision is a default method to pretrain and full fine tune right now. It is especially good in transformers, because they have memory bottleneck in activations (outputs of …

There is no option to select quantized version yet. Will work on that!

By default, SGD w momentum is enabled as optimizer. You may try selecting Adam and it will list second moments as well.

Hey HN, I'm working with Llama 2 and have hit a snag regarding VRAM usage during the forward pass in inference. Despite understanding that only the largest activation tensor is sto…

It doesn't really matter if the app claims to use E2E when it actually discloses message content [0] [1]. WhatsApp is also filled with backdoors [2]. [0] https://therecord.media/fb…

This may sound stupid, but from my perspective renting random VMs on vast.ai is safe in general and might be safer than using traditional cloud providers in particular. Consider th…

In IM, there's a balance between total privacy and widespread use. Apps like Signal offer high privacy but have fewer users, while popular ones like WhatsApp are less secure. Teleg…

It's true that fine-tuning models on personal messages could be simplified, but many, like myself, can't use third-party services due to sensitive data in our messages. I'm curious…

I agree that usually 'more is more' for training LLMs. However, for fine-tuning with limited data, it seems crucial to focus the task as much as possible. Since the model still enc…

First I download the weights of the base pre-trained model to the VM instance. Then I upload my data there. Afterward, I fine-tune either LoRA or full and when training finishes, f…