411–420 of 436 posts
Earlier quoted context omitted.
What's c++xx?
C++11, and greater.
Earlier quoted context omitted.
> Python is superior to all of those just for syntax alone, it is easier and cleaner to both read and write. Do you have any argument to support this, aside from personal bias?
I can make some arguments but it all boils down to personal bias and anecdotes. The forced use of spacing to delineate blocks means you will never see a bunch of brackets eating up screen space and the common error where someone adds another line to an if statement but doesn't add braces. Semicolons not being conventional means less screen noise and less code golf 1 liners. The focus on imperative vs functional means…
Thanks!
Earlier quoted context omitted.
> I'm arguing against the point that it clearly did have the easiest syntax compared to the competition back then and not because Google was using it. Maybe, not sure? My point was that both the syntax and Google using it was more relevant 15 years ago than now. (I don't have much of an opinion on the 15+ years ago thing.)
I don't see any reason for it to be less true now. Is python syntax worse than any brand new languages like rust or go? Absolutely not. It's still better. Did Google stop using it? I don't think so, but I also don't think people picked it just because Google did.
Btw, I wish they would take some inspiration from Haskell's syntax.
Haskell also has significant whitespace, but its defined as syntactic sugar for a more traditionally syntax with curly braces and semicolons.
Approximately no-one uses that curly-brace syntax, but it's good for two things:
- silences the naysayers
- more importantly: allows you to copy-paste code even into forms that mess up your indentation.
Earlier quoted context omitted.
I host a llama-13B IRC chatbot on a spare old android phone.
Have a repo anywhere?
Assuming you have the model file downloaded (you can use wget to download it) these are the instructions to install and run:
pkg install git
pkg install cmake
pkg install build-essential
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j
./main
Earlier quoted context omitted.
Have a repo anywhere?
It's just the same llama.cpp repo everyone else is using. You just git clone it to your android phone in termux and then run make and you're done. https://github.com/ggerganov/llama.cpp Assuming you have the model file downloaded (you can use wget to download it) these are the instructions to install and run: pkg install git pkg install cmake pkg install build-essential git clone https://github.com/ggerganov/llama.cp…
Earlier quoted context omitted.
with nvme gen 4 ssds this might not be that huge of an issue, and for sure much cheaper than investing in ram
I don't believe the consumer ones actually have sustained sequential read speed to saturate Gen 4.
Earlier quoted context omitted.
It’s not that the performance is the issue, it’s that it’s unmaintainable and prone to break. Exceptions aren’t handled right, dependencies are a disaster (Proprietary NVIDIA drivers+CUDA+PyTorch+ the various versions of stuff are a complete disaster) This leads to all sorts of bugs and breaking changes that are cool in an academic or hobbyist setting but a total headache on a large production system.
Yeah, I've been using python for the first time in a while to try out some of the llm stuff and I can't believe how bad the dependency hell is. It's probably particularly bad due to the pace of change in this field. But I spend an hour getting dependencies fixed every time I touch anything. 80% of the Google Collabs I find are just outright broken. I wish there were other viable non python options to try out these th…
I've been very _careful_ too (using pyenv/virtualenvs etc) for dependency management, but with Nvidia driver dependencies and "missing 'sqlite3/bz2' issues related to the underlying interpreter (not to mention issues with different Python3.x versions) I'm lucky to be able to even run a 'hello world' ML sample after an afternoon of fighting with it.
My Ubuntu install w/ Nvidia card only seems to recognize the GPU in some circumstances even when using the same `conda` env. Often this is remedied by rebooting the machine(?).
No idea how companies manage this stuff in production. Absolute minefield that seems to catastrophically break if you sneeze at it.
I'll admit I am not an expert in managing ML envs, but I've dealt with a lot of python environments for typical CRUD stuff, and while rough at times, it was never this bad.
Earlier quoted context omitted.
Didn't expect to see two titans today: ggerganov AND jart. Can ya'll slow down you make us mortals look bad :') Seeing such clever use of mmap makes me dread to imagine how much Python spaghetti probably tanks OpenAI's and other "big ML" shops' infra when they should've trusted in zero copy solutions. Perhaps SWE is dead after all, but LLMs didn't kill it...
Sigh. It's not like the zero copy buzzword is going to help you during training, all your weights have to stay on GPU, you are going to sample your training data randomly and your data is on a networked storage anyway, so mmap HURTS. You'd better just O_DIRECT. Similarly, as long as you run your inference on GPU it's not like you can mmap... And I have indeed worked on inference runtimes for mobile devices and on the…
It runs on my 2015 ThinkPad!
Earlier quoted context omitted.
Funny enough, this tracks the early history of Google as well. It was originally written by Larry Page and Sergey Brin (both grad students at the time) in Python, then Sanjay Ghemawat rewrote the whole thing in C++.
That sounds very interesting. Anyone know where I can find more on this story?
Earlier quoted context omitted.
C++11, and greater.
Huh, I was proficient in Rust before "properly" learning C++, so maybe that accounts for it, but I didn't realize C++11 was controversial. Is it just move semantics, or are there some library things that are hard to implement?