331–340 of 436 posts
Is there a reason Llama is getting so much attention compared to say T5 11B? Not sure how neutral or what benchmarks are used on the following link, but T5 seems to sit a lot higher on this leaderboard? https://accubits.com/large-language-models-leaderboard/
Earlier quoted context omitted.
So let's toss management and go write good code for the principle of it, and not business bullshit calculus
Good, tell me how your company will be doing. What people sometimes fail to understand is that code is a mean to an end, not an end in itself. If you want to make code for itself, work on an opensource and/or personal project. If you are paid to work on something, you're paid for the something to get out, not for it to feature the best code ever.
Or hiring useless business people to install around the periphery of engineering. Which is funny because now tech is letting all those folks go.
Is there a reason Llama is getting so much attention compared to say T5 11B? Not sure how neutral or what benchmarks are used on the following link, but T5 seems to sit a lot higher on this leaderboard? https://accubits.com/large-language-models-leaderboard/
The pace of collaborative OSS development on these projects is amazing, but the rate of optimisations being achieved is almost unbelievable. What has everyone been doing wrong all these years cough sorry, I mean to say weeks? Ok I answered my own question.
>What has everyone been doing wrong all these years So it's important to note that all of these improvements are the kinds of things that are cheap to run on a pretrained model. And all of the developments involving large language models recently have been the product of hundreds of thousands of dollars in rented compute time. Once you start putting six digits on a pile of model weights, that becomes a capital cost t…
Earlier quoted context omitted.
It’s not that the performance is the issue, it’s that it’s unmaintainable and prone to break. Exceptions aren’t handled right, dependencies are a disaster (Proprietary NVIDIA drivers+CUDA+PyTorch+ the various versions of stuff are a complete disaster) This leads to all sorts of bugs and breaking changes that are cool in an academic or hobbyist setting but a total headache on a large production system.
The stack is very volatile and unmaintainable because it doesn't need to be maintainable. Exactly why we have unmaintainable software in other domains. During the last 10 years there are ALWAYS totally new model architecture with new operations (or in case of CV new bizarre uses of Conv). By the time you get your performant perfectly maintainable masterpiece ready it's not needed anymore. The stack optimizes for flex…
Earlier quoted context omitted.
What is another language working that well in a larger number of areas?
Clojure JavaScript Typescript OCaml Haskell F#
Ocaml is very niche, I feel it’s an hard sell for a general purpose language. Haskell, 3x that.
JS and TS, could be. But are they so much better than Python, if better at all?
Earlier quoted context omitted.
Its rarely as easy as sharing a requirements.txt. There are lots of things that can still break - for examples you get weird situations where different modules require different versions of a third module. Or all the Cuda toolkit version issues thsy seem to come up with gpu stuff. When we share python, we tend to share a docker image, and even this isn't foolproof. A big problem I think is that it doesn't incentivize…
Maybe using Nix it's a better experience for creating such an environment where you depending also on system utilities.
If you just write straight c++ (without c++xx, or anything like it) you can compile the code on machines from decades ago if you want.
Earlier quoted context omitted.
>What has everyone been doing wrong all these years So it's important to note that all of these improvements are the kinds of things that are cheap to run on a pretrained model. And all of the developments involving large language models recently have been the product of hundreds of thousands of dollars in rented compute time. Once you start putting six digits on a pile of model weights, that becomes a capital cost t…
Can the foss community really find nobody with the motivation to use their Bitcoin rig as a research machine? Or do you need even more specialized hardware than that?
Is there a reason Llama is getting so much attention compared to say T5 11B? Not sure how neutral or what benchmarks are used on the following link, but T5 seems to sit a lot higher on this leaderboard? https://accubits.com/large-language-models-leaderboard/
llama can run on an m1. T5 still needs a specialized gpu
In the end they are mathematical models, so what would prevent someone from loading T5 into a machine with plenty of RAM (like a server)? Would the codebase truly require that much refactoring? How difficult would it be to rewrite the model arhitecture as a set of mathematical equations (Einstein summation) and reimplement inference for CPU?