lol... Python for Dummies (TM)
Code Llama, a state-of-the-art large language model for coding
371–380 of 525 posts
Re: Code Llama, a state-of-the-art large language model for coding
#372Earlier quoted context omitted.
Simple, concise, more efficient: def primes_upto(limit: int): """Generate prime numbers 2 3 5 7 11 13 17 19 23 29
Yeah, but yours was generated by the "post unoptimized code to HN and wait for someone to optimize it" model, which, although free and doesn't require a GPU, is a much slower model.
But, unless you are trying to find a prime number low enough that you might as well look it up in a pre-generated table, it might still be end-to-end more efficient?
Re: Code Llama, a state-of-the-art large language model for coding
#373Earlier quoted context omitted.
Yeah, but yours was generated by the "post unoptimized code to HN and wait for someone to optimize it" model, which, although free and doesn't require a GPU, is a much slower model.
But, unless you are trying to find a prime number low enough that you might as well look it up in a pre-generated table, it might still be end-to-end more efficient?
Ah, good point :) Touché!
Re: Code Llama, a state-of-the-art large language model for coding
#374Earlier quoted context omitted.
Having "extremely basic understanding" of prime numbers immediately at one's command is important for approximately 0% of software engineering jobs. If you instant-fail a candidate for this, it says a lot more about you and your organization than the candidate.
> If you instant-fail a candidate for this, it says a lot more about you and your organization than the candidate. yes, we expect professional software developers to have basic maths skills "what is a prime number" is taught to 7 year olds, it's not vector calculus what else would you consider to be an unreasonable thing for an employer to require? reading and writing skills of a typical 7 year old?
You probably do not have a child of 7 years old because they do not know at that age what is a prime number.
Second, basic math still that you never or rarely use or with very large time between usage might get rusty. You may understand the concept but not find the optimal solution. The way you are responding here shows quite a lot about how you are short sighted by instant-failing someone with a single question instead of trying to asses the whole person as much as you can. On you side, you are wasting opportunity to have a great person that could be a key player in your team by bringing other set of skill on the table.
Re: Code Llama, a state-of-the-art large language model for coding
#375Business opportunity: I'd pay money for NICE desktop software that can run all these different models (non-subscription, "2-year updates included, then discount pricing" modal perhaps). My wishlist: - Easy plug & play model installation, and trivial to change which model once installed. - Runs a local web server, so I can interact with it via any browser - Ability to feed a model a document or multiple documents and…
I have been using refact.ai on my laptop, it has been quite good. https://github.com/smallcloudai/refact/blob/main/README.md
I wish these things worked with anything other than VSCode or JerBrains tools!
VSCode is such a bloated hog of an editor!
Every time I open VSCode it’s bugging with badges to update extensions… and it’s so slow!
Re: Code Llama, a state-of-the-art large language model for coding
#376Earlier quoted context omitted.
Apple Silicon, especially an M1 Max Studio seems to be an interesting machine to hang on to as the models become more and more efficient with using less and less. If there's nay other opinions or thoughts on this, I'd be very happy to learn as well. I have considered the eGPU route connected to a 1L PC such as a thinkcentre m80/90.
I have a 64 GB M1 Max MBP, and I'd say unless you really have some academic interest towards messing with open models, for now accessing SOTA models via a REST API has better latency for a given quality. Claude 1.2 instant is as fast as 3.5, follows instructions at a quality closer to 4, and has a 100k context window. Hard to compete with that with an open source model right now.
How does open source compete with the Claude API? Easy: actually let you use the model. From the signup page:
> Anthropic is rolling out Claude slowly and incrementally, as we work to ensure the safety and scalability of it, in alignment with our company values.
> We're working with select partners to roll out Claude in their products. If you're interested in becoming one of those partners, we are accepting applications. Keep in mind that, due to the overwhelming interest we've received so far, we may take a while to reply.
No thanks, I'd much rather not wait months to see if my app deserves their oh-so-limited attention, or "aligns with the values" of a company taking $400m from Sam Bankman-Fried.
To be more charitable to your underlying point, Claude 2 is free to chat with via Anthropic's website, Poe, or Slack, and the GPT-4 API is open to use. If you're building a prototype or just need a chatbot, these do have better results and dev experience, at least for now. But I don't think picking on your Claude API example is unfair. These companies could randomly refuse your prompts via some opaque "moderation API" (that all GPT fine-tuning data goes through!), train on your company's proprietary data, spy on your most intimate questions, or just not find you worth the trouble and cut you off, at any time. THAT is why open source beats proprietary hands down: My device, my data, my weights, my own business.
Re: Code Llama, a state-of-the-art large language model for coding
#377Earlier quoted context omitted.
I work for a Fortune 100 company with 80,000+ employees. All of us are explicitly forbidden from using any sort of AI/LLM tool without written permission from the head of legal AND the CEO. In other words, nobody is going to get permission. The concerns are 2 fold - 1. We might inadvertently use someone else’s intellectual property. 2. Someone else might gain access to our intellectual property. What you are describi…
Change company. Honestly. If you go as far as to forbid your partners in crime (workers sigh..) to explore new uncharted territory at all - well ya know someone will/might just win by not doing that.
Working for an 80.000+ employee company, one has already accepted a certain degree of inertia.
Re: Code Llama, a state-of-the-art large language model for coding
#378Works nearly out of the box with llama.cpp, which makes it easy to try locally: https://github.com/ggerganov/llama.cpp/issues/2766 Here's some output from q4_0 quantization of CodeLlama-7b-Python (first four lines are the prompt): # prints the first ten prime numbers def print_primes(): i = 2 num_printed = 0 # end of prompt while num_printed It will be interesting to see how the larger models perform, especially afte…
Funny watching HN be nerd sniped by a machine :-)
Re: Code Llama, a state-of-the-art large language model for coding
#379Business opportunity: I'd pay money for NICE desktop software that can run all these different models (non-subscription, "2-year updates included, then discount pricing" modal perhaps). My wishlist: - Easy plug & play model installation, and trivial to change which model once installed. - Runs a local web server, so I can interact with it via any browser - Ability to feed a model a document or multiple documents and…
A few folks and I have been working on an open-source tool that does some of this (and hopefully more soon!) https://github.com/jmorganca/ollama There's a "PrivateGPT" example in there that is similar to your third point above: https://github.com/jmorganca/ollama/tree/main/examples/priva... Would love to know your thoughts
I'd love to test this out as soon as you get Linux or Windows support going!
Re: Code Llama, a state-of-the-art large language model for coding
#380Earlier quoted context omitted.
I work for a Fortune 100 company with 80,000+ employees. All of us are explicitly forbidden from using any sort of AI/LLM tool without written permission from the head of legal AND the CEO. In other words, nobody is going to get permission. The concerns are 2 fold - 1. We might inadvertently use someone else’s intellectual property. 2. Someone else might gain access to our intellectual property. What you are describi…
Change company. Honestly. If you go as far as to forbid your partners in crime (workers sigh..) to explore new uncharted territory at all - well ya know someone will/might just win by not doing that.
This is particular, specifically problematic territory. I cannot imagine handing over proprietary data to a third party without a contract in place for how that data is stored and used. It’s not about innovation, it’s about using someone else’s tools without ownership. For the other case, it’s both about integrity in owning your own work, and a shield from legal consequences. These things should be very relevant to any business.
I also don’t know any professional devs who have used tools like copilot and said they were anything but a toy. I am more bullish on LLMs than most of my coworkers. I think there is a lot of potential there. I do not see that potential in the current commercial offerings, and the financial outlay to fine-tune an open-source model and run it at scale is…prohibitive.