Earlier quoted context omitted.
Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment. https://github.com/EleutherAI/gpt-neo/releases https://github.com/google-research/bert https://github.com/google-res…
I honestly think of the T5 model family to sort of be the real beginning of this open model craze - I know BERT was already popular for classification etc, but T5 was the first sort of generally useful model, was exceptionally simple to fine-tune, and is still in use today (t5 base is still averaging over a million downloads a month on huggingface), has tons of variants and sort of kickstarted this whole community. U…
Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
621–623 of 623 posts
Re: Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
#622Earlier quoted context omitted.
If the weight, training and inference code, and training data are all released under of Open Source (OSI definition) license, the it is unmistakably “open source”. As you drift from that it becomes less clearly so, and when you get to no training data, and the model weights license having extensive limitations on allowed uses, the use of even “open weights” becomes deceptive.
Is there any relevant model that meets that "open source" definition?
https://allenai.org/blog/olmo3
That 7B model also worked well in my experiments on a laptop.
Re: Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models
#623Earlier quoted context omitted.
TBH, having worked at all big-tech, startup and mid-sized companies, pretty much most has some major pros and cons of their own, and these days ever more so. With certain big-tech at-least there some chance of getting decent WLB.
I prefer hectic work and impossible deadlines at 3am over the flat-but-not-really hierarchy and the constant dick-measuring contests.
I don't know how many of big-tech have flat hierarchy, at-least not the ones I've worked at and the people/team culture have definitely better. Were there times there was bureaucracy and all, sure, but I personally would take take good WLB most of the year, over fucking up my health/relationships over making some random software on claude code all night that these days anyway most want to be AI slop. If I've the energy to work extra, I'd work on my own hobby software projects.
That said, this or that, its all personal preference. So no wrong way of life here. Whatever, you want your life to be.