201–210 of 217 posts
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
Because of vast piles of bad science fiction being taken as fact and outright hysteria.
None of these systems could work if only facts were used as input. Maybe some "factual" score could be presented with the data, but wow, that's a philosophical problem there.
GPT-3 is the most overrated game in town. And Microsoft spending $1 Billion for an exclusive license will seem really foolish a few years from now.
This move was a defensive move. Keep any other MEGA-CORP from getting access.
Earlier quoted context omitted.
PHP wasn't discredited by the incumbents. It was discredited by its creator. "I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say Yeah it works but you're leaking memory everywhere. Perhaps we should fix that. I'll just restart Apache every 10 requests." -Rasmus Lerdorf "I was really, really bad at writing parsers. I still am really bad at writing parsers."…
To most programmers that doesn't discredit PHP at all. He cares about a working product, much like 90% of programmers, who don't have the privilige to worry about theory. They just need an ecommerce, or blog or whatever, running asap. To use a pg's analogy, they are there to paint not to worry about painting chemistry. The incumbents do discredit PHP though. For instance, facebook was built on PHP, and still runs on…
Terrible code in PHP is possible, therefore likely. I say this having spent over a decade writing it and half of that time fixing OWASP bugs created in it.
The incumbents hate it because their vendors use it and everyone is worse off for having their vendors use it. And Facebook did use PHP in the first few years, but they quickly started compiling it (HipHop) and later converted their code based to use a different strongly-typed language based on PHP (Hack). They stopped using PHP because it is a starter language.
Earlier quoted context omitted.
> these work best on predictable things Umm, not really. You are talking about single and multi-label classification tasks, maybe? Bengali poetry is just like poetry in any other rich languages like English, French, etc. What these models do, from a high level, is that they learn the distribution of the data. In this case, they learn the style of the poets. Writing poetry in specific styles has been done long before…
Ah, I see the distinction now, thanks. It's an interesting subject, I think, since poetry has so many forms it can take and you need the output to capture the idiosyncratic aesthetics and "inner world" of a piece of verse. I've actually tried using GPT-3 to generate poetry and the naive approach of just sending prompts and text snippets through the API had wildly varying results. Some pretty good, some that were basi…
There are a lot of pretrained language models (preferably at HuggingFace Hub). Take one, and fine-tune it on a much smaller dataset.
You can then pass prompts to let this model write poetry.
I would also suggest learning the fundamentals of Deep Learning, RNNs, LSTM, Transformer, etc. Because even if you know all these, this kind of taks is not trivial.
I would use HuggingFace + PyTorch for these kind of tasks.
Besides Deep Learning knowledge, you will also need to know your way around tokenizers, controlling parameters, evaluation metrics, etc.
Having some basic understanding of poetry is helpful, but to be able to truly apply your knowledge of running your poetry journal, you will really need to be on the proverbial edge. I hope you get there, and maybe get some papers out!
Although I worked on a language model because none existed in my language, my expertise (and employment) lies in Computer Vision. So, take what I said with a grain of salt.
The future is not as dark as it seems because of the rat race of megacorps. You can use reduced versions of language models with extremely good results. I was involved in training the first-ever GPT2 for Bengali language, but with 117 million parameters. It took a month's effort (training + writing code + setup) and about $6k in TPU cost, but Google Cloud covered it. Anyway, it is surprisingly good. We fine-tuned the…
I have supported my beliefs on this topic in these threads to the point of exhausting myself. The tools that we use to find these agents are the underpinning of AGI, it’s coming way faster than even most people here appreciate, this development is intrinsically against the interest of human beings. Please stop and think, please.
I argue it's very much in the interest of human beings. It has been since we first picked up a rock and used it has a hammer. It's the ultimate tool and has the potential to bring unseen prosperity.
I believe this will displace the workers of places like the Internet Research Agency and some Marketing specialists. While that accounts for a large percentage of propaganda jobs, that is easy enough to adapt from (nor do I really care given who is generally doing it), and marketing people who I generally don't have respect for either (I mean their founding document is literally titled "Propaganda"). The real marketing pros are good at managing hard to handle talent or bad PR disasters, which are actually meaningful skills that this won't affect.
So basically no moral or real consequential impact on society whatsoever.
The underlying methods seem impractical. GPT-n are an existence proof - it is possible to make parrot like software that generates realistic text. But using these methods it is not practical. Maybe that is a good thing, maybe a bad thing, but unless there is a breakthrough in methods this is a dead end. Impressive though.
https://copilot.github.com/
This will just lead people into the equivalent of using the autocorrect answers too often as if you used the suggest feature of a phone keyboard on a sentence. Eventually it just breaks down and stops making any sense. The code equivalent is going to be bugs, security holes, and programmer headaches.
Earlier quoted context omitted.
https://copilot.github.com/
Yuck. More intrusive than an IDE, I'll pass. This will just lead people into the equivalent of using the autocorrect answers too often as if you used the suggest feature of a phone keyboard on a sentence. Eventually it just breaks down and stops making any sense. The code equivalent is going to be bugs, security holes, and programmer headaches.
Earlier quoted context omitted.
Yuck. More intrusive than an IDE, I'll pass. This will just lead people into the equivalent of using the autocorrect answers too often as if you used the suggest feature of a phone keyboard on a sentence. Eventually it just breaks down and stops making any sense. The code equivalent is going to be bugs, security holes, and programmer headaches.
Or this will revolutionize programming and give a 3x-4x productivity boost to those who take the time to learn to use it effectively.