311–320 of 341 posts
Earlier quoted context omitted.
i wish more people would publish the things that didn't work. i'd like everything, but the exploration of searched negative space is so wasteful.
I think you would be interested in the Journal of Trial and Error: https://journal.trialanderror.org/
Earlier quoted context omitted.
i wish more people would publish the things that didn't work. i'd like everything, but the exploration of searched negative space is so wasteful.
Everyone says this in the abstract but to concrete examples they shug and say, of course that approach doesn't work, they did X, Y, Z wrong, they should have given it more effort, it could have worked if done properly / this can obviously never work, everyone knew already, it's nothing new etc.
In general, your point stands for armchair researcher.
Earlier quoted context omitted.
There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do. Cool for Google. Problematic if you are a startup.
Google has been publishing frontier research since it was an early-stage startup ( https://research.google/pubs/the-anatomy-of-a-large-scale-hy... ).
In later years Google generally published the details of a system after they moved to the next one, to stay ahead of their competition, or if they didn't see it as important to their business (most infamously the Transformer language model). It was still generous of them to share their research at all though, and this allowed their scientists to talk publicly about their research which is a huge plus.
And it makes sense. If you publish some state-of-the-art algorithm then you're basically helping your competition, and you don't get anything back from them. So you publish bare minimum in small articles on your website, only to gain people's/investors' interest, and just hope that you didn't share too much. And it'll always be like that, unless the root cause changes - intentives. And the main incentives right now a…
What other incentives can you imagine for Homo Sapiens?
I can’t speak for other startups, but I applied to the most recent YC batch with my idea for making AI proactive instead of reactive, and pre-being selected I’ve published a paper on recursive self-improvement mapped to the Epoch AI data. I contacted a professor from a university in the UK and he responded since he was working on similar work, then asked me if I wanted to meet with him. We talked for about an hour si…
Recursive self improvement is the beginning of the end.
We could set an LLM loose on itself now, self-improving its own code, but it's going to only make small improvements, and those will quickly peter out. If you think of it in terms of calculus, the sum of the improvements converges to a finite value, or at least the rate of improvements drop with time.
And even if we build a much better AI capable of performing substantial self-improvement, a burst of such improvement might peter out quickly. Maybe the AI makes fundamental breakthroughs in computing technology once, and then even with its newly improved capabilities, the next breakthrough is smaller, and the one after that is smaller still. Or maybe we hit a period of S-shaped growth, which seems super-exponential at first but then tapers off.
It's wise to be a bit skeptical in our dreams about the future. Yes, society might transform quite radically quite quickly, or on the other hand, maybe the "singularity" isn't even possible. We don't yet know what the scientific limits are.
Earlier quoted context omitted.
Curious what you mean by proactive? Could you share a bit more?
Happily, current AI is interacted with in a reactive loop. I open the Claude app, CLI, whatever, say my prompt, get an output. I personally wanted an AI that was able to reach out to me about my life before I had to reach out to it. An example, a friend just emailed me asking to meet for at 1pm but I have class at 1:30, so a proactive AI would see that conflict and send me a notification about it, asking if the propo…
Earlier quoted context omitted.
You people really have rotten morals
Well if it's moral for gatekeepers of talent to hoard immense intellectual rights to be protected by the violence of the state, then I'm immoral in that sense. Information wants to be free
Only ever said by people who have never put in the effort to create any information of value.
Earlier quoted context omitted.
> Arguably, the point of a startup isn’t to make money, it’s to attract attention and then investors. Investors with money.
What do you mean by this? What are investors who don’t have money? Perhaps I should have said investors and speculators , but this doesn’t help you here.
OP's startup is publishing because they think there's an economic advantage to doing so. That's not good or bad; it just is.
Very, very few things in the tech world are done "just because". Even fewer things when we're talking about AI being developed in the US. For better or for worse, the leadership in that segment of our field has established profit motive as their god.
Earlier quoted context omitted.
Which is a very ironic and selfish situation when your business model dependent mostly on model training based on available published data, academic and non-academic.
That's why we should enforce copyleft
I realize everyone always wants everything for free, but if you want someone to put the energy into writing up their invention you need to give them an incentive.
I can’t speak for other startups, but I applied to the most recent YC batch with my idea for making AI proactive instead of reactive, and pre-being selected I’ve published a paper on recursive self-improvement mapped to the Epoch AI data. I contacted a professor from a university in the UK and he responded since he was working on similar work, then asked me if I wanted to meet with him. We talked for about an hour si…
That's also a reason the big labs stopped. Publishing is most valuable to people who have no other way to get the attention of smart strangers. Once you can hire nearly anyone and everyone already returns your calls, the main remaining effect of publishing is to tell your competitors which things worked. This is what happens to every field as it turns from a science into an industry. Chemists published freely until d…