Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's top startups are barely publishing their research

science.org

Post reply on HN
Print view View original

181–190 of 341 posts

Re: AI's top startups are barely publishing their research

#181

Post by geraneum »

gowld wrote: post #30
randomImmigrant wrote: post #28

What the blogificafion of AI research has done is allowed all kinds of claims and terminology related to AI to be introduced and taken up in a manner replicating social media dynamics. And that is simply not healthy. We’re fast reaching a place where any claim can be backed up with a set of numbers from a number of experiments run in some gamified environment or the other, with little concern for if it all adds up to…

How is that different from traditional research publication? It's faster?

That pesky “scientific method” bothers these corps sometimes, so one can play fast and loose with numbers and conclusions for all sort of reasons. No need to worry about reproducibility, rigor, scrutiny, etc. not good for IPO and stocks.

Re: AI's top startups are barely publishing their research

#182

Post by botacode »

There was a good paper documenting this in broader industry, and highlighting how it may account for previous research suggesting good ideas have been getting harder to find. Here's the abstract:

> Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.

https://faculty.tuck.dartmouth.edu/uploads/teresaFort/files/...

Re: AI's top startups are barely publishing their research

#184

Post by himata4113 »

threethirtytwo wrote: post #173
himata4113 wrote: post #130

Earlier quoted context omitted.

it's actually the end of the beginning, there's walls that self-improving models hit that are never overcome even when given vast amounts of time.

I don't think anyone has ever tried having a model fully autonomously train a model that is better than it.

Everyone including myself has attempted this relentlessly, it just doesn't work out beyond some arbitrary improvement in specific tested categories instead of broader capability increase.

Re: AI's top startups are barely publishing their research

#185

Post by make3 »

jandrewrogers wrote: post #136
noosphr wrote: post #66

I've been at two startups that have done genuine world first fundamental research. The first tried to publish novel results for 3 years in tier 1 journals before finally doing a preprint and telling the tier one publishers to jump in a fire. The second, and ongoing, isn't publishing anything because of my experience with the first. That and avoiding openAI and Anthropic copying our results and leaving us with nothing…

There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do. Cool for Google. Problematic if you are a startup.

I think talent would definitely take a startup that publishes more seriously, and it would be a smaller risk for top research talent to go somewhere where they can also publish than have to bear the full risk of their work at a no name startup not going anywhere

Re: AI's top startups are barely publishing their research

#186

Post by make3 »

sheeshkebab wrote: post #144
jandrewrogers wrote: post #139

Earlier quoted context omitted.

The patents are effectively unenforcible. People stopped patenting frontier computer science research 20 years ago. That is the current reality.

good.

Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable? Do you want any communication of knowledge from corporations? Should companies not be allowed to benefit from sometimes huge investments in research?

Re: AI's top startups are barely publishing their research

#187

Post by fc417fc802 »

fluoridation wrote: post #174
fc417fc802 wrote: post #169

Earlier quoted context omitted.

Well it was a joke and is obviously quite silly but I believe it would be enforceable to the extent that the licensor could terminate the agreement and sue for damages. If I can agree to pay you not to talk about something (ie an NDA) or not to work in a field (ie a non-compete clause) then why can't I pay you to be required to publish all future work you do in a given area? ("All future work" might well be overly br…

>terminate the agreement Meaning what? Claw back the ideas from people's minds? You can terminate the agreement in the sense that you revoke access to the paper, but presumably the person you find in breach has already used the research for something that you find them in breach for. You're kind of closing the gate after the horse has bolted. >sue for damages I honestly have no idea what damages you could claim from…

> Meaning what? Claw back the ideas ...

If something is so obviously wrong then perhaps take a minute to consider that your interpretation isn't what the other party intended?

If I pay you not to do something and then you breach the contract I can terminate the agreement and seek damages. Ditto if I pay you to repeatedly do something and then at some point you fail to do it. So if I pay you a recurring fee to publish all your research on a given topic and then you fail to make good on that I can seek damages, right? Now what if I paid you a lump sum up front? Now what if I licensed a patent to you in place of that lump sum? What if instead of a patent it was the right to make use of a piece of software?

> I honestly have no idea what damages you could claim from not publishing research.

Aside from whatever was stipulated for breach of contract I expect it would be extremely situational. I agree that you'd probably want to stipulate a penalty ahead of time. The original idea was a two sentence joke after all ...

> Not sure why you added the word "pay" to your clauses

Because contracts involve consideration and payment is easy to understand and reason about. FOSS software licenses obviously substitute "right to use the code" for "payment".

> A non-compete clause stops you from working in a very specific field. Your proposal has an undefined reach. What counts as research? What counts as "related"? It would seem that if I agree to such a contract, my entire life, both private and professional, after reading the paper is covered by the contract, and anything I do might come under scrutiny.

This is incredibly contrived. You could ask the equivalent about a non-compete. The reach is whatever is defined in the contract that both parties agreed to. If the contract stipulates something overly broad then possibly a judge would invalidate it. This is business 101.

> It's definitely not obvious that the terms are legally valid, but it's ambiguous enough that people don't want to test it.

An assertion entirely without evidence from my perspective. I'm going to assume that all the lawyers who have sure left me with the impression that it would be a bad idea to violate it know what they're talking about.

Re: AI's top startups are barely publishing their research

#188

Post by johnnyanmac »

chrisweekly wrote: post #163
ninjahawk1 wrote: post #90

I can’t speak for other startups, but I applied to the most recent YC batch with my idea for making AI proactive instead of reactive, and pre-being selected I’ve published a paper on recursive self-improvement mapped to the Epoch AI data. I contacted a professor from a university in the UK and he responded since he was working on similar work, then asked me if I wanted to meet with him. We talked for about an hour si…

Yes! And, whether it stems from research or not, putting yourself out there and talking to other people is one of the most fundamentally important things you can do for your career and your personal development as a human being.

Tell that to my tinder profile.

I jest, no tinder. But still, be ready for a lot of non-responses if you're not actively in college. It can feel like a lonely world out there despite theoretically being hundreds of potential people you'd be able to talk to for hours.

Re: AI's top startups are barely publishing their research

#189

Post by mkolodny »

jandrewrogers wrote: post #136
noosphr wrote: post #66

I've been at two startups that have done genuine world first fundamental research. The first tried to publish novel results for 3 years in tier 1 journals before finally doing a preprint and telling the tier one publishers to jump in a fire. The second, and ongoing, isn't publishing anything because of my experience with the first. That and avoiding openAI and Anthropic copying our results and leaving us with nothing…

There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do. Cool for Google. Problematic if you are a startup.

Google has been publishing frontier research since it was an early-stage startup (https://research.google/pubs/the-anatomy-of-a-large-scale-hy...).
Post reply on HN