Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Long Live the 'GPU Poor' – Open-Source AI Grants

a16z.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#71

Post by seism »

Hi Robin Hood. I wonder about the "continue their work without the pressure to generate financial returns" part. I don't see an expectation to work on non-commercial, or socially conscious, applications, just to commit to open source (what level of commitment also being unclear). It would actually be great if support programs along these lines were being offered by governments and public service institutions, with some serious governance and transparency around the process.

Surely there are also some interesting econometrics on the "GPU poverty line" - someone has literature to share?

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#72

Post by calderwoodra »

bodecker wrote: post #49

Congrats to the folks involved. Even knowing this is partly motivated by branding/marketing, it's great to see a16z getting more aligned with solving real pain points (vs crypto and churning out shallow media in recent years). Hope they can keep it up and hopefully more "thought leaders"/VCs follow suit. Best of luck.

I agree this initiative helps solve real pain points, and I'm not trying to defend "Crypto" with my question here but isn't solving a "real pain" kinda subjective? For example, many people/companies have the real pain/need to improve ad click-through-rates and reduce crypto/gas fees but maybe these aren't problems you care about.

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#73

Post by junipertea »

naillo wrote: post #70
redox99 wrote: post #63

Earlier quoted context omitted.

25K will either get you money for renting servers, or allow you to jerry rig yourself something like a 8x4090 machine. The servers that most people would like to run (8xA100, 8xH100) are over 200k, and even if you had the money they're probably very hard to get right now.

8x4090 machines should be a lot cheaper than that. 1599*8=12,792. I.e. closer to 16x4090 machines. Sure labor is worth something but he has it in cash, and also there's barely a lot of people (yet) paying high prices for research (non ML ops) labor hours for this yet so the salary wouldn't be that high anyway. Just a pet peeve when ppl under exaggerate how many gpus/flops a dollar can buy (if you're slighly smart wit…

What about PSU to power all of this, GPU motherboards, SSDs, CPU to not bottleneck this and sufficient RAM? You could argue it's more than 8, but definitely not 16. To utilize those GPUs to the max you need memory bandwidth to even feed it enough data.

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#75

Post by malux85 »

rajko_rad wrote: post #28
redox99 wrote: post #20

I want to congratulate you based on your current grant recipients. They are all very impactful individuals/teams. It shows that you clearly did your research and put the money in the right places.

thank you!! Indeed, we have been interacting in space very closely and doing our best to keep up and contribute where possible

Is there a way we can apply? I am building https://atomictessellator.com, and since I am 100% technical I am not sure about commercialising or open sourcing it, I definitly would like to open source some of the components (e.g. turnkey dockerized scientific computing environments, the molecular force viewer, and also open up a hub for the environments and models) - when I was in Stanford meeting with some comp chemists a few months ago they got very excited for this.

I negotiated with my day job to work 2AM to 10AM so that I can work on this passion project in the afternoons, some grant funding would really help me focus on this more. There's quite a bit of interesting stuff not on the website in catalyst discovery and molecular dynamics. Contact me if you'd like to chat / demo.

In my ideal world I would work on this full time and totally open source it.

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#76

Post by samstave »

rajko_rad wrote: post #25
latchkey wrote: post #6

Earlier quoted context omitted.

I'm curious, where are you getting the GPUs?

in most cases the teams already had their own source or platform of choice!

Maybe build a FAQ for others on how to accomplish, or resources you can point people at?

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#77

Post by redox99 »

naillo wrote: post #70
redox99 wrote: post #63

Earlier quoted context omitted.

25K will either get you money for renting servers, or allow you to jerry rig yourself something like a 8x4090 machine. The servers that most people would like to run (8xA100, 8xH100) are over 200k, and even if you had the money they're probably very hard to get right now.

8x4090 machines should be a lot cheaper than that. 1599*8=12,792. I.e. closer to 16x4090 machines. Sure labor is worth something but he has it in cash, and also there's barely a lot of people (yet) paying high prices for research (non ML ops) labor hours for this yet so the salary wouldn't be that high anyway. Just a pet peeve when ppl under exaggerate how many gpus/flops a dollar can buy (if you're slighly smart wit…

Meh, I was just ballparking. But add a threadripper, compatible motherboard, lots of RAM, fast storage, a few beast PSUs, and tax, and you're probably around 18k. So you ain't getting 2 of these machines for 25k.

Plus you'll need to set aside some money for the power bill lol

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#78

Post by rajko_rad »

samstave wrote: post #76
rajko_rad wrote: post #25

Earlier quoted context omitted.

in most cases the teams already had their own source or platform of choice!

Maybe build a FAQ for others on how to accomplish, or resources you can point people at?

Interesting, hadn’t thought of that, thank you! If you want to host end points Replicate is a great option, they also have a newer fine tuning api and solution! for raw VMs with GPUs right now it’s a bit situational and you have to try multiple different vendors tbh, also really depends on the capacity you need and which machines!!

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#79

Post by samstave »

calderwoodra wrote: post #72
bodecker wrote: post #49

Congrats to the folks involved. Even knowing this is partly motivated by branding/marketing, it's great to see a16z getting more aligned with solving real pain points (vs crypto and churning out shallow media in recent years). Hope they can keep it up and hopefully more "thought leaders"/VCs follow suit. Best of luck.

I agree this initiative helps solve real pain points, and I'm not trying to defend "Crypto" with my question here but isn't solving a "real pain" kinda subjective? For example, many people/companies have the real pain/need to improve ad click-through-rates and reduce crypto/gas fees but maybe these aren't problems you care about.

Solving pain point == A problem we can profit from. Always has been.

Plus A16Z has some questionable motives in general based on previous comments by them...

Re: Long Live the 'GPU Poor' – Open-Source AI Grants

#80

Post by thewataccount »

rajko_rad wrote: post #4

Hi HN, we are super excited to announce this new initiative!! Please note, this program was designed to support individuals, teams, and hackers who are not pursuing commercial companies. Nevertheless, these projects push the state of the art in open source AI and help provide us with a more robust and comprehensive understanding of the technology as it is developed. We are really proud to be contributing in this smal…

Hello, I think this is a great project!

If an individual is looking to contribute to the field with different training data ideas - would they need to first establish themselves and get your attention, or would there be a way to submit a proposal?

For myself compute is rather expensive, I could likely afford a few test runs for proofs of concept but beyond that it would be difficult. I've got a single 4090 so I can't run llama70b faster then 1it/s.

Post reply on HN