371–380 of 536 posts
Earlier quoted context omitted.
It won't be "happening" until Slack, Teams, and Discord leave Electron behind. They are the apps that need to be open 24/7.
It's not entirely clear what the connection is. We're not doing Electron because some popular software also using it. We're doing Electron because the ability to create truly cross-platform interfaces with the web stack is more important to us than 300 MB of user memory.
I’m a bit of an optimist. I think this will smack the hands of developers who don’t manage RAM well and future apps will necessarily be more memory-efficient.
> I think this will smack the hands of developers who don’t manage RAM well And hopefully kill Electron. I have never seen the point of spinning up a 300+Mb app just to display something that ought to need only 500Kb to paint onto the screen.
Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.
I would like a source for that statement. Additionally, I want to know by who? Because it certainly isn't end users. Inflating token usage doesn't make it any more economically viable if your user base, b2b or not, hasn't increased with it. On the contrary, that is a worse scenario for providers.
Earlier quoted context omitted.
The problem in this case seems to have sprung from a lack of collusion. Altman reportedly approached Samsung and SK independently to strike deals for a large chunk of both companies' production. Neither party apparently knew he was negotiating with the other. If they had actually been communicating or colluding with each other, they would have put the screws to him, making it harder for OpenAI to assert control over…
> Neither party apparently knew he was negotiating with the other. I don’t buy it that two of the largest manufacturers of DRAM in the world, from the same country , didn’t know this. Even of you ignore each company’s intelligence teams, that’s also the job of the country’s internal intelligence services, to make sure they know what all companies are doing and then make it so they have the best leverage to gain as mu…
By spying?
Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all th…
The market is already stagnated. Even if OpenAI doesn’t buy what they reserved other players will do so. SK Hynix CEO said there is a 20% gap between supply and demand per year. And that doesn’t account the shock effect that will take place the moment prices normalize and everyone and their dog will go out and start buying inventory to avoid the next crisis. I for one would certainly buy more than I currently need ju…
I’m a bit of an optimist. I think this will smack the hands of developers who don’t manage RAM well and future apps will necessarily be more memory-efficient.
> I think this will smack the hands of developers who don’t manage RAM well And hopefully kill Electron. I have never seen the point of spinning up a 300+Mb app just to display something that ought to need only 500Kb to paint onto the screen.
Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
> If AI makes software easier to create, that will drive the price down. Supposedly AI drives down the cost of producing software,not the "price". > How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? Currently, the cost of AI is between $20/month and around $200/month per developer.…
You can run huge local models slowly with the weights stored on SSDs.
Nowadays there are many computers that can have e.g. 2 PCIe 5.0 SSDs, which allow a reading throughput of 20 to 30 gigabyte per second, depending on the SSDs (or 1 PCIe 5.0 + 1 PCIe 4.0, for a throughput in the range 15-20 GB/s).
There are still a lot of improvements that can be done to inference back-ends like llama.cpp to reach the inference speed limit determined by the SSD throughput.
It seems that it is possible to reach inference speed in the range from a few seconds per token to a few tokens per second.
That may be too slow for a chat, but it should be good enough for an AI coding assistant, especially if many tasks are batched, so that they can progress simultaneously during a single read pass over the SSD data.
Earlier quoted context omitted.
The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.
> The usage of LLMs is continuing to increase ~exponentially I would like a source for that statement. Additionally, I want to know by who? Because it certainly isn't end users. Inflating token usage doesn't make it any more economically viable if your user base, b2b or not, hasn't increased with it. On the contrary, that is a worse scenario for providers.
The recent enterprise revenue numbers of Anthropic
Earlier quoted context omitted.
Is that relevant? The grid in every country is getting ridiculously stressed by datacenters.
High-level, I would agree with you. One thing that blows me away: I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA). How did they do it? In mainstream media, I don't see any news about a stressed power grid in this area. I guess the US gov't carefully coordinated with local power providers to con…
That's where AWS us-east-1 is, i.e. the oldest AWS region where they got started to begin with. Google and Microsoft also have a large presence there. It's not just the US government, it's everybody, and it's not new.
> How did they do it?
Here's the US nuclear plant map, guess where a bunch of them are:
https://www.eia.gov/todayinenergy/detail.php?id=65104
The area around Virginia is also a major coal producer and when this was getting started it was a source of cheap electricity, but coal is quickly being replaced with natural gas via pipelines from the Gulf coast. Their current power mix is ~30% nuclear, ~12% renewables (solar) and almost all the rest natural gas.