Live data from Hacker News

An analysis of Bitcoin's throughput bottlenecks

github.com

221–230 of 234 posts

Re: An analysis of Bitcoin's throughput bottlenecks

#221

Earlier quoted context omitted.

No I'm not confusing it. My exchange and all other exchanges charge a fee to withdrawal money. Its usually quite large.

Again, that fee is not something the exchange is charging you. The fee doesn't go to them. If it is bitcoin or ethereum there are very high fees for each transaction. The exchange is showing you those fees in your native currency. They aren't getting that money, it is going to the miners.

No you still are misunderstanding what I'm saying. The text on the exchange for me is like "We charge 0.05% fee on all withdrawals NOT INCLUDING the transaction fee to send to your wallet".

Re: An analysis of Bitcoin's throughput bottlenecks

#222

Earlier quoted context omitted.

Seriously? Bitcoin that has intentionally low transaction throuput is only two orders of magnitude slower than largest online payment processor that was designed with highest possible throuput in mind?

It is based on usage statistics not a technical benchmark. That number is not their peak throughput capacity it's how many transactions they actually do, which I'd wager is bottle-necked by user adoption not by tech.

Yes, let's wager on that.

Re: An analysis of Bitcoin's throughput bottlenecks

#223

Earlier quoted context omitted.

Thanks Jeremy! Are all those links related to congestion control? In short, that's basically allowing transactions to be shifted from high-congestion times to lower-congestion times without delaying the transfer (eg the transfer from the sender, even if the receive won't get the coins immediately). Is that right? Or are there more benefits of it?

Hi Billy, yes they all are. 1 -> high level idea, 2 -> basic simulation, 3 -> batching improvements, 4 -> the PR to bitcoin core. I have a lot more material scattered around too :) You understanding of the shifting from high congestion to low is spot on, but there are other benefits. The technique can also be used for more general smart contracts. I've been building a language, https://learn.sapio-lang.org , for expr…

Ah very interesting. So Sapio is a library for constructing somewhat arbitrary programs using CTV as a mechanism for enabling more turing complete operations? Is Sapio turing complete (or, I guess equivalently would Bitcoin be turing complete with the addition of CTV)?

So with non-interactively created channels, I don't quite understand how you would be able to immediately spend/route payments. Don't you always need to interact with your channel partner in order to do both of those things?

Re: An analysis of Bitcoin's throughput bottlenecks

#224

Earlier quoted context omitted.

Again, that fee is not something the exchange is charging you. The fee doesn't go to them. If it is bitcoin or ethereum there are very high fees for each transaction. The exchange is showing you those fees in your native currency. They aren't getting that money, it is going to the miners.

No you still are misunderstanding what I'm saying. The text on the exchange for me is like "We charge 0.05% fee on all withdrawals NOT INCLUDING the transaction fee to send to your wallet".

If you are not talking about the spread between exchanging two currencies, that should be a GIANT red flag to not use that exchange.

Re: An analysis of Bitcoin's throughput bottlenecks

#225

Earlier quoted context omitted.

>>I do mention in the paper that the assumptions are up for debate, but I did try pretty hard to justify those assumptions so they are certainly not arbitrary. Where in the paper did you justify those assumptions? I see them as not only totally arbitrary, but wrong according to any common sense analysis of threats or weighing of priorities. >>If by "biased" you mean that I decided what results I wanted to get, and th…

> Where in the paper did you justify those assumptions? I have a whole section on available machine resources which cites a ton of sources. I also wrote down specific goals and an exhaustive list of "Failure-Mode Considerations". The assumptions and requirements are based on matching available resources with the stated goals and preventing the listed failure modes. If they don't match up, I'd love to debate that. If…

>>I have a whole section on available machine resources which cites a ton of sources.

Maybe I'm just terrible at reading comprehension, but I don't see anything at all in the places you mention explaining why your assumption is that Bitcoin needs 90% of the world being able to run a full node for the network to remain safe from an attack. Can you actually copy-paste an excerpt from the analysis that explains this assumption?

>>How do you know how much on-chain space is needed by a full world of users? What assumptions do you make there?

I don't think I even have to provide a detailed analysis to make a high confidence claim that Bitcoin limited to 3 transactions per second cannot be used by the vast majority of the world population. I can't get mired down in over-analysis of every point we can already grant from a common sense deduction, as that will result in me not being allowed to even criticize your central assumptions due to be over-burdened by the analysis requirement.

>>Are you someone that thinks the lightning network won't scale and so all transctions must be on-chain?

The assumption that the lightning network will even work is extremely speculative, and the burden is on those making this assumption to support it, not on me to support the idea that it won't. There is no compelling reason to believe that an experimental network with extremely low adoption levels will ever prove highly useful, and gain widespread usage.

>>Instead it chooses goals and finds what requirements satisfy those goals.

Your paper suggests that Bitcoin can't scale to the point where less than 90% of the world population can run a full node using only 10% of hardware and network resources they already own, and remain safe from network attack, and not merely that it can't scale and satisfy the goal of 90% of the world population being able to run a full node using only 10% of hardware and network resources they already own. It doesn't justify its starting assumption.

>>That is true today becauase its a pain in the ass to run a full node.

I disagree. All it takes is to install the Bitcoin full node software on any computer with sufficient hardware resources, and start running it, with next to no configuration. You can even buy plug-and-play Bitcoin nodes where the full node software comes already installed on the device, to skip that trivial step of downloading and installing the software.

>>It is not neccessarily the case that running bitcoin is only for the minority that are "committed individuals", but instead its, in part, a function of how much resources running a full node takes. In other words, the number of people likely to run a full node is not independent of the resources a full node takes to run.

I don't see how you're justifying your argument that running a full node is not for the minority of people that are "committed individuals". You even argued that "it takes some research to configure" a Bitcoin full node, which supports my claim that under any plausible hardware requirements, it would be predominantly committed individuals who would go through the trouble of doing it.

Moreover, I clearly acknowledged that the resources a full node takes to run affects the number of people likely to run it, as I specifically restricted the percentage of the world population able to run a full node under high-scalability assumptions to 1%, or 80 million, on the basis that only they could afford to purchase those resources, My claim implies 99% of the world population would not be likely to run a full node in a high-scalability scenario, due to the unaffordability of acquiring the hardware and bandwidth resources for this cohort.

>>Part of what underlies that goal is that we need to maximize the number of people running public full nodes. The Bitcoin network is massively underserved by public full nodes, see the section on Sybil and Eclipse attacks: https://github.com/fresheneesz/bitcoinThroughputAnalysis#syb... .

Your analysis negates the existence of strategies for countering Sybil and Eclipse that would be far less costly to Bitcoin's utility, and far more likely to actually work, than imposing extreme restrictions on Bitcoin's throughput, which would only work if the extremely inconsistent assumption that tens of millions of people who currently cannot transact on Bitcoin will run a full node.

For example, there's trust-based network-peering: http://www.cs.columbia.edu/~danr/courses/6772/Fall06/papers/...

>>I disagree. Do you think 10s of millions of people are willing to do that? I very much doubt it. I certainly don't think we can rely on that to happen.

Strawman. I never said that 10s of millions of people would be willing to do that, or this would be needed to maintain resistance to cooption.

>>Lack of outcry is not what I'm claiming. I'm claiming that if >50% of the economic activity on the network is done via SPV nodes, rules can be changed in a way that will instantaneously affect those SPV nodes. People do not react instantaneously.

The attack does not need to be instantaneously neutralized for Bitcoin to survive it. It can incur significant short-term losses, and still not come close to erasing the social gains from allowing Bitcoin to be operational for billions of people for many years.

>>Regardless, even a 1% probability of that happening per century is far too high I think.

This conclusions on an extremely inconsistent set of assumptions. On what basis is 99.9% of the world population not being able to use Bitcoin due to the current extreme restrictions on throughput, a better scenario than there being a 1% probability of a scalable and mass-adopted Bitcoin suffering a failure of the type you describe once per century?

>>Wallets are likely to connect to random SPV servers on the network in the future, just like full nodes connect to random nodes.

There are relatively low-cost ways to prevent that from being the case in the future.

>>A centralized authority is not strictly necessary. This may simply be a contentious change that miners want. Maybe there's a loud minority of users on board too. Think about the bch fork.

It would require more than just miners that want it. It would require the majority of major stakeholders, including those running SPV data servers, to want it, and for that change not to face significant public opposition, or else those SPV data servers who want the change would face social backlash and a mass migration away from reliance on them.

>>One kind of political attack I can think of is one where some group decides they don't like bitcoin, and want to kill it. So they try to make laws that make bitcoin harder to use, or perhaps outright ban it. Increasing bitcoin's adoption would help reduce the number of people who don't have a stake in bitcoin, and thus reduce the number of people who would support such an attack. Increasing on-chain throughput may increase adoption (at what rate, I don't know), which would increase the rate at which we achieve some minimum level of adoption (perhaps 20%-30%?) to where an attack is unlikely to succeed.

Yes, and I think of all the ways to mitigate the risks to Bitcoin, this is the most effective one, that results in the greatest reduction of risk to Bitcoin.

>>Another kind of political attack I can think of is one where some group decides to advocate for some dangerous change to bitcoin. They convince a lot of people to want this change, and thus have the backing to attempt to get people to use that change.

If they can convince real people to support the change, they can also support real people running full nodes to support the change, so this risk is not alleviated by restricting transaction throughput to lower the cost of running a full node.

> convince me that the assumptions make any sense at all from the perspective of someone who wants to see people financially empowered by cryptocurrency

It sounds like you're asking me to bias my assumptions, or to explain to you how my assumptions are biased in a way you like. That honestly sounds a bit hypocrticial.

>>My goal is to find the ideal tradeoffs that make the Bitcoin system be secure against adversarial attacks, not to find tradeoffs that maximize the adoption rate of bitcoin but as a result leave the system vulnerable to significant attacks.

Pursuing the goal of "seeing people financially empowered by cryptocurrency" would also involve pursuing the goal of "mak[ing] the Bitcoin system be secure against adversarial attacks", but it does not hold it as the only goal. It's a broader and more sensible goal to pursue, since a highly secure Bitcoin network is only useful to the extent that people can use it.

>>I'm just going to ignore your accusations for now but if you continue to insult me I might just stop talking to you.

If I see evidence of bias in your writings, I will express my opinion on it.

Re: An analysis of Bitcoin's throughput bottlenecks

#226

Earlier quoted context omitted.

> Where in the paper did you justify those assumptions? I have a whole section on available machine resources which cites a ton of sources. I also wrote down specific goals and an exhaustive list of "Failure-Mode Considerations". The assumptions and requirements are based on matching available resources with the stated goals and preventing the listed failure modes. If they don't match up, I'd love to debate that. If…

>>I have a whole section on available machine resources which cites a ton of sources. Maybe I'm just terrible at reading comprehension, but I don't see anything at all in the places you mention explaining why your assumption is that Bitcoin needs 90% of the world being able to run a full node for the network to remain safe from an attack. Can you actually copy-paste an excerpt from the analysis that explains this ass…

> I don't see anything at all in the places you mention explaining why your assumption is that Bitcoin needs 90% of the world being able to run a full node for the network to remain safe from an attack. Can you actually copy-paste..?

I don't have any good clips of text that would connect them better. A lot of the connection between them is left vauge and implied. I agree this can be improved. I've created an issue for this: https://github.com/fresheneesz/bitcoinThroughputAnalysis/iss... Feel free to comment there.

> I don't think I even have to provide a detailed analysis to make a high confidence claim that Bitcoin limited to 3 transactions per second cannot be used by the vast majority of the world population.

I certainly agree that 3 tps on chain is not enough for everyone to only transact on chain. However, there are many off chain ways people transact, some better than others. If your analysis were to make that a critical piece of the discussion, it would have to be justified in more depth. But I'm not trying to ask you to do hours of analysis here.

> Your paper suggests that Bitcoin can't scale to the point where less than 90% of the world population can run a full node using only 10% of hardware and network resources they already own, and remain safe from network attack

You may read what you want, but the intention of the paper was primarily to create a reusable way to evaluate the safe/secure throughput limits of bitcoin. The intention was explicitly not to come up with the "correct" or "best" assumptions or requirements of the network. It simply chose conservative limits and rolled with those. I was very clear about this in the paper. I'm happy to discuss the assumptions, but if your problem with the paper is that the assumptions aren't where you think they should be, you're asking of the paper something outside of its scope.

> You even argued that "it takes some research to configure" a Bitcoin full node, which supports my claim that under any plausible hardware requirements, it would be predominantly committed individuals who would go through the trouble of doing it.

I don't think you're understanding me. You are explaining the current state of things. I'm thinking about a future where bitcoin doesn't take research to configure, and is a no brainer to run. You don't have to worry about reaching bandwidth caps or slowing down your apartment's internet or making your machine slow when you're playing a game. We agree that it currently takes committed people to run bitcoin. I think that needs to change.

> Your analysis negates the existence of strategies for countering Sybil and Eclipse that would be far less costly to Bitcoin's utility

While I would love to hear about new strategies my paper doesn't incorporate, the first half of the paper explicitly doesn't incorporate new technology. It analyses the network as created by current bitcoin software.

> For example, there's trust-based network-peering

That's interesting. I worry that relying on trust might introduce issues, but its interesting nonetheless. I'll read through that. Thanks!

> Strawman. I never said that 10s of millions of people would be willing to do that

When I wrote that sentence, I still misunderstood what you meant. I forgot to revise after realizing you just meant that 10s of millions of people would need to be able to run full nodes, but it seems you think that far fewer would actually have to run full nodes.

> The attack does not need to be instantaneously neutralized for Bitcoin to survive it. It can incur significant short-term losses, and still not come close to erasing the social gains from allowing Bitcoin to be operational for billions of people for many years.

Perhaps you're right. But this paper neither refutes nor confirms that, and analysizing the optimal tradeoffs there is, again, out of scope for the paper.

> There are relatively low-cost ways to prevent that from being the case in the future.

Yes and many of those ways are even less secure. If you're referring to the trust-based network-peering you mentioned, perhaps there's something there, but trust-based is by definition not trustless, which is generally what bitcoin is built to be. I agree some kinds of small but less-than-minimal trust could definitely be useful. But the fact is that bitcoin doesn't currently have those mechanisms.

> It would require the majority of major stakeholders, including those running SPV data servers, to want it, and for that change not to face significant public opposition

It would be quite easy for an attacker to Sybil the network of SPV servers, so those wouldn't help in an attack. I think our disagreement here is that I'm talking about whether a damaging attack would be feasible, and you're talking about whether the attack would destroy bitcoin. Those are very different statements. Do you agree that a damaging attack could happen in the scenario you're describing?

> It's a broader and more sensible goal to pursue

Perhaps. But the scope of the paper is broad enough. I hope you can appreciate that ideas an techniques in the paper can be useful even if the goal is not something you agree with.

Re: An analysis of Bitcoin's throughput bottlenecks

#227

Earlier quoted context omitted.

Hi Billy, yes they all are. 1 -> high level idea, 2 -> basic simulation, 3 -> batching improvements, 4 -> the PR to bitcoin core. I have a lot more material scattered around too :) You understanding of the shifting from high congestion to low is spot on, but there are other benefits. The technique can also be used for more general smart contracts. I've been building a language, https://learn.sapio-lang.org , for expr…

Ah very interesting. So Sapio is a library for constructing somewhat arbitrary programs using CTV as a mechanism for enabling more turing complete operations? Is Sapio turing complete (or, I guess equivalently would Bitcoin be turing complete with the addition of CTV)? So with non-interactively created channels, I don't quite understand how you would be able to immediately spend/route payments. Don't you always need…

Yep! It's an e-dsl essentially, since you write a program and it compiles to bitcoin transactions.

Sapio is turing complete (trivially, you can call any program you can write in regular rust from a Sapio contract). However, the output is a static set of transactions, which is not turing complete. However, if you have an updatable "finish!" clause, then Sapio can generate the logic for a "next step if N parties agree" or something similar, which lets you express the continuation logic in Sapio.

For your last question, I think it is semantics. An "interaction" is (by my perhaps non-standard and definitely inconsistent terms) a back and forth communication between two parties. A non-interactively created channel is set up by either party (or a third party) and then the funds cannot move without the signoff of either party. However there's no guarantee that the funds can move, unless the parties receive a transmission from the creator informing them of the setup (think of this kind of like a memory leak in rust?). This isn't really an interaction as it can happen fully asynchronously, but it's sort of weird because if you don't get it you don't get paid... but assuming the person paying you wanted to pay you, they can try to let you know later!

Then, payments (in one direction, i.e. A->B but not B->A) can be done in the same async half-interactive way. You just get the broadcast and save it to get more money.

This might seem really niche (it is) but carving out these little pockets of what constitutes an interaction opens the door for some really interesting types of devices. E.g., imagine a "low power smart meter" that you can open a channel to and pay multiple times, but the smart meter only has a public key on it, no private key. All state (incl block headers) can be SPV checked and stored locally. Requiring a full interaction means that these devices have to have a private key accessible. This is sort of a contrived example, because there are better ways to make the smart meter, but it shows you the edge that exists at least.

Re: An analysis of Bitcoin's throughput bottlenecks

#228

Earlier quoted context omitted.

Ah very interesting. So Sapio is a library for constructing somewhat arbitrary programs using CTV as a mechanism for enabling more turing complete operations? Is Sapio turing complete (or, I guess equivalently would Bitcoin be turing complete with the addition of CTV)? So with non-interactively created channels, I don't quite understand how you would be able to immediately spend/route payments. Don't you always need…

Yep! It's an e-dsl essentially, since you write a program and it compiles to bitcoin transactions. Sapio is turing complete (trivially, you can call any program you can write in regular rust from a Sapio contract). However, the output is a static set of transactions, which is not turing complete. However, if you have an updatable "finish!" clause, then Sapio can generate the logic for a "next step if N parties agree"…

Well, Sapio seems pretty cool. Composable scripts is definitely something we need in Bitcoin.

So basically, the idea of non-interactive channel creation sounds basically like a way for someone to lock in their payment to someone without interacting with them. I suppose I can see the usefulness in that in certain situations. Especially situations where someone that you can interact with does care that you've paid the person who can't interact at the moment.

Re: An analysis of Bitcoin's throughput bottlenecks

#230

Earlier quoted context omitted.

Yep! It's an e-dsl essentially, since you write a program and it compiles to bitcoin transactions. Sapio is turing complete (trivially, you can call any program you can write in regular rust from a Sapio contract). However, the output is a static set of transactions, which is not turing complete. However, if you have an updatable "finish!" clause, then Sapio can generate the logic for a "next step if N parties agree"…

Well, Sapio seems pretty cool. Composable scripts is definitely something we need in Bitcoin. So basically, the idea of non-interactive channel creation sounds basically like a way for someone to lock in their payment to someone without interacting with them. I suppose I can see the usefulness in that in certain situations. Especially situations where someone that you can interact with does care that you've paid the…

Yeah, it's also just simpler in general.

E.g., Alice and Bob request that coinbase pay them into a channel. Coinbase can just do it according to the instructions from Alice and Bob, and Alice and Bob can be offline thereafter.

Post reply on HN