Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: 0xFast – Faster Web3 APIs

0xfast.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Show HN: 0xFast – Faster Web3 APIs

#61

Post by com2kid »

IanCal wrote: post #54
com2kid wrote: post #43

Earlier quoted context omitted.

> The only thing relevant here is that blockchain data is almost exclusively append-only and so you can build something neater because you've got guarantees that many other problems don't have. Append only databases have been around for decades. Heck Kafka is an obscenely popular example today. I've talked to people working in fields with legally mandated immutable databases, again decades old technology. In the most…

I think you're arguing around something very different to what I'm saying. None of this is to do with whether web3 stuff is a good idea, or whether the underlying Blockchain setup/concept is useful or whether distributed whatevers are good. This is a product launch giving access to that data under a common API.

I agree that the product itself seems like a nicer way to access web3 stuff! And I'd actually be interested in some blog posts about how the authors got such dramatic speedups.

Re: Show HN: 0xFast – Faster Web3 APIs

#62

Post by dmitriid »

IanCal wrote: post #47
dmitriid wrote: post #12

Earlier quoted context omitted.

> The only thing relevant here is that blockchain data is almost exclusively append-only So, Apache Kafka. Edit: as anyone who actually does this for a living: you never use the append-only log as your source of data. Precisely because it's very expensive to reconstruct it > so you can build something neater because you've got guarantees that many other problems don't have. So, first you have to find problems for you…

> So, Apache Kafka. Does that give access to the full history quickly? > Edit: as anyone who actually does this for a living: you never use the append-only log as your source of data. Precisely because it's very expensive to reconstruct it Yeah I don't want to do that, but I do need to in order to process the data. For those that don't want to do this, and want to quickly access just the latest state, that's exactly…

> Does that give access to the full history quickly?

It doesn't because no append-only log does. Apache Kafka is that "unique web3-only thing never seen before in traditional systems" that no only has been seen in traditional systems, but has been successfully used in those systems at scale dwarfing all of crypto, and has well-understood behaviours and patterns.

Edit: In 2019 Linkedin (who developed Kafka) were processing 7 trillion messages a day: https://engineering.linkedin.com/blog/2019/apache-kafka-tril...

Edit: In 2018 Pinterest was handling 15 million messages a day with 1.2 petabytes of data per day https://medium.com/pinterest-engineering/how-pinterest-runs-...

A view of the final state of data is literally one of the things they require, and everyone knows how to make one.

> For those that don't want to do this, and want to quickly access just the latest state, that's exactly what services like this offer.

So this service just creates a "traditional" database with a "traditional" access method (actually an ancient access method, RPC) with a "traditional" ElasticSearch on top and pretends it's something new, never made at scale, and something no "traditional system" was ever prepared for.

Edit: see LinkedIn and Pinterest above. Or Facebook. Or Amazon. Or AliExpress. Or... "Omg they traditional web2 tech cannot work at the amazing scale of web3" is such a bold-faced lie that I can't imagine people who can look at it and say "true, true". The amazing scale of web3 can barely process a few hundred thousand transactions per day, all of them with minuscule payloads.

Re: Show HN: 0xFast – Faster Web3 APIs

#63

Post by dmitriid »

IanCal wrote: post #60
CharlesW wrote: post #24

Earlier quoted context omitted.

> Translation: you are dumping blockchain data into traditional databases running on traditional servers and backed up by a traditional Elastic Search. It's ironic that the solution to blockchain problems is so often re-centralization.

I'm not sure of the larger right term for this but it's worth pointing out this is one of a bunch of different providers all offering the same API to the same public data, and you can create your own with different open source offerings.

> I'm not sure of the larger right term for this

Scam. Lies and scam. That is the term for it.

> one of a bunch of different providers all offering the same API to the same public data

It's not the API I object to.

Re: Show HN: 0xFast – Faster Web3 APIs

#64

Post by dmitriid »

dsco wrote: post #38
Geee wrote: post #23

Earlier quoted context omitted.

Web3 is decentralized web, like web 1.0 - but with tokens this time! Tokens solve the issue because you can buy and hold them and get rich! Buy my shitcoin! Seriously web3 is just another way to create hype for arbitrary crypto tokens and Ethereum. In a few years there will be something else - quantum web or something to suck new buyers in.

This doesn’t provide any value to the conversation. Imagine having good intentions, working in web3, and always seeing this type of comment on HH. Would you want it that way in whatever field you’re working in?

> Imagine having good intentions,

I have very hard time imagining anyone working in web3 with good intentions.

Re: Show HN: 0xFast – Faster Web3 APIs

#65

Post by dmitriid »

com2kid wrote: post #61
IanCal wrote: post #54

Earlier quoted context omitted.

I think you're arguing around something very different to what I'm saying. None of this is to do with whether web3 stuff is a good idea, or whether the underlying Blockchain setup/concept is useful or whether distributed whatevers are good. This is a product launch giving access to that data under a common API.

I agree that the product itself seems like a nicer way to access web3 stuff! And I'd actually be interested in some blog posts about how the authors got such dramatic speedups.

> And I'd actually be interested in some blog posts about how the authors got such dramatic speedups.

You did mention Kafka. They got the speed ups exactly how you get views from Kafka: dump them into a PostgreSQL, have an ElasticSearch on top.

The entire throughput of the most popular chains is ~20-50 transactions per second. You can probably handle this immense load at scale that "no traditional system is suited for" on a free tier of any cloud hosting provider.

Re: Show HN: 0xFast – Faster Web3 APIs

#67

Post by IanCal »

dmitriid wrote: post #62
IanCal wrote: post #47

Earlier quoted context omitted.

> So, Apache Kafka. Does that give access to the full history quickly? > Edit: as anyone who actually does this for a living: you never use the append-only log as your source of data. Precisely because it's very expensive to reconstruct it Yeah I don't want to do that, but I do need to in order to process the data. For those that don't want to do this, and want to quickly access just the latest state, that's exactly…

> Does that give access to the full history quickly? It doesn't because no append-only log does. Apache Kafka is that "unique web3-only thing never seen before in traditional systems" that no only has been seen in traditional systems, but has been successfully used in those systems at scale dwarfing all of crypto, and has well-understood behaviours and patterns. Edit: In 2019 Linkedin (who developed Kafka) were proce…

> It doesn't because no append-only log does.

Then it doesn't solve the problem people in the space have. I understand the visceral hate for web3, I do, but it really gets in the way of talking about a very bland technical issue.

I think you're misunderstanding what this service is. It's about access to Blockchain data not about running a chain.

> A view of the final state of data is literally one of the things they require, and everyone knows how to make one.

That's one thing it needs, but only one thing. Keeping up to date with this is a problem of monitoring mostly to keep on the right side of any forks. It's not a data problem, but that's not what the API is providing.

> Omg they traditional web2 tech cannot work at the amazing scale of web3" is such a bold-faced lie

It totally would be, but it's not what the site said. It's literally just saying that they're doing better than off the shelf tools because of guarantees most other problems don't have. And I'm not surprised, the data isn't of a crazy scale though is a bit annoying, access patterns are extremely predictable and depending on which APIs you're supporting building a specific thing for it is fairly straightforward.

> The amazing scale of web3 can barely process a few hundred thousand transactions per day, all of them with minuscule payloads.

Nobody is talking about ingest speed really. Unless you're also applying all the state transitions because that's quite annoying.

Re: Show HN: 0xFast – Faster Web3 APIs

#68

Post by dmitriid »

IanCal wrote: post #67
dmitriid wrote: post #62

Earlier quoted context omitted.

> Does that give access to the full history quickly? It doesn't because no append-only log does. Apache Kafka is that "unique web3-only thing never seen before in traditional systems" that no only has been seen in traditional systems, but has been successfully used in those systems at scale dwarfing all of crypto, and has well-understood behaviours and patterns. Edit: In 2019 Linkedin (who developed Kafka) were proce…

> It doesn't because no append-only log does. Then it doesn't solve the problem people in the space have. I understand the visceral hate for web3, I do, but it really gets in the way of talking about a very bland technical issue. I think you're misunderstanding what this service is. It's about access to Blockchain data not about running a chain. > A view of the final state of data is literally one of the things they…

This is what the website literally claims:

=== start quote ===

Web2 Databases were meant to deal with datasets that change frequently. They have to provide ACID guarantees to ensure correctness.

The solution to this problem becomes more complex with scale. Distributed transactions, sharding, replication, and fault tolerance are notoriously slow.

Web2 databases are just NOT designed for web3

A system designed to take full advantage of the fact that blockchain data is immutable, and correctness has been dealt with already.

We built a cheaper and simpler storage and indexing system from scratch that can scale to the heaviest blockchain workloads out there.

=== end quote ===

Then here's what you claimed

=== start quote ===

Web3 data is different from what most classic databases are optimised for, because of how it's accessed and the (mostly) immutable guarantees.

The only thing relevant here is that blockchain data is almost exclusively append-only and so you can build something neater because you've got guarantees that many other problems don't have.

=== end quote ===

So, to give you a doze of reality and dispel the lies on the web site I showed that your view of the world does not reflect what the world is:

- Blockchain isn't the only thing to have an immutable log. Apache Kafka is one, and is being used at scale that dwarfs anything in crypto space

- "The heaviest blockchain workloads out there" as claimed by the web site have literally nothing on "traditional/classical" whatever because traditional/classical handle workloads that exceed blockchains by multiple orders of magnitude

- There's nothing revolutionary in the service despite its bullshit claims: they dump blockchain data into a traditional database, slap a traditional (I'd say conservative) JSON-RPC API on top, and probably add a traditional Elastic Search on the side. Everything that they accuse "traditional systems" incapable of doing and incapable of handling. Even though, again, traditional systems can probably handle all this on a free tier of any cloud hosting provider

We've known how to handle append-only mostly-immutable data at scale using "traditional" methods for ages. Literally every banking system is all but required to have an append-only log of transactions for audit purposes. And yet here we are with clueless gullible fools pretending this is something new and something that can't be handled by "outdated slow technologies of yore".

Re: Show HN: 0xFast – Faster Web3 APIs

#69

Post by IanCal »

dmitriid wrote: post #68
IanCal wrote: post #67

Earlier quoted context omitted.

> It doesn't because no append-only log does. Then it doesn't solve the problem people in the space have. I understand the visceral hate for web3, I do, but it really gets in the way of talking about a very bland technical issue. I think you're misunderstanding what this service is. It's about access to Blockchain data not about running a chain. > A view of the final state of data is literally one of the things they…

This is what the website literally claims: === start quote === Web2 Databases were meant to deal with datasets that change frequently. They have to provide ACID guarantees to ensure correctness. The solution to this problem becomes more complex with scale. Distributed transactions, sharding, replication, and fault tolerance are notoriously slow. Web2 databases are just NOT designed for web3 A system designed to take…

I'm only going to reply this last time as I come to HN to get away from reddit style back and forth arguments.

They don't say that standard databases can't handle this. They say that they can do it cheaper and faster without them, because you don't need a bunch of the features. They are literally telling you that their solution is simpler not revolutionary.

You are conflating two very different things, the production and indexing - and then consumers.

> - Blockchain isn't the only thing to have an immutable log. Apache Kafka is one, and is being used at scale that dwarfs anything in crypto space

Sure, and doesn't solve the problem here as you have already told me. Kafka would be ridiculous overkill IMO for the part it would play. The problem is simpler than kafka is designed for, so it makes sense that someone could make a simpler tool to handle it.

> Even though, again, traditional systems can probably handle all this on a free tier of any cloud hosting provider

That very much depends on your users - it's this kind of statement that makes me think you're not picturing what this actually is.

> slap a traditional (I'd say conservative) JSON-RPC API on top,

Yes, this exact API is very much a requirement for this kind of service.

As a final thought, if you think you can throw this together for a large customer base easily and host it basically for free while having better performance - go for it. You'll make money because the alternative options are expensive.

Re: Show HN: 0xFast – Faster Web3 APIs

#70

Post by dmitriid »

IanCal wrote: post #69
dmitriid wrote: post #68

Earlier quoted context omitted.

This is what the website literally claims: === start quote === Web2 Databases were meant to deal with datasets that change frequently. They have to provide ACID guarantees to ensure correctness. The solution to this problem becomes more complex with scale. Distributed transactions, sharding, replication, and fault tolerance are notoriously slow. Web2 databases are just NOT designed for web3 A system designed to take…

I'm only going to reply this last time as I come to HN to get away from reddit style back and forth arguments. They don't say that standard databases can't handle this. They say that they can do it cheaper and faster without them, because you don't need a bunch of the features . They are literally telling you that their solution is simpler not revolutionary . You are conflating two very different things, the producti…

Again you're responding to what you imagine I was writing even when I provided direct quotes from both the website and you.

But since this was your last reply, adieu.

Post reply on HN