Live data from Hacker News

Show HN: 1 Million Rows

1mrows.pages.dev

11–20 of 44 posts

Re: Show HN: 1 Million Rows

#11
post #9

Nice one! Coincidentally I worked on a large table renderer too this weekend: https://github.com/markwylde/react-massive-table I noticed you didn't quite get to a million rows. For me, it cut off at 671088. Same thing happened when I built my one. I came across the same thing. In the end I just manual made the rows appear at their absolute position. Seemed to work well.

What browser are you using? Some browsers cut off early due to scroll limitations. I could get Firefox to about 300,000

Re: Show HN: 1 Million Rows

#15

Not 100% sure what I'm looking at here? Am I missing something or is it just table of data with an "infinite scroll" that loads 200 records at a time?

Loading more than a few thousand rows on a web page will make unusably slow. Especially when you add a lot more features to it.

This is such that only what's seen or about to be seen is put on the page. The rest is kept ready on the server on local memory depending on what the user is doing.

This allows for a scalable solution that allows you to view thousands of records and interact with them

Re: Show HN: 1 Million Rows

#17
When I started working on Baserow (this seems similar based on the roadmap), a couple of years ago, I thought it would be a big challenge to quickly render a million rows in the browser. Introducing a system that fetches a page of rows based on the scroll offset, and with a small debounce did the trick. We only had a couple of field types, and it was all incredibly fast

The thing that make performance complicated for a no-code database is when you have 30 interconnected tables, some tables with 200 fields, containing many formulas or other computed fields like lookups or rollups. Updating a single cell, can result in thousands of other rows that must be updated across different tables. If there are 30 users making constant changes, locking PostgreSQL rows under the hood while the formulas are recalculated, and then a couple of n8n workflows making a many API requests to those tables, that's when things get interesting. Especially in combination with features like webhooks, real-time updates, 100+ filters, grouping, 26 field types, date dependencies, aggregations, importing/exporting whole databases.

When implementing a new feature, I've heard users say that's not complicated because it's just adding a checkbox. Making to run it at scale and keeping things performant is what's making it complicated.

Post reply on HN