Are these websites just completely unable to handle a 1000 simultaneous requests?
This is more in the range of 10s of requests per second at this position on HN
Database of Databases
11–20 of 65 posts
Re: Database of Databases
#12They're using SQLite which is single threaded and single user by design. So this website will not be able to service that much traffic.
Should be fine for reads. Sqlite.org is dynamic, pulling from sqlite data for ~20% of the pages, and it does fine with HN piling on. The single threaded would be an issue for writes, but I don't see why they would be doing writes for a page view. See https://www.sqlite.org/whentouse.html
Also, caching layers.
Presumably a service uses SQLite to simplify their ops. So long as the caching layer is equally simple to maintain, then SQLite continues to make sense to me
Re: Database of Databases
#13Thought it was some groundbreaking new database technology....
Speaking of,
What would an interesting version of "database of databases" look like?
Re: Database of Databases
#14Earlier quoted context omitted.
Should be fine for reads. Sqlite.org is dynamic, pulling from sqlite data for ~20% of the pages, and it does fine with HN piling on. The single threaded would be an issue for writes, but I don't see why they would be doing writes for a page view. See https://www.sqlite.org/whentouse.html
Also, caching layers. Presumably a service uses SQLite to simplify their ops. So long as the caching layer is equally simple to maintain, then SQLite continues to make sense to me
Their repo shows the "DummyCache" (no caching) as the default for the Python/Django setup: https://github.com/cmu-db/dbdb.io/blob/master/dbdb/settings....
I wonder if that's how it is in production.
Re: Database of Databases
#15Thought it was some groundbreaking new database technology.... Speaking of, What would an interesting version of "database of databases" look like?
The closest I can think of would be the "Data Governance" space. Tools like Collibra: https://www.collibra.com/
Screenshot that makes it easier to understand: https://www.collibra.com/wp-content/uploads/Blog-DataLineage...
Re: Database of Databases
#16This article reminded me of this site:
Re: Database of Databases
#17Any good learning resources for implementing your own DBMS, and DB internals? (not DB theory, not SQL, not intro).
Re: Database of Databases
#18Also reminded me of the awesome list of awesome lists:
Re: Database of Databases
#19Earlier quoted context omitted.
Also, caching layers. Presumably a service uses SQLite to simplify their ops. So long as the caching layer is equally simple to maintain, then SQLite continues to make sense to me
Their repo shows the "DummyCache" (no caching) as the default for the Python/Django setup: https://github.com/cmu-db/dbdb.io/blob/master/dbdb/settings.... I wonder if that's how it is in production.
For a mostly static site you would want to cache "over" Django, probably in a caching proxy or CDN. Of course there are a lot of details, such as many CDNs will always go to the origin on a edge miss instead of locating another copy in the CDN. Of course running a caching nginx on the same box as the Django is probably way more performant than caching "under" Django
Re: Database of Databases
#20Any good learning resources for implementing your own DBMS, and DB internals? (not DB theory, not SQL, not intro).
You might find useful guidance here: http://www.aosabook.org/en/