101–110 of 148 posts
Earlier quoted context omitted.
Who exactly is defining triviality purely in terms of request rate?
I think most. You don't need a database to do 1 request a day, for instance.
Earlier quoted context omitted.
The problem is that one at a time times 100,000 DBs takes a long time. For every second a migration takes its 1.1 days. You can parallelize it, of course, but that's a challenge in of itself.
> You can parallelize it, of course, but that's a challenge in of itself. Not a challenge at all really.
It's so insane that he links to a whole another post to clarify what he means by "the edge", and he struggles to state that in the first 5 paragraphs of it.
I would say "low computing power" but a raspberry pi is 4Ghz these days.
SQLite is great, but it is not great for the edge. Handling conflicts and replication is just too complex for SQL based systems.
I am looking at moving us towards 1 gigantic SQL Server Hyperscale DB for most things. Force our clients into a PaaS solution over time, enforce one standardized product model, etc.
Where you probably don't want to be is somewhere in the middle where half your data lives in some centralized place and the other half is in your edge/client nodes. Synchronizing across these domains can rapidly become a nightmare.
Earlier quoted context omitted.
Good question. As a non-racer who enjoys watching racing, I'd imagine at least some accelerometers/gyros and load cells if they got them, and possibly Pitot tubes or similar for measuring aerodynamics when testing, to get a detailed understanding on the forces on the car and how it behaves. Motor currents probably up there too, to catch current spikes. Curious to see how far off I am :)
> detailed understanding on the forces on the car and how it behaves But at 4000 samples per second? You'd think twenty would be plenty. NyQuist: 4000 samples per second means we are looking to get information about an up to 1800 Hz signal. A6 on the piano keyboard (excluding harmonics). Sorry, I mean Nyquist. I was confused by NyQuil, the night time flu medicine.
Earlier quoted context omitted.
Good question. As a non-racer who enjoys watching racing, I'd imagine at least some accelerometers/gyros and load cells if they got them, and possibly Pitot tubes or similar for measuring aerodynamics when testing, to get a detailed understanding on the forces on the car and how it behaves. Motor currents probably up there too, to catch current spikes. Curious to see how far off I am :)
> detailed understanding on the forces on the car and how it behaves But at 4000 samples per second? You'd think twenty would be plenty. NyQuist: 4000 samples per second means we are looking to get information about an up to 1800 Hz signal. A6 on the piano keyboard (excluding harmonics). Sorry, I mean Nyquist. I was confused by NyQuil, the night time flu medicine.
Doesn't seem unreasonable to want to sample at that rate, especially for tweaking suspension and such.
I extensively used SQLite in a telemetry system for an electric race car. The car has an onboard computer, first a Raspberry Pi then a dual core Arm processor. Onboard code logs ~4000 messages a second into three SQLite databases. After a drive session a script merges the three databases into a single SQLite session log. The session log is decoded on a different computer to ~400 columns of time series data again stor…
Other have asked, but I'd also be very interested in knowing more if you can share more details. I had to spent a non trivial amount of time convincing others to use SQLite instead of dumb files in similar situations, stories like yours are pure gold.
I was really pleased to add it to my toolkit.
> It’s borderline impossible to compare it against networked database management systems like MySQL or Postgres, because SQLite is a library that operates on a local file — it bypasses all the costs incurred by the network, layers of serialization and deserialization, authentication, authorization, and more. Postgres can run locally, communicating via a Unix socket. You should try benchmarking this before stating tha…
Slight tangent: Is a UNIX socket faster than TCP over loopback?
> It’s borderline impossible to compare it against networked database management systems like MySQL or Postgres, because SQLite is a library that operates on a local file — it bypasses all the costs incurred by the network, layers of serialization and deserialization, authentication, authorization, and more. Postgres can run locally, communicating via a Unix socket. You should try benchmarking this before stating tha…
Slight tangent: Is a UNIX socket faster than TCP over loopback?
Earlier quoted context omitted.
The maximum size will increase to at least 1GB in the near future, enabled by our recent rewrite of the underlying storage layer. Perhaps we can push it further, even, we'll see. But yes, I think the next step is then some sort of sharding. Sharding by user would be an obvious approach for many apps. I think we should build a framework to help manage this, so apps would only need to provide some callbacks e.g. to com…
do you think you will have any API driven access to the data held in these services? IE, I don't per se want to have to use a worker to arbitrage between the Cloudflare platform and HTTP access to the data within Cloudflare systems. Much like R2 has an S3 compat API as well as a way to retrieve files outside of workers (IIRC) will this be true of all forms of storage on the platform? Sometimes it'd be nice to prime t…
For D1, yes — on the roadmap is a native HTTP API. KV has a HTTP API as well, so you can write directly.