171–180 of 189 posts
Earlier quoted context omitted.
Agreed. How can a media file sharing app possibly saturate Sqlite's write limit? I would use an app-level global lock on all writes to Sqlite.
Probably during scanning libraries? They read hundreds of files and for each of them look for metadata in the internet like discogs and similar. So sure if implemented as async in c# you could run into this issue.
Earlier quoted context omitted.
I share my Jellyfin with about a dozen people, and it's not weird to have several people streaming at the same time. I have a two gigabit connection so bandwidth isn't generally an issue, but I've had issues when three people all streaming a VC-1 encoded video to H264 in software. This is something that I think I could fairly easily ameliorate if I could simply load-balance the application server by user, but histori…
> I've had issues when three people all streaming a VC-1 encoded video to H264 in software. I don't quite get the "in software" part. I assume you mean that the video needs to be transcoded to h.264 on your server for their client to play it. The way I mostly solved this is to ask people to install and use the native app (jellyfin-media-player or Android app) whenever possible, as it is compatible with more codecs. Y…
I'll look into the distributed ffmpeg.
Earlier quoted context omitted.
Exactly, there are use cases where SQLite makes sense but you also want to make it faster. I really don't get why there isn't a more portable Postgres.
There is, you can even run PG under wasm if you are desperate. :) SQLite is probably the better option here and in most places where you want portability though.
Earlier quoted context omitted.
With Go it's quite straightforward actually: use WAL mode + two connection pools, one for reads and the other, with MaxConnections set to 1, for writes. This way you should never encounter any concurrency issues, and Go will serialise writes for you too
I have no experience in Go, but won't that be a full database lock and prevent transactions with more than one operation?
Note that the application needs to be aware of that there are two pools — one for write operations and one for reads (the latter with no or high connection limit). The separation can be ensured on SQLite level too by adding ?_query_only=1 to connection parameters or setting the respective pragmas in the read-only pool.
Earlier quoted context omitted.
That's much more likely flash degradation than actual fragmentation. Did you use cheap tablets with eMMC storage?
We had that thought too. I'll have to try dig out what the tablets were to find out exactly what type - this would have been 3 or 4 years ago now. We sort of ruled that out because: The other workaround to get a speed boost was the user to uninstall and reinstall the app (and then wait for all the data to download again) but that didn't fly because the users would delete before they'd synced off all their data and th…
Earlier quoted context omitted.
> One of the biggest contributors I've had in the past for SQLite blocking was disk fragmentation. Is that even still a thing? I thought modern filesystems like ext4 were supposed to be largely immune to that.
Ext4 isn't used on Android, and it isn't immune to fragmentation. The way ext4 reduces fragmentation is with some basic heuristics: mainly, it spreads files across the full disk instead of finding the next free spot. So they have room to grow without fragmenting. When the space gets low, it fragments just as badly as older file systems unfortunately.
Earlier quoted context omitted.
Jellyfin isn’t meant to be some highly available distributed system, so of course this happens when you try to operate it like one. The typical user is not someone trying to run it via K8s.
Yeah, I agree, though making software that can run in a distributed configuration is a matter of following a few basic principles, and would be far less work than what the developers have spent chasing down trying to make SQLite work for their application. The effort required to put an application on Kubernetes is a pretty good indicator of software quality. In other words, I can have a pretty good idea about how dif…
[1] https://jellyfin.org/posts/efcore-refactoring [2] https://jellyfin.org/posts/jellyfin-release-10.11.0/
> So an application that wants to use SQLite as its database needs to be the only one accessing it. No. It uses OS level locks. fcntl(). You can access it from how many ever processes. The only rule is, single writer (at a time). > When another part of the application wants to read data, it reads from the actual database, then scans the WAL for modifications and applies them on the fly. Also wrong. WAL does not conta…
C# devs*
Does this mean I can finally load-balance with multiple Jellyfin instances? A million years ago, back when I still used Emby, I was annoyed that I couldn't use it across multiple in Docker Swarm due to locking of SQLite. It really annoyed me, enough to where I started (but never completed) a driver to change the DB to postgres [1]. I ended up moving everything over to a single server, which is mostly fine unless I ha…
Jellyfin have just gone through a massive refactor and pulled all their data access code into EFCore. This opens the path for supporting different RBDMSs which think is next on their list.