Nul Characters in Strings in SQLite
sqlite.org
Nul Characters in Strings in SQLite
1–10 of 22 posts
Re: Nul Characters in Strings in SQLite
#2>* The length() SQL function only counts characters up to and excluding the first NUL.*
>The quote() SQL function only shows characters up to and excluding the first NUL.
>The .dump command in the CLI omits the first NUL character and all subsequent text in the SQL output that it generates. In fact, the CLI omits everything past the first NUL character in all contexts.
That's just all kinds of "oh no", wow.
I mean, I can't come up with a better strategy, but... oof. C-style strings being a thing at all really hurts.
Re: Nul Characters in Strings in SQLite
#3>* The length() SQL function only counts characters up to and excluding the first NUL.* > The quote() SQL function only shows characters up to and excluding the first NUL. > The .dump command in the CLI omits the first NUL character and all subsequent text in the SQL output that it generates. In fact, the CLI omits everything past the first NUL character in all contexts. That's just all kinds of "oh no", wow. I mean,…
It's not really a big problem. You should be storing arbitrary binary strings as blobs (and length() works properly with them), but the underlying encoding is nearly identical: https://www.sqlite.org/fileformat.html#record_format
Re: Nul Characters in Strings in SQLite
#4As long as we're supporting in-band signals in strings, how about making DEL rub out the previous character?
Re: Nul Characters in Strings in SQLite
#5>* The length() SQL function only counts characters up to and excluding the first NUL.* > The quote() SQL function only shows characters up to and excluding the first NUL. > The .dump command in the CLI omits the first NUL character and all subsequent text in the SQL output that it generates. In fact, the CLI omits everything past the first NUL character in all contexts. That's just all kinds of "oh no", wow. I mean,…
The better strategy is that it should just work, isn't it?
Re: Nul Characters in Strings in SQLite
#6As long as we're supporting in-band signals in strings, how about making DEL rub out the previous character?
Add forward-cursor and back-cursor, and maybe we can embed a Turing machine in a string.
Re: Nul Characters in Strings in SQLite
#7>* The length() SQL function only counts characters up to and excluding the first NUL.* > The quote() SQL function only shows characters up to and excluding the first NUL. > The .dump command in the CLI omits the first NUL character and all subsequent text in the SQL output that it generates. In fact, the CLI omits everything past the first NUL character in all contexts. That's just all kinds of "oh no", wow. I mean,…
It's not really a big problem. You should be storing arbitrary binary strings as blobs (and length() works properly with them), but the underlying encoding is nearly identical: https://www.sqlite.org/fileformat.html#record_format
Then DBMS should not allow to save a string that would make some function return wrong results. Either truncate with a warning on INSERT (MySQL approach) or better throw an error.
Re: Nul Characters in Strings in SQLite
#8So, one fun consequence of this is that Unicode multi-byte strings (not UTF-8 but something like UTF-32) cannot be stored as strings in sqlite without a huge pain. Not that I ever planned to use multi-byte fixed length encodings, but good to know!
A good moment to appreciate the elegance of UTF-8 which allowed to encode multi-byte characters preserving the semantics of C strings.
Re: Nul Characters in Strings in SQLite
#9>* The length() SQL function only counts characters up to and excluding the first NUL.* > The quote() SQL function only shows characters up to and excluding the first NUL. > The .dump command in the CLI omits the first NUL character and all subsequent text in the SQL output that it generates. In fact, the CLI omits everything past the first NUL character in all contexts. That's just all kinds of "oh no", wow. I mean,…
> C-style strings being a thing at all really hurts.
Well, C strings exist and are probably here to stay.
I believe Pascal strings, where the length is stored in the 0th character, were much worse (also, obligatory reference to Joel Spolsky's “Back to Basics” and “fucked strings” [1]).
Looks like we only got enough memory to spare to allow arbitrary length strings with arbitrary characters – that is, being able to use 2 or 4 bytes for the length of every string – in 1990s or so, when C++ strings and similar types became popular.
[1]: https://www.joelonsoftware.com/2001/12/11/back-to-basics/
Re: Nul Characters in Strings in SQLite
#10>* The length() SQL function only counts characters up to and excluding the first NUL.* > The quote() SQL function only shows characters up to and excluding the first NUL. > The .dump command in the CLI omits the first NUL character and all subsequent text in the SQL output that it generates. In fact, the CLI omits everything past the first NUL character in all contexts. That's just all kinds of "oh no", wow. I mean,…
It's not really a big problem. You should be storing arbitrary binary strings as blobs (and length() works properly with them), but the underlying encoding is nearly identical: https://www.sqlite.org/fileformat.html#record_format
Note that this breaks a lot of core functionality unless you CAST(s as TEXT) before you use it, eg iirc LIKE won’t work. Then you’re back to the same problem.