Would this work with the Boehm Garbage Collector given the "pointer to the middle of the object" trick?
Simple Dynamic Strings library for C, compatible with null-terminated strings
21–30 of 85 posts
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#22UTF support?
Exactly. You should not name a buffer lib "string", when it does not support the basic unicode operations: case fold, normalize => compare, search. In utf-8 of course. I'm also missing stack allocation support, needed for fast short strings. It should be even included in sdsnew, for len < 128.
Unfortunately, yes. Limited usefulness at best without unicode support, at least to a degree. Even UTF-16 or 32 internally would suffice, treating UTF-8 only as ser/de format is good enough these days.
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#23Just looking at the API "sds" seems to just be a typedef for char* - unfortunately, that means that accidentally passing a char* as an sds into any of the functions will be instant UB and not even a compiler warning. Considering this is C, there is no way to prevent this easily since you can't express a type that is one-way convertible (i.e. sds -> char* ok, char* -> sds not ok). I do have to wonder though if avoidin…
It is not a real issue in modern days. You just enable address sanitizer in all your debug builds and these issues are obvious at runtime (still not perfect, but an improvement from dark days).
Address sanitizer can only catch errors on branches your debug build exercises.
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#24Hi, author here. May make sense to make SDS in perspective given a few comments I'm reading here.
1. Yep, more than "strings" SDS may be consider a library for dynamic buffers, especially from people coming from C++ or higher level languages. However I think that for C, it makes sense to provide a very low level thing like that.
2. In practice, if you see how SDS is used (extensively) inside Redis, it normally models things where you would do realloc magics, pointer math, and so forth: from client output buffers, to actual strings to accumulate error messages, to the Redis String object itself.
3. Probably an UTF-32 layer could be implemented on top of that, but I would think the two modules completely separated from the POV of the implementation. Like having an additional utf32.c file that provides additional interfaces on top of SDS.
4. The peculiar approach used by SDS header-before-pointer creates some problem with Valgrind and similar tools, however Valgrind will just report "possibly lost", that are messages mostly safe to ignore. However the advantage of using SDS strings directly with C function libraries is very handy.
SDS strings are not perfect and tend to be extremely optimized for Redis, if you look at the API, they allow to do things that are very low level, like pre-allocating the internal buffers to improve performances when you know you are going to read a big chunk of data from a socket or alike. However while imperfect and very tuned, these kind of libraries show how much you can easily improve C, with little work, and how many unsafe things in C are about lack of abstractions.
The value of SDS is "less is more" in the simple API they provide that pretends that SDSs are just plain-strings++. You can see this in a few features: plain C pointers interface, and the policy of always terminate the string. They need more love anyway, and to be less specialized for Redis, adding more useful APIs. Maybe at some point I'll find the time.
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#25Just looking at the API "sds" seems to just be a typedef for char* - unfortunately, that means that accidentally passing a char* as an sds into any of the functions will be instant UB and not even a compiler warning. Considering this is C, there is no way to prevent this easily since you can't express a type that is one-way convertible (i.e. sds -> char* ok, char* -> sds not ok). I do have to wonder though if avoidin…
What you do is something like the following:
typedef struct sds_s {
char data[0];
} sds;
char* sds_cstr(sds *str);
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#26I rarely work with C but when I do I am always reminded at how painful it is to use strings. Is this a complete drop-in replacement?? If so it could occasionally make my life much easier!
I don't find it painful at all. You need to think about the lifetime of your strings. The lifetime is usually convenient to split into two (consecutive) phases: the first phase is when the string gets constructed / modified, and the second phase is the immutable phase up until the string is released again. You'll often have separate string types for these phases: A string builder type for the mutable phase, and a simpler immutable type for the immutable phase.
Often the former will need just a single shared allocation (per thread). That works if you're only ever constructing a single string at a time.
Often the latter won't be dealing with memory management at all. You will often represent it by a plain char pointer and optionally a length field. Management-wise, at most you will need to construct it initially by making an immutable string from the mutable one, and a call to a release function when the string is no longer needed.
The fact that the string isn't modified in the immutable phase makes it trivial to pass it around as an ordinary char pointer (plus optional length), which is as convenient as it gets.
In rare cases you might want to make a hashmap for string interning. I've used interning in a past compiler project, where it was used to collapse identifier strings.
That will probably cover more than 95% of all your string needs. And it's very easy to be vastly more efficient than any from-the-shelf GC'ed string type with this. All you need to invest is to write a little near-boilerplatey code, but it's far from "hard".
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#27Would this work with the Boehm Garbage Collector given the "pointer to the middle of the object" trick?
The bdwgc has a #define to set whether or not it should check for interior pointers.
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#28Just looking at the API "sds" seems to just be a typedef for char* - unfortunately, that means that accidentally passing a char* as an sds into any of the functions will be instant UB and not even a compiler warning. Considering this is C, there is no way to prevent this easily since you can't express a type that is one-way convertible (i.e. sds -> char* ok, char* -> sds not ok). I do have to wonder though if avoidin…
> Considering this is C, there is no way to prevent this easily since you can't express a type that is one-way convertible (i.e. sds -> char* ok, char* -> sds not ok).
typedef struct {
char *s;
} sds_t;
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#29There is also Better String Library, similar self-contained library with C-style string compatibility: http://bstring.sourceforge.net/
TFA specifically names what is different between SDS and libraries like bstring:
> Normally dynamic string libraries for C are implemented using a structure that defines the string. The structure has a pointer field that is managed by the string function, so it looks like this:
struct yourAverageStringLibrary {
char *buf;
size_t len;
... possibly more fields here ...
};
Re: Simple Dynamic Strings library for C, compatible with null-terminated strings
#30UTF support?
Should just use the ICU string library. You get utf support and much more that is needed for modern string usage.
https://unicode-org.github.io/icu-docs/apidoc/released/icu4c...