Live data from Hacker News

We don't need a string type (2013)

mortoray.com

61–70 of 70 posts

Re: We don't need a string type (2013)

#61

Earlier quoted context omitted.

Yes, Julia really lets one get wild with Unicode. There are certain classes of unicode characters that we have marked as invalid for identifiers, some which are used for infix operators, and some which count as modifiers on previously typed characters which is useful for creating new infix operators, e.g. one might define julia> +²(x, y) = x^2 + y^2 +² (generic function with 1 method) such that julia> -2 +² 3 13 If s…

Nice ! Python allows to define operators too, but AFAIK you can't use Unicode in those ? And ² (or any other sub/superscript number - at least some letters are fine) is not allowed in identifiers either. The point is to get closer to math notation though, if anything x +² y is IMHO even farther away than (x + y)*2 ! Any way to have (x + y)² or √(x + y) to work ? –––– The new AZERTY has a lot of improvements : ∞, ±, ≠…

> if anything x +² y is IMHO even farther away than (x + y) * 2 !

Yeah, it was just a random example that came to mind, not to be taken seriously. Here's perhaps one example of unicode being used in a way that's pleasing to some and upsetting to others: https://www.reddit.com/r/programminghorror/comments/jqdi4i/y...

> Any way to have (x + y)² or √(x + y) to work ?

The sqrt one works out of the box actually, no new definitions required:

    julia> √(1 + 3)
    2.0
The second one does not work because we specifically ban identifiers from starting with superscript or subscript numbers. If it was allowed, we could work some black magic with juxtaposition to make it work.

Here's an example with the transpose of an array:

    julia> struct ᵀ end

    julia> Base.:(*)(x, ::Type{ᵀ}) = transpose(x)

    julia> [1, 2, 3, 4]ᵀ
    1×4 transpose(::Vector{Int64}) with eltype Int64:
     1  2  3  4

Basically, we have a system called 'juxtaposition' where 2x is parsed as 2*x (but not x2). It generalizes in funky ways one can abuse if they really want (kinda discouraged though)

Re: We don't need a string type (2013)

#62

Curious. I have to come to exactly the opposite conclusion — that we should drop the idea of a fixed-length character type, and instead _only_ have (Unicode) string types. Actually, I'd prefer something like `std::text` to finally be free of the baggage of "string". Operations on text should work on logical text concepts. For example, something like `someText.firstCharacter()` would have a return type of `text`, with…

My future perfect programming language will have explicit native types for strings: ansi, utf8, utf16, and unicode.

The nth of any array works as expected for that type. Convert to bytes as needed.

  ansi "abcd"[1] -> byte
  utf8 "abcd"[1] -> char
  utf16 "abcd"[1] -> char
  utf8 "abcd".toBytes()[1] -> byte
  unicode "abcd"[1] -> word
Were it still the 90s, I'd probably care about locales, so somehow imbue ansi arrays with that metadata.

Re: We don't need a string type (2013)

#63
post #36
post #20

Earlier quoted context omitted.

It is not clear to me if you're elaborating or think you're disagreeing, but that is what Go does. It is generally assumed in Go that strings are UTF-8, but in practice what they actually are are just bags of bytes. Nothing really "UTF-y" will happen to them until you directly call UTF functions on them, which may produce new strings. It's something that I don't think could work unless your language is as recent as G…

I'm saying it's useful to have valid strings and paths as separate types, but Go conflates the two types. Conflating the two is likely to lead to confused usage (such as programmers assuming there's a bijective mapping between valid paths and valid sequences of Unicode codepoints.) Pervasive confused usage of this sort in the wild in Python 2 was the motivation behind splitting bytes and strings in Python 3.

As you pointed out, path types are awfully specialized to the OS and really even the file system itself. It is not clear that "Go" could provide such a thing. It doesn't need to, really, you can relatively easily create a type for the specific case you have.

    type PathSegment struct {
        path string // not exported, so only the empty one can be created externally
    }

    func MakePath(in string) (PathSegment, error) {
        // validate the input here
    }
You'll need some more supporting types, of course, but it doesn't have to be provided by "Go" itself. (I have something rather like this in my codebase, though it is specialized to just Unix paths since I have no need to care about all the cross-platform details in this code base.)

I wouldn't expect this to be something the language itself provides, and I'm not even that worried about it being missing from the standard library because it's awfully detail-oriented even for that.

Re: We don't need a string type (2013)

#64

Earlier quoted context omitted.

Really, they don't run into string/character issues regularly ? Because I do...

I certainly run into them rarely, and if I do have an issue it is usually solved by bunging it into some purpose built standard or third party library and calling it a day. I’m sure people have jobs that deal with this, but the low-level form of the problem is not something that I could see one encountering in a meaningful way for building a standard CRUD app or service.

I completely agree with you, but everyone who doesn't have to deal with unicode/strings on a regular basis should consider themselves lucky.

Once your add RTL text (with the matching bidi algorithm) or grapheme-based written system such as Devanagari which doesn't really have characters at all it becomes such a mess so fast.

Re: We don't need a string type (2013)

#65
I think that the problem with text is that the basic operation you want to do is inserts. The way memory works in computer makes that an inherently inefficient operation. I'm a bit fascinated by how bad computer are at text give that that is what we use so much of them for.

As a C programmer I think that its not really possible to implement an efficient text processing library, because there is no good universal way to store text. So much depends on the pattern of the processing functions. If you want to avoid allocating new memory and moving a lot of text for each operation, the implementation needs to make speculative choices about how text can best be stored. How you store text depends so much on your access pattern. Do you need to be able to get to a line fast? or know how long the text is? Or insert something? and if so how much?

A C style string would for instance be terrible for something like a text editor, because every key press would cause a complete copy of the document to have to be allocated, and then copied over. So maybe a linked list? But you dont want just one character in each link because that trashes the cache right? but then its still slow to just skip forward fast, so maybe an array of pointers to snipets? or maybe a linked list of pointers to snippets? So many possibilities that all impact performance differently depending on what you do with it.

When I see higher languages with nice easy to use string functionality, I always consider, the impossible choices that had to be made under the hood.

Re: We don't need a string type (2013)

#66

I think that the problem with text is that the basic operation you want to do is inserts. The way memory works in computer makes that an inherently inefficient operation. I'm a bit fascinated by how bad computer are at text give that that is what we use so much of them for. As a C programmer I think that its not really possible to implement an efficient text processing library, because there is no good universal way…

I think you want a "gap buffer".

Re: We don't need a string type (2013)

#67
post #66

I think that the problem with text is that the basic operation you want to do is inserts. The way memory works in computer makes that an inherently inefficient operation. I'm a bit fascinated by how bad computer are at text give that that is what we use so much of them for. As a C programmer I think that its not really possible to implement an efficient text processing library, because there is no good universal way…

I think you want a "gap buffer".

A gap buffer is an example of a data structure for text that is optimized for one usage pattern, and performs badly with other patterns. Generalized text structures are hard.

Re: We don't need a string type (2013)

#68
post #29

Earlier quoted context omitted.

Posix thinks paths are strings. See https://pubs.opengroup.org/onlinepubs/009695399/functions/op...

"String" has multiple meanings in this context. In the context of that manpage, it means "nul-terminated array of char" which is the C language meaning. In the context of what you're replying to, a "string" is a sequence of bytes (octets) in a specific Unicode Transformation Format. Those are very different things when it comes to programmatic manipulation of those things.

What you can do with a "string" that you can't do with a C string?

Re: We don't need a string type (2013)

#69
post #29

Earlier quoted context omitted.

"String" has multiple meanings in this context. In the context of that manpage, it means "nul-terminated array of char" which is the C language meaning. In the context of what you're replying to, a "string" is a sequence of bytes (octets) in a specific Unicode Transformation Format. Those are very different things when it comes to programmatic manipulation of those things.

What you can do with a "string" that you can't do with a C string?

From on-screen to in-memory representation, we go from glyphs to grapheme clusters, to unicode 'characters', to codepoints, to encoded bytes. None of these steps are bijections (ligatures, multi-character graphemes, invalid characters, encoding errors).

I'd argue a 'proper' string type should operate at the grapheme cluster and/or character level and take care of things like normalization (eg for string comparisons) and validation.

Re: We don't need a string type (2013)

#70
post #4

The article is an argument against types, in general. The point that characters can be stored in other containers is meaningless: the question is whether, conceptually, a specific sequence of character values distinct from another sequence has compile-time meaning. It does. Therefore, it needs a type. Such a sequence has numerous special characteristics. In particular, element at [i] often has an essential connection…

You can mess up any ordered sequence in this way.

That is an argument for making and using ordered-sequence types, not an argument against a string type.

Generic ordered-sequence containers have not appeared in standard libraries, except where the container itself depends on the ordering, for various practical and historical reasons, but are very useful to wrap (say) a vector instantiated on a particular type, often with some metadata stuck on.

Post reply on HN