Live data from Hacker News

TXR – A Programming Language for Convenient Data Munging

nongnu.org

51–60 of 74 posts

Re: TXR – A Programming Language for Convenient Data Munging

#51
post #49

Earlier quoted context omitted.

I don't want to reopen this 50+ years old can of worms, but I have 2 questions: 1. Is it about homoiconicity in general or specifically s-expressions [EDIT: I see GP writes about s-exps specifically, missed it at first]? Prolog, Erlang, TCL, and Rebol (just some examples) are homoiconic, but not s-exps based. What do you think about them? 2. Do you often read code without syntax highlighting and proper indentation? A…

1. It's homoiconicity in general. It's most obvious in arithmetic expressions, but homoiconicity (as far as I've seen) eschews semantic indicators besides function name and argument position. I find that keywords and symbols are almost indispensable for smooth eye-parsing of code. 2. No, I use syntax highlighting and indentation all the time. Trying to put words to my vague thoughts, I think the main issue is that, f…

> I find that keywords and symbols are almost indispensable for smooth eye-parsing of code.

Sure, but that's a totally separate issue from homoiconicity. All the examples I mentioned are infix languages (although TCL requires you to opt-in for that):

https://github.com/adambard/learnxinyminutes-docs/blame/mast... (I couldn't find Rebol, but Red is very similar)

https://github.com/adambard/learnxinyminutes-docs/blame/mast...

https://github.com/adambard/learnxinyminutes-docs/blame/mast...

https://github.com/adambard/learnxinyminutes-docs/blame/mast...

(sorry for the links to blames, but it's impossible to link to a specific line otherwise)

Prolog lets you define your own operators and you have control over associativity and precedence. TCL `expr` works like `$(( 2 + 3 ))` in BASH (which also doesn't support mathematical operators normally) and by reimplementing it you can also add your own operators. Erlang also allows you to define new operators, although it's much more hassle there, as you need to write a parse transform (which is only possible to do so easily because of homoiconicity of the language). I'm not sure about Rebol/Red - I had only passing contact with it long ago.

In other words, homoiconicity itself has little to do with where the "keywords and symbols" are placed in the code, or whether they are used at all.

Accidentally, Lisps also support infix syntax just like TCL: for most Lisps you can grab a lib/package/source of the `infix` macro, which allows you to write infix arithmetic (and code in general) without problems.

> I think the main issue is that, for example, in a let block the only indication of the meaning of all the items is the single function/macro name at the beginning of the block. Whereas in an algol-type language you have assignment operators in each item to indicate what they mean.

Ok, that's one way of looking at this. On the other hand, one can also say that the repeated use of the operator where the meaning of each line is already determined (by the fact that it's inside `let` block) is useless cruft, which actually hinders comprehension by forcing you to mentally parse one element more on every single line of the assignment block. In Lisp, it's actually trivial to extend the language to include this form of `let`:

  (let
      ((a = some_expr)         ;; could be `:=` `
In Scheme, the fundamental conditional construct called `cond` even has an operator-like construct built-in

    ;; in Racket
    (cond
      [some_condition => a_function_called_on_the_result_of_condition_expr_if_its_not_false])
Yet, including this kind of additional syntax is not widespread in the community. Unless the additional syntactic element actually changes the meaning of the code, like in `cond` case, it is seen as superfluous and not needed.

Both opinions have some merit to them. How you feel about both styles is largely determined by what you're familiar with and what you're used to. In the first style, you have to learn to ignore some token in some places as they don't add any meaning to the code. In the second, you need to be careful not to mistake one kind of block for another, and you need to learn what is the relation between subexpressions in each kind of block.

Both approaches require learning. The difference is that you already learned the first one, while you're not familiar with the other. But - and that's what I really wanted to say - there's no difference between readability of the two approaches once you learn them. In other words, Lisp code is as readable for Lisp programmers as JavaScript is for JS programmers. Further, all Lisps are equally readable to (a specific) Lisp programmers, just like all Algol-like languages are readable for JS programmers.

With a bit - and I mean a bit, like a few days; if you want a language you won't find readable after half a year, go for J - of practice you could read Lisp-like code without problems. There's no inherent unreadability to either Lisp- or Algol-like syntaxes - is what I'd like to convince you to :)

Re: TXR – A Programming Language for Convenient Data Munging

#52
post #40

Earlier quoted context omitted.

I know that Python and Javascript have their warts (as do all languages in my experience), but what decisions in particular are you thinking of?

Lambdas are limited to a single expression. Threads don't work the way anyone would ever want threads to work because o the GIL. That said, homoiconicity (s-expressions) is not a feature I want. It makes code like reading a wall of text compared to a nicely laid out magazine.

Racket also has the GIL though. It's a problem in every dynamic language I've looked at. Last I checked it was true for Ruby too.

Re: TXR – A Programming Language for Convenient Data Munging

#53
post #18

Earlier quoted context omitted.

Way off topic, but as someone who has recently switched to using a non-standard background color in my browser... that page is horrendous to read: https://i.imgur.com/pvCnmSa.png I can accept that doing something non-standard leads to some rough edges like this, but i'm not sure how many web developers know this is an issue. At least it has surprised me how many websites have this issue of assuming the default color…

Hi; try it now! I GIMP-ed the image such that the non-transparent pixels are pure red, and only slightly opaque, instead of 100% opaque pinkish white. It looks about the same on a white background. Thanks, again. I tested it with a lightly grey background, as well as heavy gray. This little experiment really made me notice HN's hard-coded light grey background box, BTW.

Yeah, looks great here. Really didn't expect you to dig into it at all let alone so quickly. Thanks very much :-)

Re: TXR – A Programming Language for Convenient Data Munging

#54

Earlier quoted context omitted.

Lambdas are limited to a single expression. Threads don't work the way anyone would ever want threads to work because o the GIL. That said, homoiconicity (s-expressions) is not a feature I want. It makes code like reading a wall of text compared to a nicely laid out magazine.

Racket also has the GIL though. It's a problem in every dynamic language I've looked at. Last I checked it was true for Ruby too.

Common Lisp implementations often have no GIL, and, btw, are able to reach C speeds in certain cases.

Re: TXR – A Programming Language for Convenient Data Munging

#55
post #43

Earlier quoted context omitted.

Lambdas are limited to a single expression. Threads don't work the way anyone would ever want threads to work because o the GIL. That said, homoiconicity (s-expressions) is not a feature I want. It makes code like reading a wall of text compared to a nicely laid out magazine.

Gotcha. I do agree about homoiconicity. It's great for writing macros, and terrible for everything else. Of course, go too far in the other direction and you get perl, so... yeah.

>I do agree about homoiconicity. It's great for writing macros, and terrible for everything else.

I don't know; i've written code in C, C++, C#, Java, Python, Ruby, Pascal, Delphi, Assembler x86, TCL, Javascript and Common Lisp. Lisp codebases are the cleanest and clearest i've seen by far, although ReasonML/SML/OCaml might be as clean too.

Re: TXR – A Programming Language for Convenient Data Munging

#56

We already have this, it is R with tidyverse. What we need is a fully baked transpiler from R/tidyverse to sql.

Yep. Seriously. R w/tidyverse is a ridiculously powerful data wrangling tool especially when dealing with text files. I tend use Notepad++ when starting out on a data-wrangling adventure. It has an uncanny ability, unlike any other editor, to open hundreds of files at the same time and to perform regex operations on all of them without dropping dead. I uses Notepad++ for initial manual exploration to get the lay of t…

>I tend use Notepad++

I assume, then, that your file sizes are not so big. N++ is not good with big (>25% of your ram) file sizes, refusing to open them.

Is R/tidyverse also limited on the size of the file it can handle? In my job i routinely work with up to 100GB files.

Re: TXR – A Programming Language for Convenient Data Munging

#57

Very interesting. I'm wondering why they didn't implement the Lisp version on top of CL with macros

I can summarize this as follows. TXR is my research platform into various topics, including many Lisp topics. It contains numerous innovations. As a whole, that requires working at the implementation level, ground up.

Thanks Kaz! I had the same question.

Re: TXR – A Programming Language for Convenient Data Munging

#58

Earlier quoted context omitted.

Lambdas are limited to a single expression. Threads don't work the way anyone would ever want threads to work because o the GIL. That said, homoiconicity (s-expressions) is not a feature I want. It makes code like reading a wall of text compared to a nicely laid out magazine.

Racket also has the GIL though. It's a problem in every dynamic language I've looked at. Last I checked it was true for Ruby too.

That's true about Racket, but it has a quite original way of dealing with it with their futures[1].

To be honest, I'm not sure about the details, but it should let other threads run truly in parallel as long as it's "safe" to do on the VM implementation level. So, if the code inside the future doesn't perform any "future unsafe" operations, it can execute within a separate OS thread without worrying about the main thread.

Examples of "future unsafe" actions were given as memory allocation, and JIT compilation. Further, it's mentioned that some simple (for the language users, at least) operations may be too complex internally to be "future safe". An example of this is using a generic number comparison operators - ``, etc. Apparently, these have to handle the full numeric tower of Racket and in the process perform some future unsafe operations.

In the Mandelbrot function given as an example in the guide, simply replacing the generic comparisons with the ones specialized for work on floats specifically (and assuming that contract is not broken, which would immediately stop the future) allows the future to execute fully in parallel.

What is important to note here is that `set!` and friends, and so mutation of shared memory, is considered "future safe", ie. it's permitted to use them! (although then it's you who deals with the usual problems that brings).

I think it's worth mentioning here, because it's a novel strategy that seems to be between the two usual solutions (1. we've got GIL, live with it; 2. spawn more processes and get them to work - well, now you have many GILs...) and is showing some promising results. Plus they have a neat visualization tool!

Currently, it's limited and works best for purely numerical computations (which is also where you'd need it 99% of the time), but in some cases, it appears to work: the programmers of the language (not the implementation of the language) are given a tool to work outside the GIL in a structured manner plus a tool for closely inspecting low-level operations that happen in their code which would suspend or stop the future.

I'm not aware of any other dynamic or not language which has both the GIL and a nice, language-level tool for freeing it and running in parallel. Because what is considered "future unsafe" depends on the details of the implementation, I'm full of hopes for Racket-on-Chez, although I think I read somewhere that work on futures is not a priority at this time.

Also, to confirm the sibling comment, SBCL is happy to spawn truly parallel threads. There are other Scheme implementations (I think Chicken at least, but not sure right now) who allow the same.

[1] https://docs.racket-lang.org/guide/parallelism.html#%28part....

https://docs.racket-lang.org/reference/futures.html

Re: TXR – A Programming Language for Convenient Data Munging

#59

Earlier quoted context omitted.

Lambdas are limited to a single expression. Threads don't work the way anyone would ever want threads to work because o the GIL. That said, homoiconicity (s-expressions) is not a feature I want. It makes code like reading a wall of text compared to a nicely laid out magazine.

Racket also has the GIL though. It's a problem in every dynamic language I've looked at. Last I checked it was true for Ruby too.

Tcl does not have GIL and is possibly one of the most dynamic languages that has seen nontrivial use. Guile does not have GIL. Running multiple interpreterd in different threads is a common notion in Tcl.

Re: TXR – A Programming Language for Convenient Data Munging

#60
post #9

It would be interesting to have a DSL for data munging, but I am afraid TXR is not it. My requirements would be that the language should be functional and total. Most transformations that we do on data do not require Turing completeness or recursion. I think it would be useful to write these down in a language with semantics that is easy to analyze.

The funny thing is, I originally didn't intend the TXR pattern language to be recursive. It needed functional decomposition (pattern functions) to break up a big pattern match into simpler units. When those were implemented, I realized after the fact, hey we have a push-down automaton that can now grok recursive grammars. I don't see why we would want to rule out a pattern function invoking itself (directly, or throu…

I found in practice that arbitrary recursion depth is (even on languages with formal recursive grammar) very rarely needed. And where it's needed it can probably be implemented as a primitive in the language (map total function over all the nodes) that can do a similar thing.
Post reply on HN