Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TXR – A Programming Language for Convenient Data Munging

nongnu.org

Post reply on HN
Print view View original

31–40 of 74 posts

Re: TXR – A Programming Language for Convenient Data Munging

#31

Post by kazinator »

auvrw wrote: post #23
otoburb wrote: post #8

"TXR Lisp programs are shorter and clearer than those written in some mainstream languages "du jour" like Python, Ruby, Clojure, Javascript or Racket. If you find that this isn't the case, the TXR project wants to hear from you; give a shout to the mailing list. If a program is significantly clearer and shorter in another language, that is considered a bug in TXR." That section made me chuckle. Admirable if true.

i agree that the general-purpose programming language space is fairly crowded ... the lisp dialect/user ratio especially so. DSLs, otoh, are in short supply. while awk or plain sed are great for shell programming, this is the only (open source) DSL i'm aware of targeting certain types of NLP-esque "munging". this space is mostly full of statistical approaches, which, while conceptually pure, don't allow the kind of f…

N. Westbury has been cloning it in Java:

https://github.com/westbury/txr-java

Re: TXR – A Programming Language for Convenient Data Munging

#32

Post by kazinator »

mark_l_watson wrote: post #24
buckminster wrote: post #22

Earlier quoted context omitted.

Exactly. It's GNU hosting for non-GNU projects.

Oh, that makes sense. thanks!

Of course, Non-GNU projects that are licensed in such a way that they could be GNU projects.

From the registration page, the kind of software project that can be hosted on Savannah is [a] free software package that can run on a completely free operating system, without depending on any nonfree software. You can only provide versions for nonfree operating systems if you also provide free operating systems versions with the same or more functionalities. Large software distributions are not allowed; they should be split into separate projects.

Re: TXR – A Programming Language for Convenient Data Munging

#33

Post by taeric »

crispyambulance wrote: post #20
uptownfunk wrote: post #15

We already have this, it is R with tidyverse. What we need is a fully baked transpiler from R/tidyverse to sql.

Yep. Seriously. R w/tidyverse is a ridiculously powerful data wrangling tool especially when dealing with text files. I tend use Notepad++ when starting out on a data-wrangling adventure. It has an uncanny ability, unlike any other editor, to open hundreds of files at the same time and to perform regex operations on all of them without dropping dead. I uses Notepad++ for initial manual exploration to get the lay of t…

The irony, of course, is that txr predates tidyverse.

Re: TXR – A Programming Language for Convenient Data Munging

#34

Post by flavio81 »

otoburb wrote: post #8

"TXR Lisp programs are shorter and clearer than those written in some mainstream languages "du jour" like Python, Ruby, Clojure, Javascript or Racket. If you find that this isn't the case, the TXR project wants to hear from you; give a shout to the mailing list. If a program is significantly clearer and shorter in another language, that is considered a bug in TXR." That section made me chuckle. Admirable if true.

>That section made me chuckle. Admirable if true

Mostly true for very high level languages like Lisp/Scheme, or ML/OCaml/F#/Haskell, when faced against not-so-high-level languages like C, C++, Java.

Against Racket, i wouldn't be so sure. Nor against Ruby.

Python and Javascript are high level languages but they are crippled by some bad design decisions.

Re: TXR – A Programming Language for Convenient Data Munging

#35

Post by kazinator »

js8 wrote: post #9

It would be interesting to have a DSL for data munging, but I am afraid TXR is not it. My requirements would be that the language should be functional and total. Most transformations that we do on data do not require Turing completeness or recursion. I think it would be useful to write these down in a language with semantics that is easy to analyze.

The funny thing is, I originally didn't intend the TXR pattern language to be recursive. It needed functional decomposition (pattern functions) to break up a big pattern match into simpler units. When those were implemented, I realized after the fact, hey we have a push-down automaton that can now grok recursive grammars.

I don't see why we would want to rule out a pattern function invoking itself (directly, or through intermediaries); if that hurts, then just don't do that.

(Though I understand that there are languages deliberately designed without unbounded loops or recursion, for justifiable reasons.)

Re: TXR – A Programming Language for Convenient Data Munging

#36

Post by kazinator »

jdmoreira wrote: post #6

Very interesting. I'm wondering why they didn't implement the Lisp version on top of CL with macros

I can summarize this as follows. TXR is my research platform into various topics, including many Lisp topics. It contains numerous innovations. As a whole, that requires working at the implementation level, ground up.

Re: TXR – A Programming Language for Convenient Data Munging

#37

Post by Jach »

kazinator wrote: post #25
anentropic wrote: post #3

> The PDF rendition of the reference manual, which takes the form of a large Unix man page, is over 600 pages long, with no index or table of contents. There are many ways to solve a given data processing problem with TXR. "Good luck, you're on your own!"

The "no index or TOC" isn't being touted as a feature, just that the page count is that without these (in documents like these, these features can contribute dozens to the page count). An index would be nice; patches welcome! The HTML version that most people would be using has a TOC with two-way navigation to the section headings and is hyperlinked. Of course, man page reading allows easy searching.

I guess threads like this remind me why it's nice to have professional doc writers review my customer-facing text at work. ;) Congrats on your project getting some more attention! If you'll indulge a bit of bikeshedding, this particular miscommunication could probably be avoided in the future by changing the sentence to the short "The PDF rendition of the reference manual is over 600 pages long." Even if you add extra things to the PDF later the statement won't be incorrect and so you won't have to deal with nitpickers coming by next time with a comment like "But if you remove the index it's only 597 pages!"

Another edit preserving more of the original would be to replace the final "with no" with something like "even excluding any"...

Re: TXR – A Programming Language for Convenient Data Munging

#38

Post by kazinator »

Author here. Currently working on a debugger. (Threw the old crappy one out.) Backtraces are working. Some of the remaining work is going to require long, uninterrupted concentration that is hard to come by due to taking care of a six-month-old baby.

I have over 50 unreleased patches. There are some bugfixes, including a compiler one, involving dynamically scoped variables used as optional parameters:

      (defvar v)
      (defun f (: (v v)))
      (call (compile 'f)) ;; blows up in virtual machine with "frame level mismatch"
Patch for that:

  diff --git a/share/txr/stdlib/compiler.tl b/share/txr/stdlib/compiler.tl
  index e76849db..ccdbee83 100644
  --- a/share/txr/stdlib/compiler.tl
  +++ b/share/txr/stdlib/compiler.tl
  @@ -868,7 +868,7 @@
                                       ,*(whenlet ((spec-sub [find have-sym specials : cdr]))
                                           (set specials [remq have-sym specials cdr])
                                           ^((bindv ,have-bind.loc ,me.(get-dreg (car spec-sub))))))))))
  -                 (benv (if specials (new env up nenv co me) nenv))
  +                 (benv (if need-dframe (new env up nenv co me) nenv))
                    (btreg me.(alloc-treg))
                    (bfrag me.(comp-progn btreg benv body))
                    (boreg (if env.(out-of-scope bfrag.oreg) btreg bfrag.oreg))
There is now support in the printer for limiting the depth and length.

I added a derived hook into the OOP system; a struct being notified that it is being inherited.

Re: TXR – A Programming Language for Convenient Data Munging

#39

Post by srean »

js8 wrote: post #9

It would be interesting to have a DSL for data munging, but I am afraid TXR is not it. My requirements would be that the language should be functional and total. Most transformations that we do on data do not require Turing completeness or recursion. I think it would be useful to write these down in a language with semantics that is easy to analyze.

Then I think you will like https://tkatchev.bitbucket.io/tab/index.html

"It's statically-typed and type-infered.

It also infers memory consumption and guarantees O(n) memory use.

It is designed for concise one-liner computations right in the shell prompt.

It features both a mathematics library and a set of data slicing and aggregation primitives.

It is faster than all other interpreted languages with a similar scope. (Perl, Python, awk, ...)

It is not Turing-complete. (But can compute virtually anything nonetheless.)

It is self-contained: distributed as a single statically linked binary and nothing else.

It has no platform dependencies."

I am a little suspicous that you may be the author ;)

Re: TXR – A Programming Language for Convenient Data Munging

#40

Post by Zaak »

flavio81 wrote: post #34
otoburb wrote: post #8

"TXR Lisp programs are shorter and clearer than those written in some mainstream languages "du jour" like Python, Ruby, Clojure, Javascript or Racket. If you find that this isn't the case, the TXR project wants to hear from you; give a shout to the mailing list. If a program is significantly clearer and shorter in another language, that is considered a bug in TXR." That section made me chuckle. Admirable if true.

>That section made me chuckle. Admirable if true Mostly true for very high level languages like Lisp/Scheme, or ML/OCaml/F#/Haskell, when faced against not-so-high-level languages like C, C++, Java. Against Racket, i wouldn't be so sure. Nor against Ruby. Python and Javascript are high level languages but they are crippled by some bad design decisions.

I know that Python and Javascript have their warts (as do all languages in my experience), but what decisions in particular are you thinking of?
Post reply on HN