Live data from Hacker News

Just what does “code as data” mean anyway? (2014)

adambard.com

111–120 of 178 posts

Re: Just what does “code as data” mean anyway? (2014)

#111
post #41

Earlier quoted context omitted.

Refactoring lisp gets harder the more complex the program gets. Approachs that are clever and reasonable for a 1000 line program can become hell in a 100,000 line program. Effectively programming languages get less powerful the longer the program becomes. Breaking up programs into more powerful little pieces is never a clean separation and you need to make real trade offs between more separation and more power.

>Refactoring lisp gets harder the more complex the program gets. Approachs that are clever and reasonable for a 1000 line program can become hell in a 100,000 line program. Can you substantiate this claim? One of the reasons Lisp was used in the past was because it was particularly suited to complex, big projects... Like autopilot for spaceship, wind tunnel simulation, CAD/CAM systems, etc. Lisp (as 'Common Lisp', th…

> Care to explain?

Some of this comes down to what you consider a 'large' program. Suppose you want to build an MMO in lisp, now you need to consider things running on both a client, and server, as well as rendering, path-finding etc. Choosing reasonable primitives becomes difficult as you get pulled in several directions. Should I be running the same code on client an server or should I have multiple different models communicate etc etc.

My point is you start dealing with trade offs and no approach is universally better. Even with a clean API between different pieces of code you still get conceptual dependency's.

Re: Just what does “code as data” mean anyway? (2014)

#112
In Java, you have code as string. That is, your code is represented using a big string. In Lisp, your code is represented using the list data structure.

So its really code-as-data-structure.

A string is difficult to parse and modify, inserting things in the middle, removing elements, changing the order of the words, that's all really difficult with a string. So if you want to transform Java code, its going to be hard and error prone.

A list is easy to manipulate in contrast. Inserting elements in the middle is trivial, so are deletes and swaps. So in Lisp, if you want to transform code its pretty easy.

Meta-programming is when you write a program that writes a program. An example is say you want to you wanted to add a semi-colon at the end of all your lines of code. You need a macro to do it for you. A macro is a program that acts upon your code to transform it. Eclipse has them. So now its really easy to add a semi-colon at the end of each line. What if you wanted to add a comma between all words in a selection? Now its trickier if your macro operates over a big string, you might need a regex for example. This is meta programming though. Instead of adding the commas yourself, which are required for the program to run, you write another program to add them for you.

Now if all the words were elements in a list, that macro would be a lot easier to write.

This is in essence what code-as-data(structure) means. Its in contrast with code-as-string. You don't have to choose lists as your datastructure either, as long as its something that allows you to represent a turing complete program and is easy to manipulate.

Now, homoiconicity is the fact that your text of code looks like a data-structure too. Making it trivial to parse it into one. So back to my example, you could parse the java code string into a list, and then add commas, and then convert it back to a string. But Java code doesn't map logically into a list. Some construct don't nest like lists, and how do you define what goes into each node of the list? Do you group public and String together? In Lisp, the syntax is an unambiguous AST already, its thus trivial to parse into a list of lists. So you can easily get that AST you need to easily add commas where it make sense.

Finally, there's a third aspect. Code-as-data also implies that your language can accept code as argument in the form of raw data. The best way to think of it is, how would you send a function over the wire to a program and have that program run the code I sent? You need a way to serialize that function, which is code, into raw data that can be transmitted over the wire. The receiving program doesn't have that function defined, so it needs to know how to deserialize it, but also at runtime it must be able to take this raw data, which represents code, and be able to parse it, compile/interpet it and execute it.

Think of SQL, SQL is often used in a code-as-data way. You want "select * from %s". Now you'd take this as a string, and you'd use a string replace, and replace %s with something the user picked in a drop down. At runtime, you are dynamically creating the SQL code, and once you have it, running it. You might have methods that accept SQL and return SQL. Now again, your SQL is a big string, which isn't ideal. But this is still an example of code as data. Now in Java, you can not do that with Java code. Java does not have this concept of code-as-data. In other words, there's no eval.

So when you combine all these three aspects, a homoiconic syntax that parses easily and logically into a data-structure which is easy to manipulate, and where you can then execute data which represent code at runtime and pass it around to other functions, even over the wire, you get a very powerful combo that turns into a Meta-programming powerhouse. This is the strength of Lisps, the one strength all Lisps share.

Re: Just what does “code as data” mean anyway? (2014)

#113
post #55
post #41

Earlier quoted context omitted.

Refactoring lisp gets harder the more complex the program gets. Approachs that are clever and reasonable for a 1000 line program can become hell in a 100,000 line program. Effectively programming languages get less powerful the longer the program becomes. Breaking up programs into more powerful little pieces is never a clean separation and you need to make real trade offs between more separation and more power.

> Refactoring lisp gets harder the more complex the program gets. I have found that to be true in dynamically or duck typed languages. I have found it to be not true in strongly typed languages with good tooling. Example: refactoring large java program in, say, Eclipse. > Breaking up programs into more powerful little pieces is never a clean separation I would tend to disagree, to some extent. In Lisp(s), there are l…

> layers of separation

Layers are exactly the kind of power trade-off I am talking about. If code X can't(1) impact code Y, you have less freedom in code X because it can't do some things, but can more easily reason about code Y.

(1) either as an actual limit or a self imposed one.

Granted, in theory you may be able to find perfect points of separation such that you don't lose power by doing so.

Re: Just what does “code as data” mean anyway? (2014)

#114
post #45

One of the failures of conventional programming libraries is that parsing libraries such as yacc work in only one direction. In 2018 we could easily have libraries that work both ways by default, but we don't. Bidirectional parsing is great for code generation and opens up a lot of things you could do easily, but because common parsing libraries are unidirectional, people aren't aware of what you can do and don't cla…

The way I look at it, it is difficult enough to do "1d" programming correctly. This is "2d" programming, in a sense - programming at multiple levels, if I understand correctly. I would absolutely hate to try to parse code from someone else written in this manner. In my mind, this is wasted brain cells. Professional use of software favours simple, clean maintainable code. Simple style code, saves brain cells for the a…

This is absolutely correct, and considerate Lisp writers make a point of using macros only when necessary for exactly this reason.

That said, Lisp writers also use macros every day -- for example, in Clojure, only `if` is defined as a special form, and other conditional operators (`when`, `cond`, `case`, `condp`, `if-not`, etc.) are implemented as macros that use `if` under the hood. Good macros are almost necessarily difficult to implement, because they're deployed to solve problems that functions can't, but if done well they can be easy to read and use.

Re: Just what does “code as data” mean anyway? (2014)

#115
post #94

Earlier quoted context omitted.

True, but. sum(1,2,sum(3,4)) isn't a list. so sum() isn't a list member and things passed to it aren't list either. Lisp code is more consistent.

No, it's equally consistent. In lisp, everything is a list. In the example I just gave, everything is a scalar. Like I said, this is a very bad justification for lisp.

Everything isn't a scalar in the example you mentioned. sum() is a function and other things are numbers.

In lisp there is only rule, everything is a list and the first element of the list is a operation to the remainder of the list. This is basically the whole language.

Re: Just what does “code as data” mean anyway? (2014)

#116
post #93

Earlier quoted context omitted.

I really value C syntax. I'm convinced it's much easier to navigate with the eyes than "oatmeal with fingerclips mixed in" (quoting Larry Wall). And >90% of the time the relevant syntactic unit is on its own line or ranges of lines, and that's really super easy to handle in vim. I don't think there is much to improve by building more complex abstractions on top. If you use vim as a C programmer, you should know the b…

> "oatmeal with fingerclips mixed in" (quoting Larry Wall) I still chuckle at that quote and Larry Wall is great, but given the look of traditional Perl I wouldn't put much stock in Wall's language syntax aesthetic ;-)

>>I still chuckle at that quote

That quote isn't one statement. Its basically a part of talk/essay and he actually says a lot of nice things about Lisp in that essay, and then mentions this as a joke to close it all.

Re: Just what does “code as data” mean anyway? (2014)

#117
post #110

As someone who uses Clojure for only side projects, macros seem to get a lot of attention, both good and bad, for something I very rarely write. Maybe I'm missing something though, and programmers using these languages professionally resort to them more often than I do.

code-is-data also enables structural editing–instead of typing strings we can transform data–, there are gifs of that here: https://cursive-ide.com/userguide/paredit.html

Re: Just what does “code as data” mean anyway? (2014)

#119
post #102
post #91

Earlier quoted context omitted.

>>Sounds like your experience was with Emacs Lisp, a very clunky Lisp. Emacs lisp is actually a very practical, easy to use and fun lisp to do a lot of great work in. Richard Matthew Stallman notes: It was Bernie Greenberg, who discovered that it was (2). He wrote a version of Emacs in Multics MacLisp, and he wrote his commands in MacLisp in a straightforward fashion. The editor itself was written entirely in Lisp. M…

Please do not conflate MacLisp and Emacs Lisp, they are different languages. GNU Emacs is not directly derived from Multics Emacs, it's one of the many reimplementations of the original Emacs. The closest thing to MacLisp today is actually Common Lisp, and it's much more advanced than Emacs Lisp, which is not surprising considering that Stallman was not interested in creating a state-of-the-art Lisp implementation wi…

Thanks for mentioning this.

I was just trying to point out the fact that its not hard to code in Lisp. Especially given the fact that non-programmers were doing it long back, in an era where there was nothing like Internet available to ask for help.

Re: Just what does “code as data” mean anyway? (2014)

#120
post #111

Earlier quoted context omitted.

>Refactoring lisp gets harder the more complex the program gets. Approachs that are clever and reasonable for a 1000 line program can become hell in a 100,000 line program. Can you substantiate this claim? One of the reasons Lisp was used in the past was because it was particularly suited to complex, big projects... Like autopilot for spaceship, wind tunnel simulation, CAD/CAM systems, etc. Lisp (as 'Common Lisp', th…

> Care to explain? Some of this comes down to what you consider a 'large' program. Suppose you want to build an MMO in lisp, now you need to consider things running on both a client, and server, as well as rendering, path-finding etc. Choosing reasonable primitives becomes difficult as you get pulled in several directions. Should I be running the same code on client an server or should I have multiple different model…

This looks more like software design problem and not a Lisp problem.

Building anything complex requires, apart from tools, getting organized in a lot of other things as well.

Post reply on HN