Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear sir, you have built a compiler

rachitnigam.com

Post reply on HN
Print view View original

61–70 of 177 posts

Re: Dear sir, you have built a compiler

#61

Post by neurotrace »

Koshkin wrote: post #26
throwfaangus wrote: post #9

Earlier quoted context omitted.

Just use flex / bison?

Looks like these are grossly underappreciated these days. They are wonderful, easy to use tools.

I think "easy to use" might be an overstatement. I spent a fair bit of time trying to get in to the flex/bison workflow and it never seemed to click. I've found it much easier to use PEG parsers, parser combinators, or a hand-written recursive descent parser. Based on their ubiquity and the high reviews, I'm sure they're great tools once the initial hump is passed but that initial hump is quite large.

Re: Dear sir, you have built a compiler

#62

Post by agumonkey »

alexjplant wrote: post #57
hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

IMHO "programming language" would refer specifically to the grammar, type system, semantics, keywords, etc. and "compiler" would refer to the implementation that takes something written in that representation and outputs some type of executable. A pedantic distinction but probably a useful one, especially given how some languages have multiple implementations of their standard. The Programming Languages course I took…

does it affect how you code other kinds of apps too ?

knowing how to think about a program interpretation space gives quite a broader view IMO

Re: Dear sir, you have built a compiler

#63

Post by mattgreenrocks »

Karunamon wrote: post #41
jimmaswell wrote: post #38

Earlier quoted context omitted.

Implementing a small domain-specific language is fairly easy. Why avoid it?

Because the complexity of the problem tends to scale to your willingness to address said complexity. In other words, it likely won't stay small for long if it gets any users, and now you're the maintainer for a tool used by others.

This applies to all software in general. You'll get eaten alive by your success if you don't manage expectations and tightly scope things.

At least with a DSL, you can often define a small core language and de-sugar down to that.

Re: Dear sir, you have built a compiler

#64

Post by a_shovel »

meinte37 wrote: post #36

The article is “addressed to those who did not want to build a compiler”: what's the point of this article then? It seems to implore to not build compilers, but those who did not want to in the first place, probably actually did not build compilers. The article does point out numerous challenges with building compilers, and, by extension, with software language engineering, which concerns itself not just with buildin…

It's not a warning not to build a compiler, it's more of a story about how feature bloat might cause one to inadvertantly build a compiler without realizing that's what they were doing. The last line reveals it by telling us that each of those "features" was actually a standard compiler component:

> Done at last, you say to yourself, without having to build a compiler.

> A parser, an intermediate representation, transformation passes, and a code generator. Dear Sir, you have built a compiler.

Re: Dear sir, you have built a compiler

#66

Post by mepian »

noblethrasher wrote: post #22
rachitnigam wrote: post #20

Earlier quoted context omitted.

Racket is often heralded as the "programming language programming language" and comes with tons of features to build full blown languages: https://racket-lang.org/ The most complete language/ecosystem that really showcases Racket's capabilities is Rosette IMO: https://github.com/emina/rosette/

Also worth noting that language that runs this very website was implemented in Racket.

I think it was implemented in PLT Scheme before it stopped being a Scheme implementation and became Racket.

Re: Dear sir, you have built a compiler

#67

Post by mattgreenrocks »

agumonkey wrote: post #62
alexjplant wrote: post #57

Earlier quoted context omitted.

IMHO "programming language" would refer specifically to the grammar, type system, semantics, keywords, etc. and "compiler" would refer to the implementation that takes something written in that representation and outputs some type of executable. A pedantic distinction but probably a useful one, especially given how some languages have multiple implementations of their standard. The Programming Languages course I took…

does it affect how you code other kinds of apps too ? knowing how to think about a program interpretation space gives quite a broader view IMO

I'd argue the idea of an "interpreter" is one of the most foundational concepts of CS.

It sounds really basic, but the idea of being handed a description, and doing/creating something based on that is everywhere. It is really quite beautiful.

Re: Dear sir, you have built a compiler

#68

Post by TheCoelacanth »

TillE wrote: post #18

YAGNI is a good principle here. Whenever I've found myself thinking about reaching for a parser library, I was over-complicating or over-generalizing the problem. Write the code you need to solve the problem you actually have.

YAGNI works well as long as you are willing to re-write your code in the rare cases when you do "need it".

It's fine to start without a real parser, and you often will end up not ever needing a real one, but if the complexity of your ad hoc solution starts getting anywhere close to the complexity of a real parser, you are better off re-writing it as a real parser because parsing is an extremely well-studied area of CS while your ad-hoc "not parsing" is not.

Re: Dear sir, you have built a compiler

#69

Post by derefr »

hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

> and then it's probably better to commit and learn the ins and outs of that specific problem domain.

Even more, it's probably better to start off with a full understanding of these systems, so you can recognize when you're Greenspunning one of these abstractions, and just switch to using a robust off-the-shelf implementation of the same abstraction. (E.g. switch from writing a hand-rolled parser, to using a real parser-combinator library, or writing a formal grammar for a PEG compiler.)

Emphasis on "off-the-shelf": the most important thing to get from experience with these abstractions, is the understanding that you it will be less work to use the abstraction that has more power; and, in fact, that much of the tooling around these abstractions is built to assume that you want the full power of the tool by default, such that you have to add configuration to get less. So the workflow is actually "start off ordering 'one with everything', and then streamline your system over time, removing bits and pieces, as you notice properties of your particular situation that mean that you can do with something less powerful at particular stages."

Re: Dear sir, you have built a compiler

#70

Post by derefr »

olivierduval wrote: post #3

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

Most people who want this end up just selecting a regular programming language with a powerful-enough macro system + flexible-enough compile-time parser; and then using said language to define a "more or less complete DSL", by redefining all the primitive syntax features of the language to instead be primitive syntax features of the DSL.

For example, the "numerical definition" DSL within Elixir's Nx library (https://github.com/elixir-nx/nx/tree/main/nx#numerical-defin...).

Users using this "full" DSL are still using the host language — they're invoking its compiler/runtime and so forth — but they don't necessarily need to care about that. They can still technically access the power of a full-blown programming language, "tucked in" around the edges of the DSL — but you as the DSL's creator, don't have to document that power as being available. As long as users of the DSL can do everything they need to do without any need for "breaking out" of the DSL's little world, they don't need to be aware that they could e.g. make fully-namespaced calls to regular stdlib functions of your runtime.

Post reply on HN