Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear sir, you have built a compiler

rachitnigam.com

Post reply on HN
Print view View original

81–90 of 177 posts

Re: Dear sir, you have built a compiler

#81

Post by mnw21cam »

hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

My first job out of university, I ended up doing one of those by mistake. My second job after university, I ended up doing another three of those, but at least we confronted the thing head on and deliberately, knowing that we were aiming to create them. Trust me, it's better to do it deliberately than by mistake.

Re: Dear sir, you have built a compiler

#82

Post by jacquesm »

K0balt wrote: post #6
jacquesm wrote: post #5

Earlier quoted context omitted.

There's Forth. Many would argue it's not a full blown programming language, it's more like a toolkit to make DSLs.

It’s great when you don’t need to have a huge codebase.IME forth gets out of control quickly from a maintainability standpoint as you go big, but it is excellent at small dsl type tasks. The maintainability issues with forth might actually be solvable if the right IDE paradigm could be found.

I don't think so. Let me try to explain: regular languages are only viable as long as someone is willing to speak them and learn them. Latin being the obvious exception, but that's mostly because of medical science and biology, two very large branches of the scientific tree.

A DSL is a language with only a handful of users, each of which is usually in the normal path of employment (obvious exception: Chuck Moore). As soon as someone in a team that relies on a DSL leaves the loss of knowledge is immediate. Unlike any other language that you may have picked up in some other organization the DSL has instantly lost a sizeable fraction of those versed in it. It doesn't take many losses like to kill such a project.

Another problem is that even if the original authors hang around, the chances that they will stay fluent in their own DSL diminishes with increasing intervals between work on a particular project. If the code would be written in a 'normal' language then they would stay fluent.

So DSLs are not a good match for how we normally think about software, even though there is a 1:1 correspondence between the words in your typical DSL (say a FORTH word) and a function call. Functions tend to be written in terms of other functions, just like FORTH words are written in terms of other words, and yet, the standard library is never far away whereas with FORTH words it could easily be 30 words down before you hit the actual language.

I'd love to see something like a DSL based language but with longer term prospects, for a long time I hoped that Smalltalk would be that language but I have mostly given up on that.

Re: Dear sir, you have built a compiler

#83

Post by all2 »

voidmain0001 wrote: post #13
jacquesm wrote: post #5

Earlier quoted context omitted.

There's Forth. Many would argue it's not a full blown programming language, it's more like a toolkit to make DSLs.

There's also Eric Lippert's blog series on OOP and business rules [1] that culminates [2] in suggesting a DSL with Inform7 as a possible candidate. The comments on the final post are great as people rail against the use of a DSL. [1] https://ericlippert.com/2015/04/27/wizards-and-warriors-part... [2] https://ericlippert.com/2015/05/11/wizards-and-warriors-part...

I started working with C# about 6 months ago. I'd read that post before, but I'm only just now understanding it. I think I really missed the boat when it comes to things like OO programming patterns. I'm still kinda fuzzy on the visitor pattern he mentioned. I think I get it?

The abstraction he offered, of having a tree and then visiting every node in that tree, made the most sense.

> One of the most common usages of this pattern is for the scenario where you’ve got a complex tree where tree nodes can be of many types, and you wish to traverse the tree — that is, visit every node in the tree — taking some actions or performing some computations as you go. The “double dispatch” part of it is that the actions you take as you visit nodes in the tree can depend on the runtime types of both the nodes in the tree, and the code performing the action.

That sounds like a parser. It accepts an AST and evaluates productions on its way through. Saying "actions you take ... depend on the runtime types of ... nodes ... and the code performing the action" sounds similar to sorting out how to mash tree-traversal and productions together.

In my compilers class we first wrote a code linter for our "Wombo" language (a simplified Java). Then we wrote something that built an AST. The core tree traversal code was the same, but the "productions" paired with nodes were different.

Re: Dear sir, you have built a compiler

#84

Post by hobofan »

eulers_secret wrote: post #72
hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

> …But sometimes you actually _do_ want to build that… That’s not true for everyone. I’ve been a dev for 20 years and have never done any of those, and I have no desire to at all. That is ok! Programming isn’t some kind of progression to run from “n00b hello world” to “1337 compiler hax0r”, it’s a tool to solve problems with. Many of us will never have to solve these problems or have no interest in these problems. So…

I think I might have used the expression "do want" to vaguely here. Same as you, I have very little desire to write any of those systems, and still try to avoid them as much as possible (as those endeavors usually require more time/energy, etc. and come with higher risks).

Maybe it's better put as "But sometimes all your product requirements strongly suggest that you have to build X and there isn't really a way to avoid it" (and what's available off-the-shelf doesn't fit)?

Re: Dear sir, you have built a compiler

#86

Post by rachitnigam »

a_shovel wrote: post #64
meinte37 wrote: post #36

The article is “addressed to those who did not want to build a compiler”: what's the point of this article then? It seems to implore to not build compilers, but those who did not want to in the first place, probably actually did not build compilers. The article does point out numerous challenges with building compilers, and, by extension, with software language engineering, which concerns itself not just with buildin…

It's not a warning not to build a compiler, it's more of a story about how feature bloat might cause one to inadvertantly build a compiler without realizing that's what they were doing. The last line reveals it by telling us that each of those "features" was actually a standard compiler component: > Done at last, you say to yourself, without having to build a compiler. > A parser, an intermediate representation, tran…

[OP] Yup, that was the intent. I've seen a lot of academic and industrial hand wringing about not wanting to build a compiler because it's an overkill for the solution only to have the people come back 6 months later having built a compiler.

I'm not saying everyone should build a compiler for everything–but when you should build a compiler, you really should bite the bullet and build a compiler.

Re: Dear sir, you have built a compiler

#88

Post by [deleted] »

derefr wrote: post #69
hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

> and then it's probably better to commit and learn the ins and outs of that specific problem domain. Even more, it's probably better to start off with a full understanding of these systems, so you can recognize when you're Greenspunning one of these abstractions, and just switch to using a robust off-the-shelf implementation of the same abstraction. (E.g. switch from writing a hand-rolled parser, to using a real par…

[deleted]

Re: Dear sir, you have built a compiler

#89

Post by LeifCarrotson »

eulers_secret wrote: post #72
hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

> …But sometimes you actually _do_ want to build that… That’s not true for everyone. I’ve been a dev for 20 years and have never done any of those, and I have no desire to at all. That is ok! Programming isn’t some kind of progression to run from “n00b hello world” to “1337 compiler hax0r”, it’s a tool to solve problems with. Many of us will never have to solve these problems or have no interest in these problems. So…

I don't think the parent was suggesting that the individual developer has a personal desire to build a compiler for the fun of it.

It's much wiser and far easier (assuming your goal is to have a successful career and run a profitable business) if you Keep It Simple, Stupid! Building a compiler/database/CMS/ERP instead of using an off-the-shelf one is, for well-selected business problems, a bad decision, increasing complexity, adding risk, and allowing scope creep.

But sometimes, in spite of your efforts to adhere to the KISS principle, the real world is hairy and some complicated new problems need complicated innovative solutions, and instead of turning a crank to glue together some off-the-shelf libraries, you'll find yourself, in my particular example, way off in the deep end getting PCBs manufactured or, in the article, writing your own compiler. It's not bad, it just means you've got a niche problem to solve! Or that sales did a terrible job of understanding what they were offering, and something basically equivalent to the customer is just an off-the-shelf Wordpress plugin, but instead you're stuck writing a custom CMS.

Re: Dear sir, you have built a compiler

#90

Post by Communitivity »

I feel like this easily could have been addressed to a much younger version of myself on a particular project more than a decade ago (~12 years). My story happened in the early days of SPARQL, when SPARQUl (SPARQL Update Language) was very new, a draft, and definitely not merged into SPARQL yet. Oh, and dinosaurs roamed the earth. That last is from my daughter.

My project involved semi-autonomous software agents that each maintained a model of one aspect of a thing being monitored by different types of sensors. Agents were divided into nerves, and brains. The model maintained by a nerve was of a limited aspect, containing explicit, semantic representations of sensor reported values. The models maintained by brain agents were implicit, derived from one or both of two stages of constrained semantic reasoning (closed world model, and some other constraints).

Some nerve agents did the first stage of semantic reasoning, more simplistic using description logic programming rules encoded into the model. The results were stored into the model so that they, and parts of the more complex aggregate models derived from them up the chain, could be removed on the next pass.

Brain agents did a second pass of semantic reasoning. This pass is where the 'You have implemented a compiler' comes in.

That pass used rules encoded in human readable form. We looked at CWM for our rules, but that didn't have enough expressivity and didn't mesh well with some other tools we had.

We encoded the rules a format we created, called Sparql++. It had SPARQL, SPARQL Update Language, for loops, while loops, variables, and macros. Our code took the rules and put them into an intermediate form, then translated that intermediate form to EulerSharp rules and ran them, then injected the results into our model. The intermediate form and EulerSharp rules were saved so parts that hadn't changed didn't generate new EulerSharp rules.

It was ungainly. It worked well. And it was effectively a compiler.

Post reply on HN