Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear sir, you have built a compiler

rachitnigam.com

Post reply on HN
Print view View original

141–150 of 177 posts

Re: Dear sir, you have built a compiler

#141

Post by joatmon-snoo »

derefr wrote: post #69
hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

> and then it's probably better to commit and learn the ins and outs of that specific problem domain. Even more, it's probably better to start off with a full understanding of these systems, so you can recognize when you're Greenspunning one of these abstractions, and just switch to using a robust off-the-shelf implementation of the same abstraction. (E.g. switch from writing a hand-rolled parser, to using a real par…

The problem, of course, is that it's impossible to know everything :)

IME the best compromise I've found is to learn enough to recognize the problems, so that when I implement a bastardized compiler, I'm aware that that's what I'm doing and that at some point in the future I'll need to decide how deep the implementation should go or switch to actually using some library.

Re: Dear sir, you have built a compiler

#142

Post by mcguire »

wpietri wrote: post #78
jasoncabot wrote: post #56

This reminds me of a great post about the typical software evolution going from hard coded values, through configuration and rules engines to a DSL, only to be back where you started. I have seen it happen on multiple projects, it’s hard to spot it at the time as all the changes seem sensible but hindsight is a wonderful thing http://mikehadlow.blogspot.com/2012/05/configuration-complex...

This line really struck me: “They will have to change at some point, and you don’t want to recompile and redeploy your application just to change the VAT tax rate.” Continuous deployment has changed this for me. I'm still not going to leave tunable constants in the middle of random if statements. But nowadays instead of trying to avoid recompiling and redeploying, I'd rather invest that time into making that very eas…

Did you happen to see the recent discussion of the maintainer-hoarks-up-a-dependency problem? How does your CD system handle testing?

Re: Dear sir, you have built a compiler

#143

Post by krapp »

danuker wrote: post #52
noblethrasher wrote: post #22

Earlier quoted context omitted.

Also worth noting that language that runs this very website was implemented in Racket.

Hacker News runs on a language that runs on Racket? Which language is that?

Arc Lisp[0]. This forum was concieved as a MVP for the language.

Anarki, a divergent open source fork, can be found here[1].

[0] http://arclanguage.org/

[1] https://github.com/arclanguage/anarki

Re: Dear sir, you have built a compiler

#144

Post by cjbgkagh »

prionassembly wrote: post #130
cjbgkagh wrote: post #116

Earlier quoted context omitted.

Lol, I’ve built all those things. I think the main thing is having respect for the mountain of engineering that went into existing solutions and not rolling your own unless you really really have to. I think an Embedded DSL with post compilation constraint solver based analyzers could take the place of restricted mini-languages while taking advantage of the existing engineering and tooling available for modern langua…

So like a restricted subset? That's a really neat idea; you should develop it into a paper or something. This restricted subset concept has the huge advantage that you can slip out of the harness in a pinch. So you can reuse a relatively developed Float-less Python program in Full Python and later walk back removing floats. That's really clever. It essentially obviates the whole configuration DSL tango; it also gets…

You can restrict all sort of things, including of course a restricted subset of the language. Such solutions already have headcount at MS which is about as good as it gets for bringing a new thing like this to masses.

When it is released and as it matures we can extend it to do a bunch of other cool things like configuration DSL which would really help clean up Linux. Maybe even build scripts.

You could do stuff like this before with LISP, or Scala compiler plugins. But the tooling and accessibility will make a big difference.

Re: Dear sir, you have built a compiler

#145

Post by imoverclocked »

derefr wrote: post #69
hobofan wrote: post #31

I feel like you can apply the same sentiment to many of the "big scary things" in programming. Things that you don't want to build (as far as "common engineering wisdom" is to be believed): - a compiler - a programming language (not sure that there is a difference to compiler as stated in the article) - a database (query engine) - a CMS - a ERP But sometimes you actually _do_ want to build that (even if every alarm b…

> and then it's probably better to commit and learn the ins and outs of that specific problem domain. Even more, it's probably better to start off with a full understanding of these systems, so you can recognize when you're Greenspunning one of these abstractions, and just switch to using a robust off-the-shelf implementation of the same abstraction. (E.g. switch from writing a hand-rolled parser, to using a real par…

> more power

eg: log4j -- so much power you can enable random bitcoin miners with your infrastructure (tongue in cheek)

ie: Sometimes it's not about more power but about the smallest/least powerful implementation that accomplishes the job.

I agree that using off-the-shelf is often important but it also comes with the need to vet the thing in the first place. That's not always trivial if you care about packaged licenses from dependencies or a lack of hidden features or actively developed or a plethora of other dimensions to what the "shelf" provides.

Re: Dear sir, you have built a compiler

#146

Post by mcguire »

TillE wrote: post #18

YAGNI is a good principle here. Whenever I've found myself thinking about reaching for a parser library, I was over-complicating or over-generalizing the problem. Write the code you need to solve the problem you actually have.

Once upon a time, a project I was working on had a particular sub-problem: we sucked in a data file from an external source that consisted of a collection of records that had to be read, validated, and either diddled around in our DB or, in the case of errors, passed off to a human for pondering.

The guy who was originally assigned was quite skilled, but apparently he'd never written an interpreter before. He started with the typical Spring Beans Uber Alles infrastructure and processing for the records, but then somehow managed to get pulled off on some other aspect of the project and this part was passed on to me.

After my first look at what he had, I realized that error handling, which ranged from misformatted records to nonsensical values, would be a giant ball of razor wire and lemon juice if I continued with his design. So I tossed out much of his infrastructure---keeping his good-record handling---and wrote a simple recursive descent parser that recorded errors (including their locations in the data file) for human processing. The result was simple, clean (if you've seen an interpreter before), and I haven't heard of any bugs reported in it.

Turns out, IDNI.

Re: Dear sir, you have built a compiler

#147

Post by yccs27 »

meinte37 wrote: post #36

The article is “addressed to those who did not want to build a compiler”: what's the point of this article then? It seems to implore to not build compilers, but those who did not want to in the first place, probably actually did not build compilers. The article does point out numerous challenges with building compilers, and, by extension, with software language engineering, which concerns itself not just with buildin…

As I interpret it, it is adressed to those who have a problem which needs a compiler, but don‘t want to build one because it sounds scary and difficult.

Re: Dear sir, you have built a compiler

#148

Post by mcguire »

0xbadcafebee wrote: post #39

Ways you can avoid building a compiler: 1. Lower the scope of the solution. If your solution is designed to address 100% of the users' problems, it's probably too grand of a solution. Start with 80% of the problems. Identify all the problems, rank them by priority and difficulty, and leave the most difficult and least priority problems out. 2. Lower your expectations. Imagine your great idea. Now imagine how you will…

And in six months, the users will be hacking around the parts you left missing, probably by emailing Excel spreadsheets around.

If you did it right, you will be off to your next place of employment before the wheels completely come off.

Re: Dear sir, you have built a compiler

#149

Post by DoctorDabadedoo »

mcguire wrote: post #136
DoctorDabadedoo wrote: post #118

Earlier quoted context omitted.

Because if it succedds, it needs to be supported long term. If you're doing that inhouse, chances are slim of original designers staying 10-20 years at the same company. Then what you have in your hands is a legacy monolith neglected over its life which might double as core of your stack and it's awful to move away from down the line. Been there, don't recommend it.

Oh, my god! If it succeeds you'll have to support it forever! " Unfortunately, that's rather true of all software.

Indeed, but maintaining software that is a core part of your business is a very different game than supporting (complex) internal tooling. They don't bring revenue, are a time/money sink and not many companies can spare the resources to do it properly.

Anedoctal experience: a company I used to work for had all the core stack developed inhouse: programming language (yeah), compiler, database, etc. No one worked on it apart from the founder and a couple of other engineer that were no longer there.

A few years down the line a big client asked if we were able to integrate our software with their PLC and this had to be developed from scratch, as it wasn't in the scope of the programming language. The quote sent to them was in the six figures and up to 2 years for it to be ready. Every major programming language has a library that does this. Eventually the company lost the client.

Re: Dear sir, you have built a compiler

#150

Post by derefr »

imoverclocked wrote: post #145
derefr wrote: post #69

Earlier quoted context omitted.

> and then it's probably better to commit and learn the ins and outs of that specific problem domain. Even more, it's probably better to start off with a full understanding of these systems, so you can recognize when you're Greenspunning one of these abstractions, and just switch to using a robust off-the-shelf implementation of the same abstraction. (E.g. switch from writing a hand-rolled parser, to using a real par…

> more power eg: log4j -- so much power you can enable random bitcoin miners with your infrastructure (tongue in cheek) ie: Sometimes it's not about more power but about the smallest/least powerful implementation that accomplishes the job. I agree that using off-the-shelf is often important but it also comes with the need to vet the thing in the first place. That's not always trivial if you care about packaged licens…

I think "tools that build tools" is a very different ecosystem from most others. When you pull in something like e.g. LLVM, you're not really having to deal with any of that. It's a clean bolt-in API, freely-licensed, no weird integration points, with many active users who've sanded off any weird expectations.

I think the difference is that something like log4j has no pressure to be "the least power it can while doing the biggest job it can" — it has mostly enterprise customers who want the power, but don't care about the scope creep. Something like LLVM, on the other hand, has both big enterprise use-cases, and tiny embedded use-cases, with big players invested in both. So its size and scope have been much more well-considered, with any components that do something other than "compile these data structures representing inputs, into these other data structures representing outputs" having been long factored out into additional "plugin" sort of downstream-dependency libs.

Post reply on HN