Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear sir, you have built a compiler

rachitnigam.com

Post reply on HN
Print view View original

151–160 of 177 posts

Re: Dear sir, you have built a compiler

#151

Post by TillE »

jimmaswell wrote: post #135
DoctorDabadedoo wrote: post #118

Earlier quoted context omitted.

Because if it succedds, it needs to be supported long term. If you're doing that inhouse, chances are slim of original designers staying 10-20 years at the same company. Then what you have in your hands is a legacy monolith neglected over its life which might double as core of your stack and it's awful to move away from down the line. Been there, don't recommend it.

In my experience it's fine if you keep its scope limited. The scripting language for an MMO I worked on for example is simple and effective.

Writing your own scripting language is really not something game developers should be doing for the past decade or so. There are so many good off-the-shelf options ranging from Lua to C#, plus a litany of obscure (but good!) ones.

For performance-critical code, even a tiny DSL is not a solution I like unless you're transpiling to C++ or something.

Re: Dear sir, you have built a compiler

#152

Post by derefr »

joatmon-snoo wrote: post #141
derefr wrote: post #69

Earlier quoted context omitted.

> and then it's probably better to commit and learn the ins and outs of that specific problem domain. Even more, it's probably better to start off with a full understanding of these systems, so you can recognize when you're Greenspunning one of these abstractions, and just switch to using a robust off-the-shelf implementation of the same abstraction. (E.g. switch from writing a hand-rolled parser, to using a real par…

The problem, of course, is that it's impossible to know everything :) IME the best compromise I've found is to learn enough to recognize the problems, so that when I implement a bastardized compiler, I'm aware that that's what I'm doing and that at some point in the future I'll need to decide how deep the implementation should go or switch to actually using some library.

I somewhat disagree — there really is a finite amount of Computer Science to know. The more you dive deep into different domains, the more you find yourself ending up in the same places, learning the same things with different names. What is a SQL query planner but a compiler? Is an OS scheduler really that different from an HPC workload manager, or your GPU's computer-shader arbitrator? Is RDMA against an NVMe storage pool fundamentally new to you if you've already written hierarchical chunk persistence+caching for a distributed key-value store? Etc.

At some point, it's all just Computer Science. And not too much of it, either! An amount that could easily be learned in a (good) 4-year undergraduate program.

Re: Dear sir, you have built a compiler

#153

Post by Karunamon »

cbsmith wrote: post #129
Karunamon wrote: post #41

Earlier quoted context omitted.

Because the complexity of the problem tends to scale to your willingness to address said complexity. In other words, it likely won't stay small for long if it gets any users, and now you're the maintainer for a tool used by others.

That principle is not specific to a DSL, so that hardly seems like a reason to avoid a DSL.

I'd say it's more of a risk with languages and libraries (things that get deeply embedded into other people's stuff), but I get what you're saying.

Re: Dear sir, you have built a compiler

#155

Post by prionassembly »

cjbgkagh wrote: post #144
prionassembly wrote: post #130

Earlier quoted context omitted.

So like a restricted subset? That's a really neat idea; you should develop it into a paper or something. This restricted subset concept has the huge advantage that you can slip out of the harness in a pinch. So you can reuse a relatively developed Float-less Python program in Full Python and later walk back removing floats. That's really clever. It essentially obviates the whole configuration DSL tango; it also gets…

You can restrict all sort of things, including of course a restricted subset of the language. Such solutions already have headcount at MS which is about as good as it gets for bringing a new thing like this to masses. When it is released and as it matures we can extend it to do a bunch of other cool things like configuration DSL which would really help clean up Linux. Maybe even build scripts. You could do stuff like…

Is there a name for this type of technique I can google?

Re: Dear sir, you have built a compiler

#156

Post by munificent »

mcguire wrote: post #132
munificent wrote: post #120

Earlier quoted context omitted.

I would also add a build system to that list, a tarpit many engineers have fallen into. > not sure that there is a difference to compiler as stated in the article It's hugely different. If you're building a compiler for an existing language, you inadvertently signed yourself up for implementing and supporting the entire language, which is a pretty large engineering task. But if you design a new language, you signed u…

" Documenting it so that others can actually learn it. Technical writing is another hard task that takes years to master. " Some of us might argue that you have to do that anyway, for any software you write. In fact, a similar argument might apply to your first point. On the other hand, the last three points are really not requirements unless you are creating a general purpose language and have a large audience.

> Some of us might argue that you have to do that anyway, for any software you write.

Yes, but if your software is another implementation of an existing language, the burden is an order of magnitude smaller because almost all of the existing docs for that language apply to your implementation too.

> On the other hand, the last three points are really not requirements unless you are creating a general purpose language and have a large audience.

Even if you have a small audience, if you want them to be productive, they need good tools and resources. Part of the reason why it's hard for DSLs to be effective is that most of the resources get amortized across all users. When you have a small audience, you can't justify building all these tools and docs, but the users still need them and their productivity suffers in the absence.

If your audience is too small, I think it often just doesn't make sense in terms of cost to make a language for them. You're usually better off using an existing language that already has those resources you can leverage.

Re: Dear sir, you have built a compiler

#157

Post by bmh100 »

Communitivity wrote: post #90

I feel like this easily could have been addressed to a much younger version of myself on a particular project more than a decade ago (~12 years). My story happened in the early days of SPARQL, when SPARQUl (SPARQL Update Language) was very new, a draft, and definitely not merged into SPARQL yet. Oh, and dinosaurs roamed the earth. That last is from my daughter. My project involved semi-autonomous software agents that…

The nerve and brain concept sounds fascinating, but I don't have enough background in sensors to quite grasp what you mean. Could you please provide a specific example?

Re: Dear sir, you have built a compiler

#158

Post by tester756 »

jerf wrote: post #32
olivierduval wrote: post #3

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

The easy options: As others have said, embed something existing. I think this is especially true for programming languages. Even if Lua isn't quite what you want, it's probably going to deliver a lot more value than anything you could bash together on your own. And there are a few other options as well. There are several of these "complete toolkits for DSLs", it's just they do take a bit of work to correctly insert i…

>(Hint: Just because your compiler accepts JSON as input doesn't mean that that's what the humans have to directly output or edit.

Ah, the JSON_IR!

Re: Dear sir, you have built a compiler

#159

Post by leeoniya »

derefr wrote: post #152
joatmon-snoo wrote: post #141

Earlier quoted context omitted.

The problem, of course, is that it's impossible to know everything :) IME the best compromise I've found is to learn enough to recognize the problems, so that when I implement a bastardized compiler, I'm aware that that's what I'm doing and that at some point in the future I'll need to decide how deep the implementation should go or switch to actually using some library.

I somewhat disagree — there really is a finite amount of Computer Science to know. The more you dive deep into different domains, the more you find yourself ending up in the same places, learning the same things with different names. What is a SQL query planner but a compiler? Is an OS scheduler really that different from an HPC workload manager, or your GPU's computer-shader arbitrator? Is RDMA against an NVMe stora…

> An amount that could easily be learned in a (good) 4-year undergraduate program.

i think any kind of undergrad program can only ever scratch the surface of some of these topics. people spend decades becoming proficient in just a single vertical (microchip architecture, distributed systems, asics/fpgas, signal processing, gpu shaders, network engineering, jit compilers, language design, color spaces, database engines, etc).

sure, it's all "just" CS/EE :D

look at how much knowledge/skill was required to make an even faster fizbuzz: https://news.ycombinator.com/item?id=29031488

Re: Dear sir, you have built a compiler

#160

Post by cjbgkagh »

prionassembly wrote: post #155
cjbgkagh wrote: post #144

Earlier quoted context omitted.

You can restrict all sort of things, including of course a restricted subset of the language. Such solutions already have headcount at MS which is about as good as it gets for bringing a new thing like this to masses. When it is released and as it matures we can extend it to do a bunch of other cool things like configuration DSL which would really help clean up Linux. Maybe even build scripts. You could do stuff like…

Is there a name for this type of technique I can google?

For C# there is Roslyn Analyzers which was a foot in the door for a lot of things. F# has LiveCheck which AFAIK is a lot more powerful. I don't think there is much public info out there; Don Syme demos the use of it to add shape checking for Deep Learning models with full IDE interaction, but any arbitrary constraints can be added and constraint solvers are probably the easiest way to manage and compose those constraints. https://youtu.be/0DNWLorVIh4?t=3410
Post reply on HN