Live data from Hacker News

How Microsoft rewrote its C# compiler in C# and made it open source (2017)

medium.com

11–20 of 136 posts

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#11
post #8

I find amusing how microsoft moved from the closed source referent in the industry into such an open source player. Right now even allows to hook some of their tools and platforms to its competing platforms and tools.

I think they are really repositioning the company as powering general computing through their cloud platform as opposed to powering general computing through their OS.

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#12
post #8

I find amusing how microsoft moved from the closed source referent in the industry into such an open source player. Right now even allows to hook some of their tools and platforms to its competing platforms and tools.

I think they are really repositioning the company as powering general computing through their cloud platform as opposed to powering general computing through their OS.

I think they will slowly turn into IBM. Maybe a better version of IBM though.

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#13
post #5

I still can't believe more people aren't leveraging the Roslyn APIs to write compiler extensions or additional tools around C#. It's conceptually powerful.

Personally speaking, I'd love to, but I'd then have to figure out how to make them work with an IDE. I'm very tired of waiting for record classes to show up and I could have written a (to be clear: inferior ) set of stuff around regular classes that magics one into "everything is readonly and we autogenerate a `copy` method", much like Kotlin does for its data classes...but my IDE isn't gonna understand it unless I d…

I'm not sure that it would be that difficult with Roslyn and Visual Studio these days. There are Roslyn-based syntax-highlighters and linters and snippet-generators and code-transformers. I use one for making sure all of my code is not just formatted the way I want it, but auto-inserting "readonly" modifiers for fields that don't get re-written outside of the constructor, one that treats not implementing IDisposable correctly as an error (it can also track the lifetime of a Disposable object and warn when it detects that it never gets disposed, which is super cool), and another one that rainbow-highlights code blocks. The tooling is there to support just a thing, it just needs someone to put all the pieces together.

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#14

I still can't believe more people aren't leveraging the Roslyn APIs to write compiler extensions or additional tools around C#. It's conceptually powerful.

At some point I wrote a barebones scripting system using roslyn for a project of mine. At runtime I got a piece of code compiled to a DLL in memory and then executed this DLL; Worked well; But at that time there was no support for destroying AppDomains or something, don't remember the exact name. Still pretty fun; But yeah, there's no real complete documentation anywhere; And the DLL hell was real. Dozens of DLL's ju…

DLL Hell doesn't refer to "lots of DLLs", it refers to conflicting versions of DLLs not being easily manageable across multiple applications. Outside of putting assemblies in the GAC (which was always a hack of last resort anyway), .NET has never had "DLL Hell".

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#15
post #6

I still can't believe more people aren't leveraging the Roslyn APIs to write compiler extensions or additional tools around C#. It's conceptually powerful.

I looked at bit when I was in preview. It’s indeed powerful but also quite verbose and it distinguish between a lot of concepts, so there is a steep learning curve. Then there was the lack of examples beyond a few blog posts.

Oh man, the verbosity thing was a huge problem until C# got "using static". Any static classes that just hold static methods can be imported into your code module as bare functions. I frequently do "using static System.Math;" and "using static System.Console;" when tossing together little mathy processor apps.

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#16

I still can't believe more people aren't leveraging the Roslyn APIs to write compiler extensions or additional tools around C#. It's conceptually powerful.

At some point I wrote a barebones scripting system using roslyn for a project of mine. At runtime I got a piece of code compiled to a DLL in memory and then executed this DLL; Worked well; But at that time there was no support for destroying AppDomains or something, don't remember the exact name. Still pretty fun; But yeah, there's no real complete documentation anywhere; And the DLL hell was real. Dozens of DLL's ju…

Roslyn is very powerful, but it is still pretty cumbersome. At one point I spent quite a while trying to get a game scripting system akin to the way that Lua is commonly used working, and I just couldn't get it working fast enough to be viable. I essentially wanted to have a core C# engine that provided services, and then have it call an initialization function and a gameloop function that were defined in designated script files, and then all of my game code would also be written in other scripts; this way I could run things, edit the code on the fly, and hot-reload.

It's entirely possible that I just don't know what I'm doing well enough to do this correctly, but I just couldn't get it to do the kinds of things that I wanted from it. My sense is that it is really great for injecting custom code that is used rather infrequently. I've had good results using it for building out reporting systems that are pluggable.

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#17

I still can't believe more people aren't leveraging the Roslyn APIs to write compiler extensions or additional tools around C#. It's conceptually powerful.

At some point I wrote a barebones scripting system using roslyn for a project of mine. At runtime I got a piece of code compiled to a DLL in memory and then executed this DLL; Worked well; But at that time there was no support for destroying AppDomains or something, don't remember the exact name. Still pretty fun; But yeah, there's no real complete documentation anywhere; And the DLL hell was real. Dozens of DLL's ju…

Maybe Assembly unloading, in which case if you wanted to swap out new dlls in memory you'd have to tear down the process and restart it. Definitely not sexy. That being said: I think Assembly Unloading is a thing now (and AppDomains don't exist in .NET Core last I heard).

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#19

Why isn't the native code compiler also now written in C#, like Java is doing?

People have done research before into having the C#->native JIT be written in C#, I recall seeing prototypes. There are a ton of barriers between a prototype and a shipping implementation though, so I'm not sure we'll ever see it. In particular you risk regressions in startup time or memory usage since the amount of infrastructure needed to run C# (pre-jitted?) to generate all your jitcode is much higher than a small blob of hand-written C/C++ that's spitting out jitcode. See https://www.mono-project.com/news/2018/09/11/csharp-jit/ for one example.

There's an old saying (from one of Unity's developers, I think?) that you can't run Hello World in C# or Java without an XML parser because so much configuration for things like locales ends up pulling in serialization libraries, reflection, etc. Things have improved in this area for both platforms but it's definitely still very difficult to trim managed code down as far as you can trim native code.

For something as critical as the JIT you also want to keep the generated code small (for cache efficiency) and if possible, PGO it - things that existing managed code generators aren't especially great at compared to clang or modern MSVC.

From my past experience writing/maintaining C# compiler and runtime code, I would never bother trying to port the JIT to C#. I don't think the returns are worth the massive investment. I'd sooner port it to a language like Rust for safety benefits or to some sort of hypothetical language that enables producing smaller/faster code to improve JIT performance. I'm not sure I'd invest in that either though because for most workloads JIT time is not the bottleneck (and you can optimize that out in many cases by pre-generating the JITcode). EDIT: Also, for workloads where JIT is the bottleneck, it's questionable whether it's possible to extract big gains out of optimizing the JIT because of the nature of JIT workloads - you may just be bottlenecked on memory bandwidth or instructions per clock. A JIT converting IR to machine code is not trivially vectorized.

People are figuring this out (again, from scratch) now with webassembly as all the modern browsers go through the churn and angst involved in answering the question 'can we actually JIT this whole 50mb executable from scratch at load time?' even though we already knew the answer back when WebAssembly wasn't a spec yet.

[DISCLOSURE: I get paid to work on Mono right now and was paid to help draft the initial WebAssembly spec, and before that my work on the JSIL MSIL->JS compiler was sponsored. So I have some massive biases here.]

Re: How Microsoft rewrote its C# compiler in C# and made it open source (2017)

#20

Earlier quoted context omitted.

At some point I wrote a barebones scripting system using roslyn for a project of mine. At runtime I got a piece of code compiled to a DLL in memory and then executed this DLL; Worked well; But at that time there was no support for destroying AppDomains or something, don't remember the exact name. Still pretty fun; But yeah, there's no real complete documentation anywhere; And the DLL hell was real. Dozens of DLL's ju…

Roslyn is very powerful, but it is still pretty cumbersome. At one point I spent quite a while trying to get a game scripting system akin to the way that Lua is commonly used working, and I just couldn't get it working fast enough to be viable. I essentially wanted to have a core C# engine that provided services, and then have it call an initialization function and a gameloop function that were defined in designated…

I think ignorance is a pretty powerful argument to be made by any developer wanting to use Roslyn. The last few times I messed with it there was literally no documentation and everything I knew/know comes from reading headers, trial and error, and experimentation.

I feel defensive about calling it cumbersome though, and I can't imagine why something like your LUA vision isn't possible (though, I've never tried I just assumed someone would inevitably do this). For example, if World of Warcraft were to switch out their UI LUA extension system with C# I could totally imagine this being possible (though it'd be suicidal for their mod community). Likewise, if Unity were to begin using it for this kind of thing (if they don't already) I'd imagine it is possible.

Post reply on HN