An update on Dart macros and data serialization • Hacker News Board

Re: An update on Dart macros and data serialization

#81

Post by ajross » Thu, Jan 30, 2025, 12:13 AM UTC

elcritch wrote: post #49

ajross wrote: post #5

Maybe it's time to just recognize that lisp-style macros-as-language-syntax features just aren't worth the struggle and grief? The big metaprogramming feature traditionally implemented in macros, type generation, is already provided in some form by all major languages already. And an awful lot (and I mean an awful lot) of good work can be done at the string replacement level with cpp. And generating code upstream of…

Oh how I enjoy trying to compile and use projects where they use some complex home brew codegen system often written in a different language entirely [1]. Luckily they often use Python as part of some core build step which never breaks compatability in their regex librwry [2]. Yes macros can be a pain and should be limited, but in my experience, a couple hundred lines of macros replaces many thousands of lines code g…

Bad code is bad code, but FWIW every time I've seen something that actually needs a "complex home brew codegen system", it's absolutely best expressed in a dedicated piece of software engineering and not within a macro environment. Zephyr leans heavily on that kind of engineering, actually.

The point is really that "macros" is a weird sandwich between "complicated metaprogramming you need to do from first principles" and "you really didn't need metaprogramming, did you?". And that over the years that sandwich has been getting thinner.

Lisp macros in the 60's were a revelation. They don't really have a home anymore.

Re: An update on Dart macros and data serialization

#82

Post by miffy900 » Thu, Jan 30, 2025, 12:31 AM UTC

eseidel wrote: post #35

zigzag312 wrote: post #17

They should have just use C# for Flutter. Without investing significant time, like they did with Dart, they would have a language with a much bigger ecosystem that is faster, already has compile time code generation and better support for data than Dart. It supports ahead-of-time compilation and hot-reload. The only feature missing in C# is compilation to JS, but with WASM is that really needed? Biggest downside of C…

[Flutter founder here.] I'm pretty sure we did look at C# (and certainly a whole bunch of other languages). I don't actually recall why we didn't use C# at the time. I remember Go binaries were waaay to big, JS (what we originally wrote Flutter in) startup time was way too slow on iOS, Swift was too deeply tied to Apple (the standard library was closed source at that time), etc. It's possible that C# was too verbose…

I don't think C# would've been a good fit for flutter in its early days, but definitely Flutter should no longer be tied to a single language like Dart. I think you should definitely consider stronger cross-language support.

Re: An update on Dart macros and data serialization

#83

Post by ollysb » Thu, Jan 30, 2025, 12:45 AM UTC

leecommamichael wrote: post #15

Good. Dart already has good support for code-generation. It would just encourage package authors and app developers to waste their time golfing.

It is incredibly slow though. I have a project with 40k lines of code which takes a minute to generate on an m1. It's a far cry from incremental compilation. It's enough that I generally avoid adding anything new that would require generation.

Re: An update on Dart macros and data serialization

#84

Post by bickfordb » Thu, Jan 30, 2025, 12:51 AM UTC

ajross wrote: post #5

Maybe it's time to just recognize that lisp-style macros-as-language-syntax features just aren't worth the struggle and grief? The big metaprogramming feature traditionally implemented in macros, type generation, is already provided in some form by all major languages already. And an awful lot (and I mean an awful lot) of good work can be done at the string replacement level with cpp. And generating code upstream of…

They seem to work well in Rust.

Re: An update on Dart macros and data serialization

#85

Post by soegaard » Thu, Jan 30, 2025, 1:06 AM UTC

_moof wrote: post #14

anothername12 wrote: post #6

Why is it that Lisp family macros are easy to implement and use, but not so in other languages?

This is just my opinion but I believe it's because the syntax tree is the syntax. In a Lisp macro you are working with lists, just like you are for any other Lisp code. Almost every other language I've used (I've been programming since the late 1980s) has, at best, a special data structure to manipulate ASTs. So it ends up being quite unnatural. Lisp macros are just Lisp.

> This is just my opinion but I believe it's because the syntax tree is the syntax.

This holds for "old" lisps. There are other options. Racket and Scheme uses "syntax objects". Syntax objects contain besides the old syntax tree also source location information and lexical information.

See for example the last part of:

https://parentheticallyspeaking.org/articles/bicameral-not-h...

Re: An update on Dart macros and data serialization

#86

Post by CharlieDigital » Thu, Jan 30, 2025, 1:14 AM UTC

whoknowsidont wrote: post #77

CharlieDigital wrote: post #69

Earlier quoted context omitted.

C# is a fantastic language that has, in recent years, evolved very quickly for the better. Nice mix of object and functional drawing a lot of influence from F#. It shares a lot of language constructs with TypeScript (and by extension, JS) and has been converging with each release so I'm often surprised that people hate on it or that more startups don't reach for it if they are on Node with TS. Same syntax for key lan…

The improvements are great for people who have to or want to use C# for whatever reason? But how does that move the needle from other tech-stacks that are for more capable, especially on non-Windows environments (and please don't imply that C# is truly cross-platform, it's fine for web API's, it's not fine when dealing with actual system calls). If you're within the Windows garden, those tools certainly make sense to…

I'm not advocating it for every use case, I'm advocating it for use cases where it's a good fit (e.g. web APIs, backends where there's a need for multi-threaded code). For teams that need to move up from Node/JS/TS or augment Node/JS/TS, it's likely a better choice than say Rust or Go (nothing wrong with Go, but C# is going to be an easier ramp than Go for JS/TS devs IMO)

I would not, for example, advocate it for web UIs or any UIs except for Windows desktop UIs (and even there, I might advocate for JS based options).

Re: An update on Dart macros and data serialization

#87

Post by smt88 » Thu, Jan 30, 2025, 1:14 AM UTC

CharlieDigital wrote: post #71

smt88 wrote: post #50

Earlier quoted context omitted.

I'm not going to log in to Xitter to read the rest of his tweets, but it doesn't seem like Miguel regrets using C# or is particularly in love with Dart. He's very complimentary of the goals and governance of Flutter, which is certainly more important than a language choice between two respectable languages. I do think C# is by far the best mainstream language, but good IDE support and library ecosystem are the dealma…

> I do think C# is by far the best mainstream language C# is a hugely underrated language that I feel like often gets overlooked when teams look to move beyond JS/TS. The language has a pretty tight syntactic congruency to JS/TS[0], Entity Framework is pretty amazing in terms of DX/perf/maturity, and it seems like we should see more C#/.NET it in the wild than we actually do. My sense is that there are some legitimat…

JVM probably has a better ecosystem, so Kotlin wins in that regard. I would also say there's a huge and painful difference between C# and TypeScript, which is that C# has nominal typing (no equivalency between types unless they're literally the same) and TS has structural/duck typing (if the members of two objects are the same, they're equivalent).

That makes sense because C# can be much lower level and has its own set of priorities during compilation, so I'm not really complaining. But ergonomically, you really miss the TS type system when you don't have it.

Re: An update on Dart macros and data serialization

#88

Post by CharlieDigital » Thu, Jan 30, 2025, 1:26 AM UTC

smt88 wrote: post #87

CharlieDigital wrote: post #71

Earlier quoted context omitted.

> I do think C# is by far the best mainstream language C# is a hugely underrated language that I feel like often gets overlooked when teams look to move beyond JS/TS. The language has a pretty tight syntactic congruency to JS/TS[0], Entity Framework is pretty amazing in terms of DX/perf/maturity, and it seems like we should see more C#/.NET it in the wild than we actually do. My sense is that there are some legitimat…

JVM probably has a better ecosystem, so Kotlin wins in that regard. I would also say there's a huge and painful difference between C# and TypeScript, which is that C# has nominal typing (no equivalency between types unless they're literally the same) and TS has structural/duck typing (if the members of two objects are the same, they're equivalent). That makes sense because C# can be much lower level and has its own s…

We're not there yet, but if you squint, named tuples kinda interesting because it's also shape-based: https://www.reddit.com/r/csharp/comments/164w8l1/how_do_yall...

Re: An update on Dart macros and data serialization

#89

Post by munificent » Thu, Jan 30, 2025, 1:54 AM UTC

Hi, I work on Dart and was one of the people working on this feature. Reposting my Reddit comment to provide a little context:

I'm bummed that it's canceled because of the lost time, but also relieved that we decided to cancel it. I feel it was the right decision.

We knew the macros feature was a big risky gamble when we took a shot at it. But looking at other languages, I saw that most started out with some simple metaprogramming feature (preprocessor macros in C/C++, declarative macros in Rust, etc.) and then later outgrew them and added more complex features (C++ template metaprogramming, procedural macros in Rust). I was hoping we could leapfrog that whole process and get to One Metaprogramming Feature to Rule Them All.

Alas, it is really hard to be able to introspect on the semantics of a program while it is still being modified in a coherent way without also seriously regressing compiler performance. It's probably not impossible, but it increasingly felt like the amount of work to get there was unbounded.

I'm sad we weren't able to pull it off but I'm glad that we gave it a shot. We learned a lot about the problem space and some of the hidden sharp edges.

I'm looking forward to working on a few smaller more targeted features to deal with the pain points we hoped to address with macros (data classes, serialization, stateful widget class verbosity, code generation UX, etc.).

Re: An update on Dart macros and data serialization

#90

Post by munificent » Thu, Jan 30, 2025, 1:57 AM UTC

kazinator wrote: post #65

> Semantic introspection, unfortunately, turned out to introduce large compile-time costs which made it difficult to keep stateful hot reload hot. They must have done something wrong. Macros are expanded when you ahead-of-time compile your code, which doesn't take place in the run-time environment where you hot load, but in the build environment. It doesn't matter whether the macro are simple, or whether they can ins…

> Macros are expanded when you ahead-of-time compile your code, which doesn't take place in the run-time environment where you hot load, but in the build environment.

The user experience with hot reload is:

1. They hit "run".

2. The compiler compiles the app.

3. The app starts running on their device.

4. They change some code in their IDE.

5. They click "hot reload".

6. The compiler compiles the changed code.

7. The IDE sends the updated code to the running app.

8. The runtime loads the changed code.

9. They see the changed behavior in their running app.

Steps 6-8 determine the total time between "user requests a hot reload" and "user sees their updated app". Compilation doesn't happen on the device, but it still takes time and is in the critical path for that experience.

Making the compiler slower makes hot reload slower. We measure hot reload time in milliseconds, so it doesn't take much for us to consider it an unacceptable performance regression.