Live data from Hacker News

Explicit term inference with Scala 3

scala-lang.org

61–70 of 77 posts

Re: Explicit term inference with Scala 3

#61
post #31

Earlier quoted context omitted.

In no particular order: - the JVM - the tooling for writing with VSCode - SBT - NullPointerExceptions bubbling up from Java libs - the split between the functional programming community and the "Scala is just Java without ;" one - everything related to implicits (creation, resolution, etc) and everyone using them - the ability to write the same 0-100 loop with 4 different keywords and 3 different styles - the inabili…

I wrote Scala for three years or so professionally and agree with most of your points. A couple of comments though that haven’t been addressed by the sibling commenter who also addressed some of your points: - JVM is JVM I guess. You either love it or you don’t. I personally really enjoyed the access to the ecosystem so that was a major boon for me. - Gradle is just so much better than SBT in almost every way aside f…

We have recently switched from SBT to Gradle and I'm finding Gradle a lot more painful than SBT.

For starters, SBT had a nice interactive shell, while Gradle takes quarter minute just to list the available tasks, after which it's another quarter minute to get anything else done...

Re: Explicit term inference with Scala 3

#62
post #31

Earlier quoted context omitted.

In no particular order: - the JVM - the tooling for writing with VSCode - SBT - NullPointerExceptions bubbling up from Java libs - the split between the functional programming community and the "Scala is just Java without ;" one - everything related to implicits (creation, resolution, etc) and everyone using them - the ability to write the same 0-100 loop with 4 different keywords and 3 different styles - the inabili…

I wrote Scala for three years or so professionally and agree with most of your points. A couple of comments though that haven’t been addressed by the sibling commenter who also addressed some of your points: - JVM is JVM I guess. You either love it or you don’t. I personally really enjoyed the access to the ecosystem so that was a major boon for me. - Gradle is just so much better than SBT in almost every way aside f…

> Gradle is just so much better than SBT in almost every way aside from simplicity.

I picking this up, because it was also mentioned in a sibling comment. Let's continue this discussion.

It doesn't matter if we have multiple alternative to SBT, some more usable than other. Rust has one package manager (+build tool +distribution tool): Cargo. That's it. Whatever the project you just cloned and whatever the platform you're using, `cargo build` is going to compile it. Now, if every Scala project starts using a different package manager, that's going to be a problem. The best solution would have been to make SBT usable without a prescription from your doctor. It didn't happen. That's okay, programming languages are just tools anyway, and they get replaced too.

Re: Explicit term inference with Scala 3

#63
post #22

Earlier quoted context omitted.

> SBT is easy to use at the beginning, simply because the build files are 1/10 the length of a Maven XML build description, the console output looks better, the build REPL has lots of commands that make things easier, and the simple projects just work fine. The build REPL has lots of commands that aren't documented, and when you search for tutorials the commands have changed (e.g. runMain is apparently now run-main,…

SBT's deeper problems are the 4-dimensional data model and the 3-layer meta-interpretation model. Probably not the best option for really complicated builds, but it's great for simple builds and I appreciate the tireless open source work the folks do to keep it maintained!

Those deeper problems are describe here https://www.lihaoyi.com/post/SowhatswrongwithSBT.html

Re: Explicit term inference with Scala 3

#64
post #53
post #33

Earlier quoted context omitted.

- compilation time

Seriously. I modify a single line, in a single file, and the scala system takes 2-3 minutes to recompile at work. Apparently it spends over a minute inferring types! Hello! The types are EXACTLY THE SAME AS ALWAYS. Please cache the whole typing phase as a build artifact based on file/directory hashes. Maybe even make this cache source-control-safe.

Upgrade your sbt version and make sure that whatever file you update is not (transitively) imported/used by every other file. If that doesn't help, check for excessive macro/typelevel usage.

Incremental compile times should be seconds not minutes, something is probably wrong with your setup.

Re: Explicit term inference with Scala 3

#65
post #22

Earlier quoted context omitted.

> SBT is easy to use at the beginning, simply because the build files are 1/10 the length of a Maven XML build description, the console output looks better, the build REPL has lots of commands that make things easier, and the simple projects just work fine. The build REPL has lots of commands that aren't documented, and when you search for tutorials the commands have changed (e.g. runMain is apparently now run-main,…

"runMain" is still "runMain", I don't know where that comes from. There are lots of available commands indeed, due to the nature of SBT, but a basic workflow always uses the same commands: compile, test, run. Any extra tasks used usually come from plugins, just like in Maven. And reading documentation is needed for configuring and running those, just like in Maven. Projects should document why each plugin is included…

> "runMain" is still "runMain", I don't know where that comes from.

Ah, I got it backwards: old tutorials say to use "run-main", and if you do run-main now you get "Expected ';'".

> There are lots of available commands indeed, due to the nature of SBT, but a basic workflow always uses the same commands: compile, test, run. Any extra tasks used usually come from plugins, just like in Maven. And reading documentation is needed for configuring and running those, just like in Maven.

But in Maven you don't need to read any documentation if you're just building the project, because every project has the exact same lifecycle. If you want to add or remove plugins to make the build do something different then you need to read the documentation of those plugins, sure, but you don't have to know anything about the plugins if you're just working on the code. E.g. if an SBT project is packaged for docker then you'll have to run some docker-plugin-specific command to do it (and you've got no way to discover which submodule you're supposed to run it in), whereas if a maven project is packaged for docker then you just run "mvn deploy" as usual.

> And if the common settings are defined within the parent project, it stops being self-contained. SBT accepts this reality and that's why multimodule projects are a thing.

Parent projects work consistently whether or not you're using a multimodule project, which is another problem with SBT - it's really confusing to share parts of a build definition between more than one project, to the point that most people copy/paste instead. With maven you can do a very natural, gradual progression from single module -> multi-module project -> multi-module project where the parent is its own module -> independent projects using a shared parent (e.g. an organisation-level parent). It's another case of a hierarchy working much better than a grid.

> By the way: creating a separate integration test config in Maven is painful, up to the point that people recommend to just add a new subproject that contains them. Yay, more nesting!

Think of the people who come to join your project! I've seen SBT modules with 5 different scala source directories and no obvious relationship between what depends on what, and good luck getting any IDE to understand whether test depends on integration-test or vice versa (most will give up and just build everything together, which is fine until you add something that works in the IDE and then errors when you build it with SBT).

A separate submodule is a much better approach - tools and people are much better at handling "module A depends on module B" than "this source folder depends on that source folder".

Re: Explicit term inference with Scala 3

#66

> Avoiding repetition with contextual parameters This is the type of stuff that made me go away from Scala. Why implement a feature that optimises for code writing? How does my editor let me know that function receives that parameter without me going "???" and having to go into its definition? Implicit conversions fall into the same category, of making code pretty to look at, quick to write, and a nightmare to unders…

It's not meant to replace the stuff that you'd use a normal function call for, it's meant to replace the stuff that you'd do completely invisibly.

Every real-world Java codebase ends up using massive amount of incomprehensible magic for e.g. DI, transaction management, web request mapping, serialisation. Every real-world Python/Ruby codebase ends up using magic proxies, metaclasses, method_missing or other such stuff. Scala is the only language where I've been able to find real, enterprise-scale codebases written in 100% plain old code without any AOP-style magic, because the implicits let the language be expressive enough that you can avoid those things without your business logic getting drowned in secondary concerns.

Re: Explicit term inference with Scala 3

#67

A summary of what Scala is great for: - Safe and high performant concurrent programming. Scala is miles ahead of most other languages here and pretty much on par with the best ones (such as Haskell) - ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. - Actor systems…

>ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. I disagree. And I've done a lot of Scala ETL work. The issue is that in Scala ETL work your data structures are either untyped (Spark Dataframe), case classes or typed tuples. If they're untyped then there's little t…

> they're case classes then you're moving around a lot of unnecessary data fields and you've got the overhead of making fifty intermediate case classes yourself.

Haskell developer here... What about case classes and lenses? Do they solve this?

Re: Explicit term inference with Scala 3

#68

Earlier quoted context omitted.

>ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. I disagree. And I've done a lot of Scala ETL work. The issue is that in Scala ETL work your data structures are either untyped (Spark Dataframe), case classes or typed tuples. If they're untyped then there's little t…

> they're case classes then you're moving around a lot of unnecessary data fields and you've got the overhead of making fifty intermediate case classes yourself. Haskell developer here... What about case classes and lenses? Do they solve this?

As I understand it lenses don't change the underlying data structure. For ETL you need a way to basically say "the code only uses fields X, Y and Z so we will only load X, Y and Z during runtime." Automatically based on usage without having to keep updating your lens definition. Modern on disk file formats are columnar so they can very efficiently read subsets of the data. If your data has 200 columns than reading the 199 unnecessary ones can be very slow.

They could help with the intermediate data structure but some of them aren't subsets or trivial derivatives. So you really need an inline way to create single use case classes. I think frameless in Scala can do some of this for standard transformations but that requires the black magic of shapeless.

Spark in Python (and the untyped Dataframe API in Scala) compiles everything internally before running it to achieve the above. So it's trivial to have unit tests on empty data structures which "type check" your Spark code.

Re: Explicit term inference with Scala 3

#69
post #53

Earlier quoted context omitted.

Seriously. I modify a single line, in a single file, and the scala system takes 2-3 minutes to recompile at work. Apparently it spends over a minute inferring types! Hello! The types are EXACTLY THE SAME AS ALWAYS. Please cache the whole typing phase as a build artifact based on file/directory hashes. Maybe even make this cache source-control-safe.

I am curious if you are using SBT incremental builds? (putting aside how hard it can be to get and keep SBT working for a project ... I was the guy that had to do that so I know that pain quite well). Because in my experience with really large scala projects that should not be the case, unless you modify a file at the very top of the dependency tree. We used what I would call the package object predef pattern, which…

Well, neither I, nor anybody at the startup I work at apparently (it would seem) has been able to figure it out.

Out of curiosity, how many hours do you think it would take to get the compile time down? I wonder if we could find somebody to help us with that.

Re: Explicit term inference with Scala 3

#70

A summary of what Scala is great for: - Safe and high performant concurrent programming. Scala is miles ahead of most other languages here and pretty much on par with the best ones (such as Haskell) - ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. - Actor systems…

>ETL / data transformations. Python is a big player here - but for stable and performant data pipelines, I strongly believe Scala is the better choice. For explorative things, python has the edge though. I disagree. And I've done a lot of Scala ETL work. The issue is that in Scala ETL work your data structures are either untyped (Spark Dataframe), case classes or typed tuples. If they're untyped then there's little t…

Another possibility is to use "typesafe heterogenuous containers" as described in Joshua Bloch, "Effective Java", 2nd edition and probably 3rd edition. The idea is to use maps with specially constructed keys, that give you type safe access to their accompanied values. That is better than completely untyped, you have a place to document stuff at the declaration of those keys and you can easily create runtime checks, that check for the presence of a set of well defined keys in the map. The fact that they are described in a Java book does not mean that they aren't a nice pattern in Scala and several other statically typed language with generics.
Post reply on HN