Reflections on using Typelevel Scala

At some point, Typelevel decided to fork Scala in order to test feature that it’s members found useful and get feedback on these features without the need for waiting for next official Scala release. Latest such version was Typelevel Scala 4 based on Lightbend Scala 2.12.4, which I decided to use at some point. Now, that everyone is migrating on 2.12.6 I can tell: what TL Scala give me for all that time?

Reasons for migration

Typelevel Scala 4 is (almost) a drop-in replacement for Scala 2.12.4. (There are some issues when it comes to settings up dependencies, but I’ll talk that in detail later on). This means, that I could take my project, which I compiled at a time with Scala 2.12.4 and replace the compiler, without giving up on any library that I use and without changing the code.

That change allowed me to test a few compiler flags, that sounded handy. I was mostly interested in -Yinduction-heuristics, -Ysysdef and -Ypredef.

Induction heuristics enabled some early version of a patch that in some cases (common for libraries like Shapeless) to derive typeclasses faster. Which mean, that in typeclass-heavy project it basically speeds up compilation a lot.

The other flags helps with a custom predef. Instead of importing your own predef in every single file, you can just set up one flag, and you are done. Isn’t it wonderful?

I decided to try out both improvements and see how things would work out.

Implicit speedup

At a time I certainly benefited from compilation speed up. Before the change CI build took about 30 minutes, while with induction heuristics enabled it to get down to 23-25 minutes.

Considering that this time was not spend only on building, but also on fetching dependencies, running unit, and integration tests, it was a really good result.

I don’t remember how exactly things looked in my local environment, only that the speedup was noticeable.

Custom Predef

I am still certain that my own predef is a quite good solution. Sure: Scala 2.13 will fix the issues with scala.Seq being an alias for scala.collection.Seq instead of scala.collection.immutable.Seq. Additionally, deprecated and error-prone parts of predef will be eventually removed. However, there are things that original predef will not give you (at least before Dotty).

If you have an issue with == not being type safe, (as opposed to e.g. === from Cats), or if you heavily use some type (e.g. @@) you might just as well put into predef to import all things you would always import anyway.

Removing the need to put import my.package.Predef._ at the top of every file was certainly welcome, and I used it for a while. It didn’t affect the build time much, just spared me a pain of compilation error when I was forgetful during the creation of a new file.

(That probably makes me a 3rd user of this feature right after the implementation’s author, Andy Scott, and Sam Halliday).

Hacks everywhere

Using TL was not painless. First of all, you have to modify both scalaVersion and scalaOrganization:

scalaOrganization := "org.typelevel"
scalaVersion      := "2.12.4-bin-typelevel-4"

and you must carefully override them everywhere you might need to set up Scala version.

Of course, dependencies that would rely on full Scala version would break if you haven’t change CrossVersion.full to CrossVersion.patch:

addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.1" cross CrossVersion.patch)

Some libraries would rely on Scala implicitly, and so you might get conflicts during assembly’s task’s merge if you don’t address that:

.settings(assembly / assemblyMergeStrategy := {
        case PathList("scala", _*)                       => MergeStrategy.first
        case PathList("compiler.properties")             => MergeStrategy.first
        case PathList("interactive.properties")          => MergeStrategy.first
        case PathList("library.properties")              => MergeStrategy.first
        case PathList("reflect.properties")              => MergeStrategy.first
        case PathList("repl.properties")                 => MergeStrategy.first
        case PathList("repl-jline.properties")           => MergeStrategy.first
        case PathList("scala-buildcharacter.properties") => MergeStrategy.first
        case PathList("scaladoc.properties")             => MergeStrategy.first
        case strategy                                    =>
          MergeStrategy.defaultMergeStrategy(strategy)
      })

 ("com.lihaoyi" %% "ammonite" % "1.1.2")
  .cross(CrossVersion.patch)
  .excludeAll(ExclusionRule("org.scala-lang"))

You can easily understand that it was a hacky way of enabling it, and the community is not really prepared for using non-Lightbend Scala.

IDE issues

I think Ensime might recognize -Ypredef and -Ysysdef flags, but that’s it. There is an old untouched task about introducing it to IntelliJ. Maybe once reviewed version of custom import is actually released something will change. There was a PR with implementation but then the version merged to 2.13 branch got rid of the flag.

Currently, if you want your IDE to be able to understand your code, and not insert import com.mypackage.Predef._ everywhere you might try another workaround:

package com.mypackage

trait Predef { ... }

package com
package object mypackage extends mypackage.Predef

package com.mypackage
package subpackage

...

Apparently, splitting package from one package definition to several definitions will cause scalac to import everything from each on intermediate packages (including the content of a package object). It is a pattern I first saw in scalac own source code.

Typelevel stack?

Are there some other benefits to Typelevel Scala that would justify all the trouble?

Well, there were few other flags enabled, that might not be available before 2.13, which are handy to people using types heavily. After Rob Norris:

-Yinduction-heuristics - speeds up the compilation of inductive implicit resolution
-Ykind-polymorphism - type and method definitions with type parameters of arbitrary kinds
-Yliteral-types - literals can appear in type position
-Xstrict-patmat-analysis - more accurate reporting of failures of match exhaustivity
-Xlint:strict-unsealed-patmat - warn on inexhaustive matches against unsealed traits

Kind polymorphism is useful once you start working with higher-kinded types with libraries like Cats or Scalaz. Literal types might change quite a lot when it comes to type-level programming. And better errors about non-exhaustive pattern matching are always welcome.

I am certain that bleeding edge Typelevel libraries benefited from a version of the compiler where certain bugs could be fixed much much earlier than by waiting for a 2.12.5 release.

Summary

Faster compilation and no need to import custom predef everywhere was a good enough reason for me to struggle for a while with Typelevel Scala 4.

However, a few weeks ago I backed down on using -Ypredef since it really complicated working with IDE. IntelliJ didn’t recognize types that were there, I had to add exceptions to optimize imports list, etc.

When I was writing this post I also decided to check how Typelevel Scala 4 compares to current Scala 2.12.6. On my computer TL4 compiles my project in 444s from scratch on cold JVM, while LB 2.12.6 does it in 485s with -Ybackend-parallelism 8 enabled (on CI that difference is unnoticeable, but matters a bit when I compile locally).

Personally, I’ll stick to TL Scala for a while, perhaps until 2.13 comes out with induction heuristics polished by Miles Sabin. However, your use case will surely be different - if you needed TL Scala, you probably knew it by now.

kubuszok.com