Scala major version incompatibilities

Scala major version incompatibilities broke some of the most basic expectations I had out of a language and ecosystem. Being from Java world, backward compatibility had become a fact of life not to be given a second thought.

If I have built a class using JDK 5, I do not have to worry about bytecode compatibility when running with or consuming it from a piece of code I am compiling with JDK 7.

Whereas in Scala, code compiled in Scala 2.8 cannot be consumed in Scala 2.9 or 2.10. It has to be compiled all over again.

This is a major hurdle for the scala platform in being enterprise ready.

Here is a quote from David Pollak, the creator of Lift.

As traits change, the classes that depend on those traits “break” unless they are recompiled.  This means that as Scala grows and evolves, all the libraries that sit on top of Scala must be recompiled against the latest version of Scala.  This has presented a significant challenge as Scala 2.8 has evolved.  Each of the layers necessary to compile libraries (e.g., Scala -> ScalaCheck -> ScalaTest -> Specs -> Lift) must be compiled against the same version of Scala.  Because of this issue, it’s been challenging to keep the growing number of Scala libraries up to date with Scala 2.8.

What did he do? he started a community initiative called “Fresh scala”. He talks about impact of version fragility problems and why he started the community here

Since then,  ”Scala guys at typesafe” took notice and have been investing in solving these problems. For example, they have introduced a “Cross compilation” feature within SBT builds. Well, It is a start!

Major version incompatibilities is a factor large enough to shape the Scala communities efforts and ecosystem as a whole.

Increased relevance of Functional programming

If we think about the computer as an FSM, Imperative program specifies commands to update the machine state. Each statement causes a change in the state. The state changes a million times per second.

Even if there is a meaning to a program beyond the hardware details, it is often obscured such that only a handful of experts can truly comprehend it.

As study of languages has progressed, different paradigms have emerged, which aim to allow the program to be structured in a more comprehensible manner; where the abstract notions are more easily representable and understandable.

Each paradigm has brought to the table a way of not just organizing code, but a way for developers to organize thoughts on how to tackle a specific problem; be it structural programming, object oriented programming, data flow programming or any other paradigm.

I found the following statement very cogent about functional programming.

"Functional programming aims to give each program a straightforward mathematical meaning. It simplifies our mental image of execution, for there are no state changes. Execution is the reduction of an expression to its value, replacing equals by equals. Most function definitions can be understood within elementary mathematics."

Functional programming paradigm and functional programming languages have been around for a long time. However, until recently their adoption has been limited, largely marginalized to academic and niche domains.

This has been due to the fact that functional programming languages have been traditionally slower then the other prevalent solutions.

However, this is changing. The change is largely dictated by changes in the direction hardware industry has gone.

Rather than focusing on creating increasingly powerful and fast processors with single core, the focus has shifted to multi-core solutions. With advances in networking and storage, the cost of having distributed systems has gone down.

If we talk about functional programming at a depth that just scratches the surface, then the following 2 properties are the most important

• Functional programming encourages writing functions that have no side effects.
• Immutability is a corner stone of functional programming paradigm. Use of immutable references, providing solid support for persistent datastructures.

Just looking at these two facts, one can immediately start working out and relating how the changes in the hardware industry are synergetic with functional programming.

A deeper analysis of how these trends have help emerge functional programming languages stronger is outside the purview of this post. We will get to that eventually.

Functional programming languages have been hovering on the horizon for a long time, may be their time to truly shine has finally come.