A review of London Scala Exchange 2018

Firstly, I would like to thank Signify Technology for sponsoring me to attend Scala Exchange; It is the premier Scala conference in the UK and always attracts great talent.

Looking at the programme now it is quite amazing how the community and ecosystem has evolved since I attended the conference for the first time back in 2013. There has been an array of advancements in FP for Scala, and it was nice to see Scala being used in a wide variety of use cases touching on A.I (presented by Chris Birchall), streaming infrastructures (demonstrated by engineers at Deliveroo + Disney) and machine learning recommendation engines (presented by the data engineering team at Elsevier).

Please click to read on: Full review of Scala Exchange 2018 by Rama Nallamilli

Scalaz Monad Transformers

Whilst gaining a deeper understanding of functional programming concepts and patterns I have found myself delving more deeply into the world of scalaz. Today the agenda is monad transformers, after some initial reading I very quickly started to see patterns in our codebase which could immediately benefit from their application.

What is a monad transformer? My definition (as a Software Engineer) is… a monad transformer is a typeclass that abstracts a monad which wraps another monad. This may not sound like something that happens a lot but it’s surprisingly common, take Future for example, all these cases below are monads nested within a monad.

Future[Option[T]]
Future[List[T]]
Future[scalaz.\/[A, B]]

Lets take the following scenario, you have two Lists of integers, and you want to add every element in List A to every element in List B and get a final List C. However both lists are wrapped in a Future. In pure Scala this may look something like this:

@ val x = Future.successful(List(1, 2, 3))
x: Future[List[Int]] = scala.concurrent.impl.Promise$KeptPromise@5da72d46
@ val y = Future.successful(List(4, 5, 6))
y: Future[List[Int]] = scala.concurrent.impl.Promise$KeptPromise@2896f3c5
@
@ x.flatMap { listA
y.map { listB =>
listA.flatMap { i =>
listB.map { j =>
i + j
}
}
}
}
res17: Future[List[Int]] = Success(List(5, 6, 7, 6, 7, 8, 7, 8, 9))

Or alternatively using the syntactic sugar of a for comprehension:

@ for {
listA <- x
listB <- y
} yield {
for {
i <- listA
j <- listB
} yield i + j
}
res18: Future[List[Int]] = Success(List(5, 6, 7, 6, 7, 8, 7, 8, 9))

Notice that there is no way of accessing the underlying data without having to map over the Future followed by the List, which leads to the nested code you see above, this situation is where monad transformers can help us.

In our example our top level type was a Future[List[Int]], when choosing which monad transformer to use, you always choose the inner most type, in this case List[Int] is our inner most type so we will use the ListT monad transformer. The ListT apply function is as follows:

def apply[A](a: M[List[A]]) = new ListT[M, A](a)

Therefore we can use this to convert our Future[List[Int]] to the ListT monad type which in this case will be a ListT[Future, Int]. We can now write our addition in terms of the new monad type which has abstracted the mapping of the Future:

@ for {
i <- ListT(x)
j <- ListT(y)
} yield i + j
res20: ListT[Future, Int] = ListT(Success(List(5, 6, 7, 6, 7, 8, 7, 8, 9)))

Notice we returned ListT[Future, Int], as with any Functor, calling map will always return the same monad type wrapping the transformed value. This allows you to chain/compose operations in terms of your monad transformers until you are ready to unwrap back to your original type, which can be done using the run method:

@ res20.run
res21: Future[List[Int]] = Success(List(5, 6, 7, 6, 7, 8, 7, 8, 9))

In summary monad transformers give you a powerful abstraction to work on the underlying data of a monadic type when it itself is wrapped in a monad. It reduces code complexity and enhances readability by abstracting the wiring of drilling down into the nested datatypes. ScalaZ provides implementations of monad transformers for many types including EitherT, ListT, OptionT and ReaderT to name a few.

Moving from Java to Scala

Functional programming was nowhere to be seen during my time at University, nor in my professional career, that is up until the point Scala came out of its shell. Object oriented programming had ruled and the only time I had heard much about functional languages were stories from more experienced engineers, most of which usually started with “back in the day”. In my mind that translated to an era of floppy disk drives and ASM programming, I took no notice.

haskell

I remember sitting in a SkillsMatter training course with a Typesafe representative, we had just finished a chapter on the fundamental basics of Scala, I was completely new to the notion of a map function, what had happened to the safe haven of imperative code?

In the beginning

Coming from a team of solid Java developers I was initially quite apprehensive, early proof of concept work had been frustrating and getting setup was cumbersome. IntelliJ was tripping over its own heels, traversing Scala documentation that spanned multiple versions of libraries was a battle, compile times seemed sluggish and SBT was another build/dependency management tool to add to the ever-growing list. All this on top of pressures to deliver project features.

Diving into a full new technology stack and language can be rather daunting, especially when there is no prior expertise in your team. Although most good developers are able to pickup Scala through self learning, having some formal training or even better, some short-term experienced Scala contractors (with the primary purpose to knowledge share) fast tracked the team to getting their skills off the ground.

Rather than move to a full Scala stack in one swoop we opted to take a more methodical approach. First we replaced our Java based integration tests with Scala ones, this allowed the developers to learn the fundamentals of Scala without compromising any production projects or code. Once the team was comfortable with the fundamentals of the language we decided to build our next production project in Scala.

Reaping the rewards

Although the surrounding landscape had initially left much to desire, the advantages of the language itself were soon apparent, to name but a few:

scala-code

more concise code – the syntax and features of Scala strip away a lot of the unnecessary boiler plate that Java enforces while native language features provide solutions to common engineering patterns. Case classes, traits, type inference, optional braces, partial functions, pattern matching, implicit conversions, lazy val and more, it all plays a part.

let’s get functional – Scala treats functions as first class citizens, the ability to compose functions, create anonymous functions and pass them around really makes the language much more powerful when designing your code architecture.

less time coding – Once the team became proficient with Scala, the productivity of our team improved, this is something I observed as a general improvement in my own velocity and my colleagues.

the JVM ecosystem – Scala is a JVM language which meant we still had all of the Java libraries at our disposable, this was particularly important for us as it reduced some of the pressure during the transition period. Familiar technologies such as Apache HTTP Commons, Camel, Spring, Apache CXF and many more could still be used which reduced risk and allowed a more methodical transition to a full Scala stack.

collections – Scala collections are immutable by default and it’s not until we really started to use Scala that a majority of problems can be solved without the use of mutable ones. Immutable collections are much easier to reason about, there is less variables and branching statements to consider. On top of this all collections support functional operators such as map, flatMap, collect, fold (and more) allowing you to perform powerful transformations with ease.

a reduction in bugs…perhaps – this is quite a subjective statement and comes from my personal experience working on multiple commercial Java projects. Logic bugs that we have seen being raised seemed reduced compared to similar past projects. By logic bugs I mean things like null pointer exceptions, undesired results due to errors in branching logic and just plain business logic mistakes.

I suspect this comes from the ability to write more concise and expressive code with focus on the “happy path” using the functional paradigm. On top of this Scala collections are immutable by default, immutably reduces complexity and makes testing easier, along with the ability to compose behaviour using higher order functions and traits. All these factors seem to contribute to the quality of our systems which I believe had a direct impact on the types and quantity of bugs. Of course some of this reduction may be attributed to more focus on automated testing, better testing frameworks and engineering practices……or maybe it’s all just placebo, I’ll let you decide.

The landscape now

The landscape for Scala development has improved greatly over the past few years, Scala 2.11 seems to have stabilised, SBT continuous compile mode offsets the longer compilation time allowing you to get feedback fast, there have been major improvements in IntelliJ support, and we also have the Typesafe Activator, which is arguably the easiest way to get started with a Scala application. On top of this we still have the full JVM eco-system at our disposable.

I would like to stress that for the same reasons Scala allows you to write succulent and elegant code, it’s versatility can be a double-edged sword.

with great power comes great responsability

There are a lot of programmers out there who are incredibly smart, sometimes too smart. In the few short years I have been working with Scala, I have seen several examples where Engineers have been lulled into writing unnecessarily complex solutions at the expense of readability and maintainability. I think Scala is a language which can be more susceptible to abuse in this area but as with any language, code quality reviews and established coding standards can mitigate these issues. My advice to those adopting Scala is to carefully choose the patterns and features you adopt, make sure your engineers are comfortable and just take it slow.

A great starting point on providing style and implementation advice is the Twitter Effective Scala guide by Marius Eriksen, I couldn’t recommend it enough. In my opinion, the best Scala learning resource on the web is The Neophyte’s Guide to Scala by Daniel Westheide, this blog series is excellent and I would highly recommend it to anybody currently learning Scala.

As a final note, I would like to leave you with a point I feel is rarely mentioned. Programming in Scala is FUN. The language is powerful, flexible and versatile giving you the opportunity to solve problems in the most elegant way, this really does make coding fun again. I’m sure you will all agree, happier engineers are always a good thing in any tech organisation, who can argue with that?!