Juanma (Habla Computing) is back with another talk on functional programming. This time, its all about applying it for real:
Functional programming is all about writing DSLs at different levels of abstraction. This talk aims at giving an example of functional programming in practice by explaining the design of a wholly abstract data transformation language prototype, i.e. a proof-of-concept DSL that allows you to implement data transformation programs that abstract away any specific feature from Spark, Flink, Scalding, or any other actual batch data processing framework. First, we will focus on how programs written in this DSL can be seamlessly compiled to Spark RDDs and the Scala Collection Library - two targets that can be regarded as lower-level abstraction DSLs aimed at production purposes and testing scenarios, respectively. Next, we will discuss the advantages and limitations of type classes versus algebraic data types when the time comes to implement our DSL. The talk will also illustrate how to exploit the type-level design patterns of the shapeless library in the ADT-based implementation. In sum, we aim at showing the great many advantages that purely functional programming can bring into the Big data computing landscape (e.g. unprecedented levels of reuse), beyond the conciseness and readability achieved through lambda expressions and higher-order functions; and, we also aim at giving hints on how to implement functional DSLs in an effective way.
Juan Manuel is CTO and co-founder of Habla Computing. He has been using Scala for the last six years in real-world applications for the banking sector, and has extensive experience in consultancy projects with scalaz, cats and other functional frameworks of the Scala ecosystem. He founded and manages the Madrid Scala Meetup group and is member of the organizing committee of lambda.world. Prior to joining Habla Computing, Juan Manuel was a University teacher for more than fifteen years in different computer science and software engineering degrees.