Getting started with desert

Desert is a binary serialization library for Scala, focusing on creating small binaries while still enabling binary compatible evolution of the data model.

It is suitable for use cases such as Akka remoting and persistence, and for any kind of short or long term storage.

First add desert as a dependency:

libraryDependencies += "io.github.vigoo" %% "desert-core" % "0.3.0"

choose either the Shapeless based or the ZIO Schema based derivation implementation:

libraryDependencies += "io.github.vigoo" %% "desert-shapeless" % "0.3.0"
// or
libraryDependencies += "io.github.vigoo" %% "desert-zio-schema" % "0.3.0"

and optionally some extra bindings:

libraryDependencies += "io.github.vigoo" %% "desert-akka" % "0.3.0"
libraryDependencies += "io.github.vigoo" %% "desert-cats" % "0.3.0"
libraryDependencies += "io.github.vigoo" %% "desert-cats-effect" % "0.3.0"
libraryDependencies += "io.github.vigoo" %% "desert-zio" % "0.3.0"
libraryDependencies += "io.github.vigoo" %% "desert-shardcake" % "0.3.0"

The most simple use case is to serialize a known type to an array of bytes and read it back:

import io.github.vigoo.desert._

val x = "Hello world"

val dataOrFailure = serializeToArray(x)
227210110810811132119111114108100
val y = dataOrFailure.flatMap(data => deserializeFromArray[String](data))
// y: Either[DesertFailure, String] = Right(value = "Hello world")

Codecs

This works because there is an implicit BinaryCodec for String in scope, imported from the desert package. Read the codecs page to learn about the available codecs and how to define custom ones.

Shapeless or ZIO Schema?

The library has two derivation implementations. The Shapeless based is the original one and is more mature, but only works with Scala 2. The ZIO Schema based derivation works both with Scala 2 and Scala 3 and it is planned to be the only available implementation in future versions.

There are only small differences in using the two implementations:

  • For Shapeless based derivation you have to define an implicit codec for each constructor of a sealed trait
  • For ZIO Schema based derivation you have to derive a Schema for each type as well

Low level input/output

The above example shows the convenient functions to work on arrays directly, but they have a more generic version working on the low level BinaryInput and BinaryOutput interfaces. These are described on the input/output page.

Evolution

One of the primary features of the library is the support for evolving the data model. The possibilities are described on a separate page.

Type registry

For cases when the exact type to be deserialized is not known at compile type, the possibilities can be registered to a type registry.

Integrations

There are three additional modules providing integrations to different environments:

  • Akka codecs and Akka Serializer implementation
  • Cats adding some Cats specific codecs
  • Cats Effect defining the serialization/deserialization as an effect
  • Shardcake defines a Shardcake serializer
  • ZIO defining the serialization/deserialization as an effect and some codecs