Angara.Serialization


Building serializer for a user type

This tutorial describes how to create and use serializer for user defined type. We need to introduce some concepts first.

InfoSet

InfoSet is an intermediate representation of objects being serialized. InfoSet is a tree-based data structure that uses limited set of .NET types. Purpose of InfoSet is to decouple serialization format from object graph traversing. InfoSets are represented by instances of Angara.Serialization.InfoSet discriminated union. Here are some examples how to create InfoSets for primitive types

let i1 = InfoSet.Int 1
let i2 = InfoSet.String "Hello, World!"

InfoSets can store arrays of primitive types and sequences of InfoSets

let i3 = InfoSet.UInt64Array [| 0UL; 1UL; 9223372036854775807UL |]
let i4 = InfoSet.Seq [ i1; i2; i3 ]

InfoSets can store dictionaries of InfoSets

let i5 = InfoSet.Map(Map.empty<string, InfoSet>.Add("intValue", i1).Add("sequence", i4))

The same InfoSet can be constructed with less code using shortcut methods

let i5' = InfoSet.EmptyMap.AddInt("intValue", 1).AddSeq("sequence", [ i1; i2; i3 ]) 

Serialization is actually performed in two steps. On the first step object is converted to InfoSet using ArtefactSerializer.Serialize method as shown in the next code snippet

#r "Angara.Serialization.dll"
open Angara.Serialization
let a = [| for i in 0..100 -> i*i |] // An object to serialize
let infoSet = a |> ArtefactSerializer.Serialize CoreSerializerResolver.Instance

Then InfoSet is converted to serialized representation. Angara.Serialization.Json class provides support for JSON representation. Method Marshal converts InfoSet to JSON. Second parameter is related to blob support and can be set to None for now. Blobs will be discussed in a separate tutorial.

#r "Angara.Serialization.Json.dll"
let json = Json.Marshal(infoSet, None)

Deserialization is performed in reverse order. Serialized format is parsed into InfoSet by `Unmarshal' method, then InfoSet is converted to CLR object

let infoSet' = Json.Unmarshal(json, None)
let a' = infoSet |> ArtefactSerializer.Deserialize CoreSerializerResolver.Instance

FromObject and ToObject methods of Json class are convenient shortcuts that combine these two steps. Code snippet above can be rewrited in one line

let a'' = Json.ToObject<obj>(json, CoreSerializerResolver.Instance)

Type IDs

The serialization library uses textual identifier (TypeID) to differentiate types of objects in the serialized representation. Primary benefit of this approach is independence of serialized representation of .NET platform. Also TypeIDs can be chosen to be shorter than .NET full type names especially in case of generic types.

Resolvers

Serialization process needs to know TypeID and function that converts object to InfoSets for each type being serialized. Deserialization process need to know CLR type and function to convert InfoSets to object for each Type ID found in serialized representation. Two way mapping of CLR types and TypeIDs with converter functions is a reponsibility of resolvers - objects that implement Angara.Serialization.ISerializerResolver interface.

Serialization library came with built-in resolver for primitives, arrays, F# lists, tuples and optional types. It is available as singleton by Angara.Serialization.CoreSerializerResolver.Instance property.

Creating custom serializer

Let's build serializer for Vector2d type

type Vector2d = { x: int; y: int }

We need to define TypeID, serializer function that converts instances of Vector2d to InfoSet and deserializer function that converts InfoSets to Vector2d instances. This is done by creating serializer type - a type that implements Angara.Serialization.ISerializer<Vector2d> interface

type Vector2dSerializer () =
    interface ISerializer<Vector2d> with
        member x.TypeId = "V2D" // This is Type ID for Vector2d type
        member x.Serialize _ v = 
            InfoSet.EmptyMap.AddInt("x", v.x).AddInt("y", v.y)
        member x.Deserialize _ i = 
            let map = InfoSet.toMap i in { x = map.["x"].ToInt(); y = map.["y"].ToInt() } 

Design of Angara.Serialization library separates serialization responsibility from other responsibilities of an object. This allows us to make any object serializable without modifying its code or creating adapter types. However this means that we need to let serialization infrastructure know about our custom serializers.

To do this we can create SerializerLibrary - a resolver that is extensible with custom serializers. SerializerLibrary.CreateDefault method creates resolver that knows about all types supported by CoreInstanceResolver and allows to register serializers for user defined type.

let lib = SerializerLibrary.CreateDefault()
lib.Register(Vector2dSerializer())

Then we pass our custom library to serialization infrastructure when performing serialization or deserialization

let v = { x = -1; y = 10 }
let vjson = Json.FromObject(lib, v)
let v' = Json.ToObject<Vector2d>(vjson, lib)

Notice V2D TypeID in the generated JSON representation

"{
  ":V2D": {
    "x:int": -1,
    "y:int": 10
  }
}"
val i1 : obj

Full name: CustomSerializer.i1
val i2 : obj

Full name: CustomSerializer.i2
module String

from Microsoft.FSharp.Core
val i3 : obj

Full name: CustomSerializer.i3
val i4 : obj

Full name: CustomSerializer.i4
module Seq

from Microsoft.FSharp.Collections
val i5 : obj

Full name: CustomSerializer.i5
Multiple items
module Map

from Microsoft.FSharp.Collections

--------------------
type Map<'Key,'Value (requires comparison)> =
  interface IEnumerable
  interface IComparable
  interface IEnumerable<KeyValuePair<'Key,'Value>>
  interface ICollection<KeyValuePair<'Key,'Value>>
  interface IDictionary<'Key,'Value>
  new : elements:seq<'Key * 'Value> -> Map<'Key,'Value>
  member Add : key:'Key * value:'Value -> Map<'Key,'Value>
  member ContainsKey : key:'Key -> bool
  override Equals : obj -> bool
  member Remove : key:'Key -> Map<'Key,'Value>
  ...

Full name: Microsoft.FSharp.Collections.Map<_,_>

--------------------
new : elements:seq<'Key * 'Value> -> Map<'Key,'Value>
val empty<'Key,'T (requires comparison)> : Map<'Key,'T> (requires comparison)

Full name: Microsoft.FSharp.Collections.Map.empty
Multiple items
val string : value:'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

--------------------
type string = System.String

Full name: Microsoft.FSharp.Core.string
val i5' : obj

Full name: CustomSerializer.i5'
namespace Angara
namespace Angara.Serialization
val a : int []

Full name: CustomSerializer.a
val i : int
val infoSet : InfoSet

Full name: CustomSerializer.infoSet
module ArtefactSerializer

from Angara.Serialization
val Serialize : res:ISerializerResolver -> a:obj -> InfoSet

Full name: Angara.Serialization.ArtefactSerializer.Serialize
type CoreSerializerResolver =
  interface ISerializerResolver
  private new : unit -> CoreSerializerResolver
  static member Instance : CoreSerializerResolver

Full name: Angara.Serialization.CoreSerializerResolver
property CoreSerializerResolver.Instance: CoreSerializerResolver
val json : Newtonsoft.Json.Linq.JToken

Full name: CustomSerializer.json
type Json =
  private new : unit -> Json
  static member FromObject : resolver:ISerializerResolver * a:'a0 -> JToken
  static member FromObject : resolver:ISerializerResolver * a:'a0 * writer:IBlobWriter -> JToken
  static member Marshal : infoSet:InfoSet * writer:IBlobWriter option -> JToken
  static member ToObject : json:JToken * resolver:ISerializerResolver -> 't
  static member ToObject : json:JToken * resolver:ISerializerResolver * reader:IBlobReader -> 't
  static member Unmarshal : token:JToken * reader:IBlobReader option -> InfoSet

Full name: Angara.Serialization.Json
static member Json.Marshal : infoSet:InfoSet * writer:IBlobWriter option -> Newtonsoft.Json.Linq.JToken
union case Option.None: Option<'T>
val infoSet' : InfoSet

Full name: CustomSerializer.infoSet'
static member Json.Unmarshal : token:Newtonsoft.Json.Linq.JToken * reader:IBlobReader option -> InfoSet
val a' : obj

Full name: CustomSerializer.a'
val Deserialize : res:ISerializerResolver -> i:InfoSet -> obj

Full name: Angara.Serialization.ArtefactSerializer.Deserialize
val a'' : obj

Full name: CustomSerializer.a''
static member Json.ToObject : json:Newtonsoft.Json.Linq.JToken * resolver:ISerializerResolver -> 't
static member Json.ToObject : json:Newtonsoft.Json.Linq.JToken * resolver:ISerializerResolver * reader:IBlobReader -> 't
type obj = System.Object

Full name: Microsoft.FSharp.Core.obj
type Vector2d =
  {x: int;
   y: int;}

Full name: CustomSerializer.Vector2d
Vector2d.x: int
Multiple items
val int : value:'T -> int (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.int

--------------------
type int = int32

Full name: Microsoft.FSharp.Core.int

--------------------
type int<'Measure> = int

Full name: Microsoft.FSharp.Core.int<_>
Vector2d.y: int
Multiple items
type Vector2dSerializer =
  interface ISerializer<Vector2d>
  new : unit -> Vector2dSerializer

Full name: CustomSerializer.Vector2dSerializer

--------------------
new : unit -> Vector2dSerializer
Multiple items
type ISerializer =
  interface
    abstract member Deserialize : ISerializerResolver -> InfoSet -> obj
    abstract member Serialize : ISerializerResolver -> obj -> InfoSet
    abstract member Type : Type
    abstract member TypeId : string
  end

Full name: Angara.Serialization.ISerializer

--------------------
type ISerializer<'T> =
  interface
    abstract member Deserialize : ISerializerResolver -> InfoSet -> 'T
    abstract member Serialize : ISerializerResolver -> 'T -> InfoSet
    abstract member TypeId : string
  end

Full name: Angara.Serialization.ISerializer<_>
val x : Vector2dSerializer
Multiple items
override Vector2dSerializer.TypeId : string

Full name: CustomSerializer.Vector2dSerializer.TypeId

--------------------
type TypeId =
  | Simple of string
  | Generic of string * TypeId list
  override ToString : unit -> string
  static member MakeArray : itemTypeId:string -> string
  static member MakeList : itemTypeId:string -> string
  static member MakeOption : itemTypeId:string -> string
  static member ToString : typeId:string * argIds:seq<string> -> string
  static member TryParse : s:string -> TypeId option

Full name: Angara.Serialization.TypeId
override Vector2dSerializer.Serialize : ISerializerResolver -> v:Vector2d -> InfoSet

Full name: CustomSerializer.Vector2dSerializer.Serialize
val v : Vector2d
type InfoSet =
  | Artefact of string * InfoSet
  | Seq of seq<InfoSet>
  | Map of Map<string,InfoSet>
  | Null
  | String of string
  | Int of int
  | UInt of UInt32
  | Int64 of Int64
  | UInt64 of UInt64
  | Decimal of decimal
  ...
  member AddBlob : key:string * suffix:string * blob:IBlob -> InfoSet
  member AddBool : key:string * b:bool -> InfoSet
  member AddDateTime : key:string * d:DateTime -> InfoSet
  member AddDecimal : key:string * d:decimal -> InfoSet
  member AddDouble : key:string * d:double -> InfoSet
  member AddGuid : key:string * g:Guid -> InfoSet
  member AddInfoSet : key:string * v:InfoSet -> InfoSet
  member AddInt : key:string * i:int -> InfoSet
  member AddInt64 : key:string * i:Int64 -> InfoSet
  member AddNull : key:string -> InfoSet
  member AddSeq : key:string * s:seq<InfoSet> -> InfoSet
  member AddString : key:string * s:string -> InfoSet
  member AddUInt : key:string * i:UInt32 -> InfoSet
  member AddUInt64 : key:string * i:UInt64 -> InfoSet
  member ToBlob : unit -> string * IBlob
  member ToBool : unit -> bool
  member ToBoolArray : unit -> bool []
  member ToByteArray : unit -> byte []
  member ToDateTime : unit -> DateTime
  member ToDateTimeArray : unit -> DateTime []
  member ToDecimal : unit -> decimal
  member ToDecimalArray : unit -> Decimal []
  member ToDouble : unit -> float
  member ToDoubleArray : unit -> double []
  member ToGuid : unit -> Guid
  member ToInt : unit -> int
  member ToInt64 : unit -> Int64
  member ToInt64Array : unit -> Int64 []
  member ToIntArray : unit -> int []
  member ToMap : unit -> Map<string,InfoSet>
  member ToNamespace : unit -> string list * InfoSet
  member ToPair : unit -> string * InfoSet
  member ToSeq : unit -> seq<InfoSet>
  member ToStringArray : unit -> string []
  member ToStringValue : unit -> string
  member ToUInt : unit -> UInt32
  member ToUInt64 : unit -> UInt64
  member ToUInt64Array : unit -> UInt64 []
  member ToUIntArray : unit -> UInt32 []
  static member EmptyMap : InfoSet
  static member ofPairs : pairs:seq<string * InfoSet> -> InfoSet
  static member toMap : si:InfoSet -> Map<string,InfoSet>
  static member tryGetByteArray : key:string -> map:Map<string,InfoSet> -> byte [] option
  static member tryGetInt : key:string -> map:Map<string,InfoSet> -> int option
  static member tryGetIntArray : key:string -> map:Map<string,InfoSet> -> int [] option
  static member tryGetMap : key:string -> map:Map<string,InfoSet> -> Map<string,InfoSet> option
  static member tryGetSeq : key:string -> map:Map<string,InfoSet> -> seq<InfoSet> option
  static member tryGetString : key:string -> map:Map<string,InfoSet> -> string option
  static member tryGetStringArray : key:string -> map:Map<string,InfoSet> -> string [] option

Full name: Angara.Serialization.InfoSet
property InfoSet.EmptyMap: InfoSet
member InfoSet.AddInt : key:string * i:int -> InfoSet
override Vector2dSerializer.Deserialize : ISerializerResolver -> i:InfoSet -> Vector2d

Full name: CustomSerializer.Vector2dSerializer.Deserialize
val i : InfoSet
val map : Map<string,InfoSet>
static member InfoSet.toMap : si:InfoSet -> Map<string,InfoSet>
val lib : ISerializerLibrary

Full name: CustomSerializer.lib
Multiple items
type SerializerLibrary =
  interface ISerializerLibrary
  new : name:string -> SerializerLibrary
  static member CreateDefault : unit -> ISerializerLibrary
  static member CreateEmpty : unit -> ISerializerLibrary

Full name: Angara.Serialization.SerializerLibrary

--------------------
new : name:string -> SerializerLibrary
static member SerializerLibrary.CreateDefault : unit -> ISerializerLibrary
abstract member ISerializerLibrary.Register : ISerializer<'a> -> unit
val v : Vector2d

Full name: CustomSerializer.v
val vjson : Newtonsoft.Json.Linq.JToken

Full name: CustomSerializer.vjson
static member Json.FromObject : resolver:ISerializerResolver * a:'a0 -> Newtonsoft.Json.Linq.JToken
static member Json.FromObject : resolver:ISerializerResolver * a:'a0 * writer:IBlobWriter -> Newtonsoft.Json.Linq.JToken
val v' : Vector2d

Full name: CustomSerializer.v'
Fork me on GitHub