spray.json.FamilyFormats.scala Maven / Gradle / Ivy
The newest version!
// Copyright: 2010 - 2018 https://github.com/ensime/ensime-server/graphs
// License: http://www.gnu.org/licenses/lgpl-3.0.en.html
package spray.json
import scala.collection.immutable.ListMap
import shapeless._, labelled.{ field, FieldType }
/**
* Automatically create product/coproduct marshallers (i.e. families
* of sealed traits and case classes/objects) for spray-json.
*
* Shapeless allows us to view sealed traits as "co-products" (aka
* `Coproduct`s) and to view case classes / objects as "products" (aka
* `HList`s).
*
* Here we write marshallers for `HList`s and `Coproduct`s and a converter
* to/from the generic form.
*
* =Customisation=
*
* Users may provide an implicit `CoproductHint[T]` for their sealed
* traits, allowing the disambiguation scheme to be customised.
* Some variants are provided to cater for common needs.
*
* Users may also provide an implicit `ProductHint[T]` for their case
* classes, allowing wire format field names to be customised and to
* specialise the handling of `JsNull` entries. By default, `None`
* optional parameters are omitted and any formatter that outputs
* `JsNull` will be respected.
*
* =Performance=
*
* TL;DR these things are bloody fast, don't even think about runtime
* performance concerns unless you have hard proof that these
* serialisers are a bottleneck.
*
* However, **compilation times** may be negatively impacted. Many of
* the problems are fundamental performance issues in the Scala
* compiler. In particular, compile times appear to be quadratic with
* respect to the number of case classes/objects in a sealed family,
* with compile times becoming unbearable for 30+ implementations of a
* sealed trait. It may be wise to use encapsulation to reduce the
* number of implementations of a sealed trait if this becomes a
* problem. For more information see
* https://github.com/milessabin/shapeless/issues/381
*
* Benchmarking has shown that these formats are within 5% of the
* performance of hand-crafted spray-json family format serialisers.
* The extra overhead can be explained by the conversion to/from
* `LabelledGeneric`, and administration around custom type hinting.
* These are all just churn of small objects - not computational - and
* is nothing compared to the performance bottleneck introduced by
* using an `Either` (which uses exception throwing for control flow),
* e.g. https://github.com/spray/spray-json/issues/133 or how `Option`
* fields used to be handled
* https://github.com/spray/spray-json/pull/136
*
* The best performance is obtained by constructing an "explicit
* implicit" (see the tests for examples) for anything that you
* specifically wish to convert to / from (e.g. the top level family,
* if it is on a popular endpoint) because it will reuse the
* formatters at all stages in the hierarchy. However, the overhead
* for not defining the `implicit val` is less that you might expect
* (it only accounts for another 5%) whereas eagerly defining
* `implicit val`s for everything in the hierarchy can
* (counterintuitively) slow things down by 50%.
*
* Logging at the `trace` level is enabled to allow visibility of when
* formatters are being instantiated.
*
* =Caveats=
*
* If shapeless fails to derive a family format for you, it won't tell
* you what was missing in your tree. e.g. if you have a `UUID` in a
* deeply nested case class in a `MyFamily` sealed trait, it will
* simply say "can't find implicit for MyFamily" not "can't find
* implicit for UUID". When that happens, you just have to work out
* what is missing by trial and error. Sorry!
*
* Also, the Scala compiler has some funny [order dependency
* rules](https://issues.scala-lang.org/browse/SI-7755) which will
* sometimes make it look like you're missing an implicit for an
* element. The best way to avoid this is:
*
* 1. define the protocol/formatters in sibling, or otherwise
* independent, non-cyclic, packages. In particular, if you define
* your domain objects in `foo.domain` and your formats in
* `foo.formats`, note that you will not be able to access the
* formats from the `foo` parent package (this catches a lot of
* people out). Another approach is to use separate projects for
* the domain and formats, which avoids the problem entirely whilst
* allowing you to provide zero dependency packages of your domain
* objects to downstream consumers (I believe this to be good
* practice in a microservices world, as it effectively means
* exporting your schema).
*
* 2. define all your custom rules in an `object` that extends
* `FamilyFormats` so that the implicit resolution priority rules
* work in your favour (see tests for an example of this style).
* The derived `familyFormat` will win over implicit formats that
* have been inherited from a lower implicit scope, so you will
* often have to explicitly bring them back into the higher scope
* by listing each -- see FamilyFormats for an example using
* `SymbolFormat` and a user-defined format. i.e. provide an
* explicit `implicit val symbolFormat = SymbolJsonFormat`,
* similarly for `JsObjectFormat`.
*/
trait FamilyFormats extends MiddlePriorityFamilyFormats {
this: BasicFormats with StandardFormats =>
// scala compiler doesn't like spray-json's use of a type alias in the sig
override implicit def optionFormat[T: JsonFormat]: JsonFormat[Option[T]] =
new OptionFormat[T]
}
object FamilyFormats extends DefaultJsonProtocol with FamilyFormats
private[json] trait MiddlePriorityFamilyFormats
extends LowPriorityFamilyFormats {
this: StandardFormats with FamilyFormats =>
/**
* Format for `LabelledGenerics` that uses the `HList` marshaller below.
*
* `Blah.Aux[T, Repr]` is a trick to work around scala compiler
* constraints. We'd really like to have only one type parameter
* (`T`) implicit list `g: LabelledGeneric[T], f:
* Cached[Strict[JsonFormat[T.Repr]]]` but that's not possible.
*/
implicit def familyFormatWithDefault[T, Repr, DefaultRepr <: HList](
implicit
gen: LabelledGeneric.Aux[T, Repr],
default: Default.AsOptions.Aux[T, DefaultRepr],
sg: Cached[Strict[WrappedRootJsonFormatWithDefault[T, Repr, DefaultRepr]]]
): RootJsonFormat[T] = new RootJsonFormat[T] {
def read(j: JsValue): T = gen.from(sg.value.value.read(j, default()))
def write(t: T): JsObject = sg.value.value.write(gen.to(t))
}
}
/* low priority implicit scope so user-defined implicits take precedence */
private[json] trait LowPriorityFamilyFormats extends JsonFormatHints {
this: StandardFormats with MiddlePriorityFamilyFormats =>
/**
* a `JsonFormat[HList]` or `JsonFormat[Coproduct]` would not retain the
* type information for the full generic that it is serialising.
* This allows us to pass the wrapped type, achieving: 1) custom
* `CoproductHint`s on a per-trait level 2) configurable `null` behaviour
* on a per product level 3) clearer error messages.
*
* This is intentionally not part of the `JsonFormat` hierarchy to
* avoid ambiguous implicit errors.
*/
abstract class WrappedRootJsonFormat[Wrapped, SubRepr](
implicit
tpe: Typeable[Wrapped]
) {
final def read(j: JsValue): SubRepr = j match {
case jso: JsObject => readJsObject(jso)
case other => unexpectedJson[Wrapped](other)
}
def readJsObject(j: JsObject): SubRepr
def write(v: SubRepr): JsObject
}
/**
* Subclass of the the `WrappedRootJsonFormat` that provide a way to
* deserialize product with a default value.
*/
abstract class WrappedRootJsonFormatWithDefault[Wrapped, SubRepr, DefaultRepr](
implicit
tpe: Typeable[Wrapped]
) extends WrappedRootJsonFormat[Wrapped, SubRepr] {
final def read(j: JsValue, default: DefaultRepr): SubRepr = j match {
case jso: JsObject => readJsObjectWithDefault(jso, default)
case other => unexpectedJson[Wrapped](other)
}
def readJsObjectWithDefault(j: JsObject, default: DefaultRepr): SubRepr
def readJsObject(j: JsObject): SubRepr =
deserError(
s"read should never be from WrappedRootJsonFormatWithDefault, $j"
)
}
// save an object alloc every time and gives ordering guarantees
private[this] val emptyJsObject = new JsObject(ListMap())
// HNil is the empty HList
implicit def hNilFormat[Wrapped](
implicit
t: Typeable[Wrapped]
): WrappedRootJsonFormatWithDefault[Wrapped, HNil, HNil] =
new WrappedRootJsonFormatWithDefault[Wrapped, HNil, HNil] {
def readJsObjectWithDefault(j: JsObject, default: HNil) =
HNil // usually a populated JsObject, contents irrelevant
def write(n: HNil) = emptyJsObject
}
// HList with a FieldType at the head
implicit def hListFormat[Wrapped,
Key <: Symbol,
Value,
Remaining <: HList,
D <: HList](
implicit
t: Typeable[Wrapped],
ph: ProductHint[Wrapped],
key: Witness.Aux[Key],
jfh: Lazy[JsonFormat[Value]], // svc doesn't need to be a RootJsonFormat
jft: WrappedRootJsonFormatWithDefault[Wrapped, Remaining, D]
): WrappedRootJsonFormatWithDefault[Wrapped,
FieldType[Key, Value] :: Remaining,
Option[Value] :: D] =
new WrappedRootJsonFormatWithDefault[Wrapped,
FieldType[Key, Value] :: Remaining,
Option[Value] :: D] {
private[this] val fieldName = ph.fieldName(key.value)
private[this] def missingFieldError(j: JsObject): Nothing =
deserError(s"missing $fieldName, found ${j.fields.keys.mkString(",")}")
def readJsObjectWithDefault(j: JsObject, default: Option[Value] :: D) = {
val resolved: Value = (j.fields.get(fieldName), jfh.value) match {
// (None, _) means the value is missing in the wire format
case (None, f) if ph.nulls == NeverJsNull =>
f.read(JsNull)
case (None, f) if ph.nulls == UseDefaultJsNull =>
default.head.getOrElse(f.read(JsNull))
case (None, f) if ph.nulls == AlwaysJsNull =>
missingFieldError(j)
case (None, f: OptionFormat[_])
if ph.nulls == JsNullNotNone || ph.nulls == AlwaysJsNullTolerateAbsent =>
None.asInstanceOf[Value]
case (Some(JsNull), f: OptionFormat[_])
if ph.nulls == JsNullNotNone =>
f.readSome(JsNull)
case (Some(value), f) =>
f.read(value)
case _ =>
missingFieldError(j)
}
val remaining = jft.read(j, default.tail)
field[Key](resolved) :: remaining
}
def write(ft: FieldType[Key, Value] :: Remaining) =
(jfh.value.write(ft.head), jfh.value) match {
// (JsNull, _) means the underlying formatter serialises to JsNull
case (JsNull, _) if ph.nulls == NeverJsNull =>
jft.write(ft.tail)
case (JsNull, _) if ph.nulls == JsNullNotNone & ft.head == None =>
jft.write(ft.tail)
case (value, _) =>
jft.write(ft.tail) match {
case JsObject(others) =>
// when gathering results, we must remember that 'other'
// is to the right of us and this seems to be the
// easiest way to prepend to a ListMap
JsObject(ListMap(fieldName -> value) ++: others)
case other =>
serError(s"expected JsObject, seen $other")
}
}
}
// CNil is the empty co-product. It's never called because it would
// mean a non-existant sealed trait in our interpretation.
implicit def cNilFormat[Wrapped](
implicit
t: Typeable[Wrapped]
): WrappedRootJsonFormat[Wrapped, CNil] =
new WrappedRootJsonFormat[Wrapped, CNil] {
def readJsObject(j: JsObject) =
deserError(s"read should never be called for CNil, $j")
def write(c: CNil) = serError("write should never be called for CNil")
}
// Coproduct with a FieldType at the head
implicit def coproductFormat[Wrapped,
Name <: Symbol,
Instance,
Remaining <: Coproduct](
implicit
tpe: Typeable[Wrapped],
th: CoproductHint[Wrapped],
key: Witness.Aux[Name],
jfh: Lazy[RootJsonFormat[Instance]],
jft: WrappedRootJsonFormat[Wrapped, Remaining]
): WrappedRootJsonFormat[Wrapped, FieldType[Name, Instance] :+: Remaining] =
new WrappedRootJsonFormat[Wrapped, FieldType[Name, Instance] :+: Remaining] {
def readJsObject(j: JsObject) = th.read(j, key.value) match {
case Some(product) =>
val recovered = jfh.value.read(product)
Inl(field[Name](recovered))
case None =>
Inr(jft.read(j))
}
def write(lr: FieldType[Name, Instance] :+: Remaining) = lr match {
case Inl(l) =>
jfh.value.write(l) match {
case j: JsObject => th.write(j, key.value)
case other => serError(s"expected JsObject, got $other")
}
case Inr(r) =>
jft.write(r)
}
}
/**
* Format for `LabelledGenerics` that uses the `Coproduct`
* marshaller above.
*
* `Blah.Aux[T, Repr]` is a trick to work around scala compiler
* constraints. We'd really like to have only one type parameter
* (`T`) implicit list `g: LabelledGeneric[T], f:
* Cached[Strict[JsonFormat[T.Repr]]]` but that's not possible.
*/
implicit def familyFormat[T, Repr](
implicit
gen: LabelledGeneric.Aux[T, Repr],
sg: Cached[Strict[WrappedRootJsonFormat[T, Repr]]]
): RootJsonFormat[T] = new RootJsonFormat[T] {
def read(j: JsValue): T = gen.from(sg.value.value.read(j))
def write(t: T): JsObject = sg.value.value.write(gen.to(t))
}
}
trait JsonFormatHints {
trait CoproductHint[T] {
/**
* Given the `JsObject` for the sealed family, disambiguate and
* extract the `JsObject` associated to the `Name` implementation
* (if available) or otherwise return `None`.
*/
def read[Name <: Symbol](j: JsObject, n: Name): Option[JsObject]
/**
* Given the `JsObject` for the contained product type of `Name`,
* encode disambiguation information for later retrieval.
*/
def write[Name <: Symbol](j: JsObject, n: Name): JsObject
/**
* Override to provide custom field naming.
* Caching is recommended for performance.
*/
protected def fieldName(orig: String): String = orig
}
/**
* Product types are disambiguated by a `{"key":"value",...}`. Of
* course, this will fail if the product type has a field with the
* same name as the key. The default key is the word "type" which
* is a keyword in Scala so unlikely to collide with too many case
* classes.
*
* This variant is most common in JSON serialisation schemes and
* well supported by other frameworks.
*/
class FlatCoproductHint[T: Typeable](key: String) extends CoproductHint[T] {
def read[Name <: Symbol](j: JsObject, n: Name): Option[JsObject] =
j.fields.get(key) match {
case Some(JsString(hint)) if hint == fieldName(n.name) => Some(j)
case Some(JsString(hint)) => None
case _ =>
deserError(s"missing $key, found ${j.fields.keys.mkString(",")}")
}
// puts the typehint at the head of the field list
def write[Name <: Symbol](j: JsObject, n: Name): JsObject = {
// runtime error, would be nice if we could check this at compile time
if (j.fields.contains(key))
serError(
s"typehint '$key' collides with existing field ${j.fields(key)}"
)
JsObject(ListMap(key -> JsString(fieldName(n.name))) ++: j.fields)
}
}
/**
* Product types are disambiguated by an extra JSON map layer
* containing a single key which is the name of the type of product
* contained in the value. e.g. `{"MyType":{...}}`
*
* This variant may be more appropriate for non-polymorphic schemas
* such as MongoDB and Mongoose (consider using the above format on
* your endpoints, and this format when persisting).
*/
class NestedCoproductHint[T: Typeable] extends CoproductHint[T] {
def read[Name <: Symbol](j: JsObject, n: Name): Option[JsObject] =
j.fields.get(fieldName(n.name)).map {
case jso: JsObject => jso
case other => unexpectedJson(other)
}
def write[Name <: Symbol](j: JsObject, n: Name): JsObject =
JsObject(fieldName(n.name) -> j)
}
implicit def coproductHint[T: Typeable]: CoproductHint[T] =
new FlatCoproductHint[T]("type")
/**
* Sometimes the wire format needs to match an existing format and
* `JsNull` behaviour needs to be customised. This allows null
* behaviour to be defined at the product level. Field level control
* is only possible with a user-defined `RootJsonFormat`.
*/
sealed trait JsNullBehaviour
/** All values serialising to `JsNull` will be included in the wire format. Ambiguous. */
case object AlwaysJsNull extends JsNullBehaviour
/** Option values of `None` are omitted, but `Some` values of `JsNull` are retained. Default. */
case object JsNullNotNone extends JsNullBehaviour
/** No values serialising to `JsNull` will be included in the wire format. Ambiguous. */
case object NeverJsNull extends JsNullBehaviour
/** Same as AlwaysJsNull when serialising, with missing values treated as optional upon deserialisation. Ambiguous. */
case object AlwaysJsNullTolerateAbsent extends JsNullBehaviour
/** Use the case class default value provided for the field when available. Ambiguous. */
case object UseDefaultJsNull extends JsNullBehaviour
trait ProductHint[T] {
def nulls: JsNullBehaviour = JsNullNotNone
def fieldName[Key <: Symbol](key: Key): String = key.name
}
implicit def productHint[T: Typeable] = new ProductHint[T] {}
}
© 2015 - 2025 Weber Informatics LLC | Privacy Policy