c# - Self referencing loop in Json.Net JsonSerializer from custom JsonConverter (Web API)

Question

The project is an Asp.Net Web API web service.

I have a type hierarchy that I need to be able to serialize to and from Json, so I have taken the code from this SO: How to implement custom JsonConverter in JSON.NET to deserialize a List of base class objects?, and applied the converter to my hierarchy's base class; something like this (there's pseudo-code here to hide irrelevancies):

[JsonConverter(typeof(TheConverter))]
public class BaseType
{
    // note the base of this type here is from the linked SO above
    private class TheConverter : JsonCreationConverter<BaseType>
    {
        protected override BaseType Create(Type objectType, JObject jObject)
        {
            Type actualType = GetTypeFromjObject(jObject); /*method elided*/
            return (BaseType)Activator.CreateInstance(actualType);
        }
    }
}

public class RootType
{
    public BaseType BaseTypeMember { get; set; }
}

public class DerivedType : BaseType
{

}

So if I deserialize a RootType instance whose BaseTypeMember was equal to an instance of DerivedType, then it will be deserialized back into an instance of that type.

For the record, these JSON objects contain a '$type' field which contains virtual type names (not full .Net type names) so I can simultaneously support types in the JSON whilst controlling exactly which types can be serialized and deserialized.

Now this works really well for deserializing values from the request; but I have an issue with serialization. If you look at the linked SO, and indeed the Json.Net discussion that is linked from the top answer, you'll see that the base code I'm using is entirely geared around deserialization; with examples of its use showing manual creation of the serializer. The JsonConverter implementation brought to the table by this JsonCreationConverter<T> simply throws a NotImplementedException.

Now, because of the way that Web API uses a single formatter for a request, I need to implement 'standard' serialization in the WriteObject method.

I must stress at this point that before embarking on this part of my project I had everything serializing properly without errors.

So I did this:

public override void WriteJson(JsonWriter writer, 
  object value, 
  JsonSerializer serializer)
{
    serializer.Serialize(writer, value);
}

But I get a JsonSerializationException: Self referencing loop detected with type 'DerivedType', when one of the objects is serialized. Again - if I remove the converter attribute (disabling my custom creation) then it works fine...

I have a feeling that this means that my serialization code is actually triggering the converter again on the same object, which in turn calls the serializer again - ad nauseam. Confirmed - see my answer

So what code should I be writing in WriteObject that'll do the same 'standard' serialization that works?

Answer 1

Well this was fun...

When I looked more closely at the stack trace for the exception, I noticed that the method JsonSerializerInternalWriter.SerializeConvertable was in there twice, indeed it was that method one off the top of the stack - invoking JsonSerializerInternalWriter.CheckForCircularReference - which in turn was throwing the exception. It was also, however, the source of the call to my own converter's Write method.

So it would seem that the serializer was doing:

• 1) If object has a converter
  • 1a) Throw if circular reference
  • 1b) Invoke converter's Write method
• 2) Else
  • 2a) Use internal serializers

So, in this case, the Json.Net is calling my converter which in turn is calling the Json.Net serializer which then blows up because it sees it's already serializing the object that was passed to it!

Opening ILSpy on the DLL (yes I know it's open source - but I want the 'callers' functionality!) and moving up the call stack from SerializeConvertable to JsonSerializerInternalWriter.SerializeValue, the code that detects whether a converter should be used can be found right near the start:

if (((jsonConverter = ((member != null) ? member.Converter : null)) != null 
   || (jsonConverter = ((containerProperty != null) ? containerProperty.ItemConverter 
                                                    : null)) != null 
   || (jsonConverter = ((containerContract != null) ? containerContract.ItemConverter 
                                                    : null)) != null 
   || (jsonConverter = valueContract.Converter) != null 
   || (jsonConverter = 
       this.Serializer.GetMatchingConverter(valueContract.UnderlyingType)) != null 
   || (jsonConverter = valueContract.InternalConverter) != null) 
   && jsonConverter.CanWrite)
{
    this.SerializeConvertable(writer, jsonConverter, value, valueContract, 
                              containerContract, containerProperty);
    return;
}

Thankfully that very last condition in the if statement provides the solution to my issue: all I had to do was to add the following to either the base converter copied from the code in the linked SO in the question, or in the derived one:

public override bool CanWrite
{
    get
    {
        return false;
    }
}

And now it all works fine.

The upshot of this, however, is that if you intend to have some custom JSON serialization on an object and you are injecting it with a converter and you intend to fallback to the standard serialization mechanism under some or all situations; then you can't because you will fool the framework into thinking you're trying to store a circular reference.

I did try manipulating the ReferenceLoopHandling member, but if I told it to Ignore them then nothing was serialized and if I told it to save them, unsurprisingly, I got a stack overflow.

It's possible that this is a bug in Json.Net - alright it's so much of an edge-case that it's in danger of falling off the edge of the universe - but if you do find yourself in this situation then you're kind of stuck!