c# - Operation is not valid due to the current state of the object (System.Text.Json)

Question

We've got an API, which simply posts incoming JSON documents to a message bus, having assigned a GUID to each. We're upgrading from .Net Core 2.2 to 3.1 and were aiming to replace NewtonSoft with the new System.Text.Json library.

We deserialise the incoming document, assign the GUID to one of the fields and then reserialise before sending to the message bus. Unfortunately, the reserialisation is failing with the exception Operation is not valid due to the current state of the object.

Here's a controller that shows the problem:-

using System;
using System.Net;
using Project.Models;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Text;
using System.Text.Json;

namespace Project.Controllers
{
    [Route("api/test")]
    public class TestController : Controller
    {
        private const string JSONAPIMIMETYPE = "application/vnd.api+json";

        public TestController()
        {
        }

        [HttpPost("{eventType}")]
        public async System.Threading.Tasks.Task<IActionResult> ProcessEventAsync([FromRoute] string eventType)
        {
            try
            {
                JsonApiMessage payload;

                using (StreamReader reader = new StreamReader(Request.Body, Encoding.UTF8)) {
                    string payloadString = await reader.ReadToEndAsync();

                    try {
                        payload = JsonSerializer.Deserialize<JsonApiMessage>(payloadString);
                    }
                    catch (Exception ex) {
                        return StatusCode((int)HttpStatusCode.BadRequest);
                    }
                }

                if ( ! Request.ContentType.Contains(JSONAPIMIMETYPE) )
                {
                    return StatusCode((int)HttpStatusCode.UnsupportedMediaType);
                }

                Guid messageID = Guid.NewGuid();
                payload.Data.Id = messageID.ToString();

                // we would send the message here but for this test, just reserialise it
                string reserialisedPayload = JsonSerializer.Serialize(payload);

                Request.HttpContext.Response.ContentType = JSONAPIMIMETYPE;
                return Accepted(payload);
            }
            catch (Exception ex) 
            {
                return StatusCode((int)HttpStatusCode.InternalServerError);
            }
        }
    }
}

The JsonApiMessage object is defined like this:-

using System.Text.Json;
using System.Text.Json.Serialization;

namespace Project.Models
{
    public class JsonApiMessage
    {
        [JsonPropertyName("data")]
        public JsonApiData Data { get; set; }

        [JsonPropertyName("included")]
        public JsonApiData[] Included { get; set; }
    }

    public class JsonApiData
    {
        [JsonPropertyName("type")]
        public string Type { get; set; }

        [JsonPropertyName("id")]
        public string Id { get; set; }

        [JsonPropertyName("attributes")]
        public JsonElement Attributes { get; set; }

        [JsonPropertyName("meta")]
        public JsonElement Meta { get; set; }

        [JsonPropertyName("relationships")]
        public JsonElement Relationships { get; set; }
    }
}

An example call looks like this:-

POST http://localhost:5000/api/test/event
Content-Type: application/vnd.api+json; charset=UTF-8

{
  "data": {
    "type": "test",
    "attributes": {
      "source": "postman",
      "instance": "jg",
      "level": "INFO",
      "message": "If this comes back with an ID, the API is probably working"
    }
  }
}

When I examine the contents of payload at a breakpoint in Visual Studio, it looks OK at the top level but the JsonElement bits look opaque, so I don't know if they've been parsed properly. Their structure can vary, so we only care that they are valid JSON. In the old NewtonSoft version, they were JObjects.

After the GUID has been added, it appears in the payload object when examined at a breakpoint but I'm suspicious that the problem is related to other elements in the object being read-only or something similar.

Answer 1

Your problem can be reproduced with the following more minimal example. Define the following model:

public class JsonApiMessage
{
    public JsonElement data { get; set; }
}

Then attempt to deserialize and re-serialize an empty JSON object like so:

var payload = JsonSerializer.Deserialize<JsonApiMessage>("{}");
var newJson = JsonSerializer.Serialize(payload, new JsonSerializerOptions { WriteIndented = true });

And you will get an exception (demo fiddle #1 here):

System.InvalidOperationException: Operation is not valid due to the current state of the object.
   at System.Text.Json.JsonElement.WriteTo(Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.JsonConverterJsonElement.Write(Utf8JsonWriter writer, JsonElement value, JsonSerializerOptions options)

The problem seems to be that JsonElement is a struct, and the default value for this struct can't be serialized. In fact, simply doing JsonSerializer.Serialize(new JsonElement()); throws the same exception (demo fiddle #2 here). (This contrasts with JObject which is a reference type whose default value is, of course, null.)

So, what are your options? You could make all your JsonElement properties be nullable, and set IgnoreNullValues = true while re-serializing:

public class JsonApiData
{
    [JsonPropertyName("type")]
    public string Type { get; set; }

    [JsonPropertyName("id")]
    public string Id { get; set; }

    [JsonPropertyName("attributes")]
    public JsonElement? Attributes { get; set; }

    [JsonPropertyName("meta")]
    public JsonElement? Meta { get; set; }

    [JsonPropertyName("relationships")]
    public JsonElement? Relationships { get; set; }
}

And then:

var reserialisedPayload  = JsonSerializer.Serialize(payload, new JsonSerializerOptions { IgnoreNullValues = true });

Demo fiddle #3 here.

Or, in .NET 5 or later, you could mark all of your JsonElement properties with [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]:

public class JsonApiData
{
    // Remainder unchanged

    [JsonPropertyName("attributes")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
    public JsonElement Attributes { get; set; }

    [JsonPropertyName("meta")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
    public JsonElement Meta { get; set; }

    [JsonPropertyName("relationships")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
    public JsonElement Relationships { get; set; }
}

Doing so will cause uninitialized elements to be skipped during serialization without needing to modify serialization options.

Demo fiddle #4 here.

Or, you could simplify your data model by binding all the JSON properties other than Id to a JsonExtensionData property like so:

public class JsonApiData
{
    [JsonPropertyName("id")]
    public string Id { get; set; }

    [JsonExtensionData]
    public Dictionary<string, JsonElement> ExtensionData { get; set; }
}

This approach avoids the need to manually set IgnoreNullValues when re-serializing, and thus ASP.NET Core will re-serialize the model correctly automatically.

Demo fiddle #5 here.