Schema Validation with Dynamic Systems

This service and its implementation would be useful in various cases where JSON objects must be validated against a predefined schema and filtered to comply.

Overall, this service's key usefulness lies in ensuring schema integrity, compliance, and resilience in applications that rely heavily on JSON-based data exchange. It aids in maintaining robust data-handling practices in APIs, microservices, and other systems processing dynamic data.

public interface ISchemaValidationService
{
    /// <summary>
    /// Validates the creation of an object based on the provided JSON representation and schema.
    /// The return object will be the fields of the dtoJson that exist in jsonSchema only.
    /// </summary>
    /// <param name="dtoJson">A JSON string representing the object to validate. May have more fields than expected.</param>
    /// <returns>
    /// A tuple containing:
    /// - A boolean indicating whether the object is valid.
    /// - A list of validation error messages, if any.
    /// - A processed object resulting from the validation.
    /// </returns>
    Task<(bool, List<string>, object)> Validate(string dtoJson);
}

public class SchemaValidationService : ISchemaValidationService
{
    public async Task<(bool, List<string>, object)> Validate(string dtoJson)
    {
        return await ValidateObject(dtoJson);
    }
    
    private async Task<(bool, List<string>, object)> ValidateObject(string dtoJson)
    {
        var validationErrors = new List<string>();

        try
        {
            // Parse JSON schema. The one to compare with.
            JSchemaGenerator generator = new();
            var schema = generator.Generate(typeof(SurveySchema));

            // Parse JSON data into a JObject
            var jsonObject = JObject.Parse(dtoJson);

            // Validate JSON structure to collect errors but treat the missing properties as warnings,
            // doing it for getting the errors only, we allow missing properties
            bool isValid = jsonObject.IsValid(schema, out IList<string> errors);
            
            if (errors.Any())
            {
                validationErrors.AddRange(errors);
            }

            // Filter the input JSON object to include only the matching properties from the schema
            var filteredObject = FilterObjectBySchema(jsonObject, schema);

            // Validation is successful if at least some matching properties exist
            bool hasMatchingProperties = filteredObject.HasValues;

            return (hasMatchingProperties, validationErrors, filteredObject);
        }
        catch (Exception ex)
        {
            // Catch any potential parsing/validation errors
            validationErrors.Add($"Exception occurred during validation: {ex.Message}");
            return (false, validationErrors, null)!;
        }
    }

    /// <summary>
    /// Filters the input JSON object to include only the properties defined in the provided JSON schema.
    /// </summary>
    /// <param name="jsonObject">The JSON object to be filtered.</param>
    /// <param name="schema">The schema defining the allowed properties.</param>
    /// <returns>
    /// A new JSON object containing only the properties that match the schema.
    /// </returns>
    private static JObject FilterObjectBySchema(JObject jsonObject, JSchema schema)
    {
        var filteredObject = new JObject();

        // Process each property in the schema
        foreach (var property in schema.Properties)
        {
            var propertyName = property.Key;

            // Check if the property exists in the input jsonObject
            if (jsonObject.TryGetValue(propertyName, out JToken? value))
            {
                // Add the property to the filtered object
                filteredObject[propertyName] = value;
            }
        }

        return filteredObject;
    }
}

Use it like:

[HttpPost("survey-v1")]
public async Task<IActionResult> Post( [FromBody] SurveyCreateDtoV1 dto )
{
    // dto to schema
    var dtoJson = JsonConvert.SerializeObject(dto);
    
    // validate
    var result = await _schemaValidationService.Validate(dtoJson);

    if (!result.Item1) return BadRequest(new { result.Item1, result.Item2 });
    
    // Convert filtered JsonObject (Item3) to a proper Json string or object
    // var filteredObject = (result.Item3 as JObject)?.ToObject<Dictionary<string, object>>();
    var filteredObject = JsonConvert.DeserializeObject<Dictionary<string, object>>(result.Item3.ToString()!);
    
    return Ok(new { result.Item1, result.Item2, filteredObject });

}

Test it with:

POST {{baseUrl}}/object/survey-v1 HTTP/1.1
Content-Type: application/json

{
  "number": "S12345",
  "description": "Sample survey description",
  "shortCode": "SC123"
}

In this case, I defined an object to extract the schema for convenience. Some implementations may just have a string representation of the schema stored in a database, settings or Library. Adjust it to your needs by creating a private method to extract the Schema, with database interaction e.t.c.

Schema based on the model:

public class SurveySchema
{
    [Key]
    public Guid Id { get; set; } = Guid.CreateVersion7(DateTime.UtcNow);
    public int Version { get; set; } = 1;
    [MaxLength(100)]
    public string? Number { get; set; }
    [MaxLength(250)]
    public string? Description { get; set; }
    [MaxLength(100)]
    public string? ShortCode { get; set; }
    [MaxLength(100)]
    public string? ProviderNumber { get; set; }
    public DateTimeOffset Created { get; set; } = DateTimeOffset.UtcNow;
    public DateTimeOffset Updated { get; set; } = DateTimeOffset.MinValue;
    public bool IsAvailable { get; set; } = true;
}

Payload to test:

public class SurveyCreateDtoV1
{
    [MaxLength(100)]
    public required string Number { get; set; }
    [MaxLength(250)]
    public required string Description { get; set; }
    [MaxLength(100)]
    public required string ShortCode { get; set; }
}

No files yet, migration hasn't completed yet!