ASP.Net JSON configuration file transforms with arrays

To understand the cause of such 'strange' behavior for overridden array settings you need to understand how those settings are stored inside configuration providers.

The reality is that all loaded settings are stored in dictionaries, own for each configuration provider. Keys are built from setting paths where nested sections are delimited with a colon. Array settings are stored in the same dictionary with an index in setting path (:0, :1, ...).

For configuration you described you will have 2 configuration providers with following sets of settings:

provider1[Values:Test:0] = "one"
provider1[Values:Test:1] = "two"

and

provider2[Values:Test:0] = "three"

Now it's clear why the final value of array setting is ["three", "two"]. Values:Test:0 from the second provider overrides the same setting from the first provider, and Values:Test:1 is left untouched.

Unfortunately, there is now a built-in possibility to overcome this problem. Fortunately, .net core configuration model is flexible enough for adjusting this behavior for your needs.

Idea is the following:

1. Enumerate configuration providers in reverse order.
2. For each provider get all its setting keys. You could call IConfigurationProvider.GetChildKeys() method recursively for this purpose. See GetProviderKeys() in below snippet.
3. With a regular expression check whether current key is an array entry.
4. If it is and some of previous providers overrides this array, then just suppress current array entry by setting it to null value.
5. If it's unseen array then current provider is marked as the only provider of values for this array. Arrays from all other providers will be suppressed (step #4).

For convenience you could wrap all this logic into extension method on IConfigurationRoot.

Here is a working sample:

public static class ConfigurationRootExtensions
{
    private static readonly Regex ArrayKeyRegex = new Regex("^(.+):\\d+$", RegexOptions.Compiled);

    public static IConfigurationRoot FixOverridenArrays(this IConfigurationRoot configurationRoot)
    {
        HashSet<string> knownArrayKeys = new HashSet<string>();

        foreach (IConfigurationProvider provider in configurationRoot.Providers.Reverse())
        {
            HashSet<string> currProviderArrayKeys = new HashSet<string>();

            foreach (var key in GetProviderKeys(provider, null).Reverse())
            {
                //  Is this an array value?
                var match = ArrayKeyRegex.Match(key);
                if (match.Success)
                {
                    var arrayKey = match.Groups[1].Value;
                    //  Some provider overrides this array.
                    //  Suppressing the value.
                    if (knownArrayKeys.Contains(arrayKey))
                    {
                        provider.Set(key, null);
                    }
                    else
                    {
                        currProviderArrayKeys.Add(arrayKey);
                    }
                }
            }

            foreach (var key in currProviderArrayKeys)
            {
                knownArrayKeys.Add(key);
            }
        }

        return configurationRoot;
    }

    private static IEnumerable<string> GetProviderKeys(IConfigurationProvider provider,
        string parentPath)
    {
        var prefix = parentPath == null
                ? string.Empty
                : parentPath + ConfigurationPath.KeyDelimiter;

        List<string> keys = new List<string>();
        var childKeys = provider.GetChildKeys(Enumerable.Empty<string>(), parentPath)
            .Distinct()
            .Select(k => prefix + k).ToList();
        keys.AddRange(childKeys);
        foreach (var key in childKeys)
        {
            keys.AddRange(GetProviderKeys(provider, key));
        }

        return keys;
    }
}

The last thing is to call it when building the configuration:

IConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
configurationBuilder.AddJsonFile("AppSettings.json")
    .AddJsonFile("appsettings.dev.json");
var configuration = configurationBuilder.Build();
configuration.FixOverridenArrays();