1

In SQL Server, how to update/merge a json with another json, without explicitly using the keys / defining columns?

Some background: I store metadata as a json in a varchar(max) column. Each record can have different metadata keys in the same table. Like storing People and Products in the same table. Similar to EAV data model but instead of value table I use json column to store the metadata as key-value pairs. That is why I am looking for a generic solution.

i.e. one record can have metadata

{"last_name":"John","first_name":"Smith","age":28,"Address":"123 Steels st…"}

another record in the same table can have metadata

{"product_name":"Box","material":"plastic","Price":1.5,"Weight":20,"Height":15}

I am looking for an efficient/modern way to update/add multiple values in a json, from a json.

i.e. source

{
    "last_name": "John",
    "first_name": "Smith",
    "age": 28,
    "weight":79
    "address": "123 Steels st…"
}

what to update/add:

{   
    "address": "567 Yonge Ave…"
    "last_name": "Johnny"
    "age": 35
    "height":1.83
}

result- source updated to:

{
    "last_name":"Smith",
    "first_name": "Johnny",         - updated
    "age": 35,                      - updated
    "weight":79
    "address": "567 Yonge Ave…"     - updated
    "height":1.83                   - added
}

My solution:

declare @j_source varchar(200) = '{"first_name": "Smith", "last_name": "Smith","age": 28,"weight":79,"address": "123 Steels st…"}'
declare @j_update varchar(200) = '{"address": "567 Yonge Ave…","first_name": "Johnny","age": 35, "height":1.83}'

print @j_source
print @j_update

-- transform json to tables
select *
into #t_source
from openjson(@j_source)

select *
into #t_update
from openjson(@j_update)

-- combine the updated values with new values with non-updated values
select *
into #t_result
from
(
    -- get key values that are not being updated
    select ts.[key],ts.[value],ts.[type] 
    from #t_source as ts
    left join #t_update as tu
    on ts.[key] = tu.[key]
    where tu.[key] is null

    union -- get key values that are being updated. side note: the first and second select can be combined into one using isnull

    select ts.[key],tu.[value],ts.[type] -- take value from #t_update
    from #t_source as ts
    inner join #t_update as tu
    on ts.[key] = tu.[key]

    union -- add new key values that does not exists in the source

    select tu.[key],tu.[value],tu.[type] -- take value from #t_update
    from #t_source as ts
    right join #t_update as tu
    on ts.[key] = tu.[key]
    where ts.[key] is null
) as x
where [value] != '' -- remove key-value pair if the value is empty

/*
openjson type column data type
https://learn.microsoft.com/en-us/sql/t-sql/functions/openjson-transact-sql?view=sql-server-2017

type    data-type
0       null
1       string
2       int
3       true/false
4       array
5       object
*/


-- transform table back to json in a generic way
select @j_source = 
        '{' + 
        STUFF((
                select replace(',"x":','x', cast([key] as varchar(4000)) COLLATE SQL_Latin1_General_CP1_CI_AS) 
                    + case [type]
                        when 1 then replace('"z"','z',[value]) -- this is a string this is a text use double-quotes
                        when 2 then [value]  -- this is int, don't use double-quotes
                        else '' 
                     end
                from #t_result 
                for xml PATH('')
        ), 1, 1, '') 
        + '}'       

print 'after update'
print @j_source

drop table #t_source
drop table #t_update
drop table #t_result

My solution works, but:

  1. Likely will not work with arrays or nested json. Fine, doesn't bother me at this point.

  2. I wonder if there is a more proper/affective/elegant way to do the entire solution, perhaps using json_modify ?

  3. The order of the key-value pairs is not kept as the source, but I guess it's not a big deal.

  4. Any normal way to convert key-value table back to json without explicitly defining columns and without the "garbage" that "for json auto" gives?

Code:

SELECT [key], [value] 
FROM t_result 
FOR JSON path, WITHOUT_ARRAY_WRAPPER

Output:

{"key":"address","value":"567 Yonge Ave…"},
{"key":"age","value":35}, {"key":"first_name","value":"Johnny"},
{"key":"height","value":1.83},{"key":"last_name","value":"Smith"}

UPDATE:

Based on Roman Pekar elegant solution, I've added another case to that solution to exclude quotes when the value is [type] = 2(int). When there are millions of records as in my case, additional quotes impacts storage.

create function dbo.fn_json_merge
(
    @a nvarchar(max),
    @b nvarchar(max)
)
returns nvarchar(max)
as
begin
     if left(@a, 1) = '{' and left(@b, 1) = '{' 
     begin
            select
                @a = case 
                        when d.[type] in (1,3) then json_modify(@a, concat('$.',d.[key]), d.[value]) 
                        else @a 
                    end,
                @a = case 
                        when d.[type] in (2) and TRY_CAST(d.[value] AS int) is not null then json_modify(@a, concat('$.',d.[key]), cast(d.[value] as int)) 
                        when d.[type] in (2) and TRY_CAST(d.[value] AS int) is null then json_modify(@a, concat('$.',d.[key]), d.[value])
                        else @a 
                    end,
                @a = case 
                        when d.[type] in (4,5) then json_modify(@a, concat('$.',d.[key]), json_query(d.[value])) 
                        else @a 
                     end
            from openjson(@b) as d;
     end 
     else if left(@a, 1) = '[' and left(@b, 1) = '{' 
     begin
            select @a = json_modify(@a, 'append $', json_query(@b));
     end 
     else 
     begin
            select @a = concat('[', @a, ',', right(@b, len(@b) - 1));
     end;

    return @a;
end;

1 Answer 1

1

Take a look at this answer. If you work in Sql Server 2017 then you can create function to merge json:

create function dbo.fn_json_merge
(
    @a nvarchar(max),
    @b nvarchar(max)
)
returns nvarchar(max)
as
begin
    if left(@a, 1) = '{' and left(@b, 1) = '{' begin
        select
            @a = case when d.[type] in (4,5) then json_modify(@a, concat('$.',d.[key]), json_query(d.[value])) else @a end,
            @a = case when d.[type] not in (4,5) then json_modify(@a, concat('$.',d.[key]), d.[value]) else @a end
        from openjson(@b) as d;
    end else if left(@a, 1) = '[' and left(@b, 1) = '{' begin
        select @a = json_modify(@a, 'append $', json_query(@b));
    end else begin
        select @a = concat('[', @a, ',', right(@b, len(@b) - 1));
    end;

    return @a;
end;

sql fiddle demo

update updated based on comments - should work with different types of values better

create function dbo.fn_json_merge
(
    @a nvarchar(max),
    @b nvarchar(max)
)
returns nvarchar(max)
as
begin
    if left(@a, 1) = '{' and left(@b, 1) = '{' begin
        select @a =
            case
                when d.[type] in (4,5) then
                    json_modify(@a, concat('$.',d.[key]), json_query(d.[value]))
                when d.[type] in (3) then
                    json_modify(@a, concat('$.',d.[key]), cast(d.[value] as bit))
                when d.[type] in (2) and try_cast(d.[value] as int) = 1 then
                    json_modify(@a, concat('$.',d.[key]), cast(d.[value] as int))
                when d.[type] in (0) then
                    json_modify(json_modify(@a, concat('lax $.',d.[key]), 'null'), concat('strict $.',d.[key]), null)
                else
                    json_modify(@a, concat('$.',d.[key]), d.[value])
            end
        from openjson(@b) as d
    end else if left(@a, 1) = '[' and left(@b, 1) = '{' begin
        select @a = json_modify(@a, 'append $', json_query(@b))
    end else begin
        select @a = concat('[', @a, ',', right(@b, len(@b) - 1))
    end

    return @a
end

sql fiddle demo

Sign up to request clarification or add additional context in comments.

5 Comments

Really nice and elegant solution! I've made a small change to exclude quotes when the value is int.
Also, it doesn't handle correctly if @a is null or empty.
@ShawnElihis very well may be, I haven't extensively tested it
Float values are not handled correctly ( converted to string ) "testFloat":3.5 --> "testFloat":"3.5"
@ErvinS you can adjust it the same way it's done for integer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.