I have 50,000+ json objects buried in nested arrays. I want to pull them out so that they can be in single array. Also these nested arrays are quite random having no pattern. E.g
[ [ [ [ [ {"a":1 } ], [ {"b":2 } ] ], [[{"c":3 }]] ] ], [{"d":4 }] ]
has to be converted to
[{"a":1},{"b":2},{"c":3},{"d":4}]
using Dataweave 2.0
Used flatten but doesn't look like its the right function.
Here's one way to do it with a recursive function:
%dw 2.0
output application/json
fun flattenAllLevels(arr: Array) = do {
arr reduce ((item, acc = []) ->
item match {
case x is Array -> acc ++ flattenAllLevels(x)
else -> acc << item
}
)
}
---
flattenAllLevels(payload)
This is another version of flattening the nested Arrays. It is most likely to be less efficient then the already accepted solution because it needs more iteration. However, this uses Tail Recursion so you do not have to worry about Dataweave recursion limit and so it can even handle deeper nesting level.
If you need to handle heavily nested Arrays (let's say > 50) you can use this. Otherwise, Shoki's solution is more effective
%dw 2.0
import some from dw::core::Arrays
output application/json
@TailRec()
fun flattenAllLevels(arr: Array) = do {
var flattenedLevel1 = flatten(arr)
---
if(flattenedLevel1 some ($ is Array)) flattenAllLevels(flattenedLevel1)
else flattenedLevel1
}
// Helper function to create deeply nested array.
// for example, calling getDeepArray(5) will create [5,[4,[3,[2,[1,[0]]]]]]
fun getDeepArray(level) =
1 to level
reduce ((item, acc=[0]) -> ( [item] << acc ))
---
flattenAllLevels(getDeepArray(500))