I have several xml files. They all have the same structure, but were splitted due to file size. So, let's say I have A.xml, B.xml, C.xml and D.xml and want to combine/merge them to combined.xml, using a command line tool.
A.xml
<products>
<product id="1234"></product>
...
</products>
B.xml
<products>
<product id="5678"></product>
...
</products>
etc.
High-tech answer:
Save this Python script as xmlcombine.py:
#!/usr/bin/env python
import sys
from xml.etree import ElementTree
def run(files):
first = None
for filename in files:
data = ElementTree.parse(filename).getroot()
if first is None:
first = data
else:
first.extend(data)
if first is not None:
print(ElementTree.tostring(first))
if __name__ == "__main__":
run(sys.argv[1:])
To combine files, run:
python xmlcombine.py ?.xml > combined.xml
For further enhancement, consider using:
chmod +x xmlcombine.py:
Allows you to omit python in the command line
xmlcombine.py !(combined).xml > combined.xml:
Collects all XML files except the output, but requires bash's extglob option
xmlcombine.py *.xml | sponge combined.xml:
Collects everything in combined.xml as well, but requires the sponge program
import lxml.etree as ElementTree:
Uses a potentially faster XML parser
_ElementInterface instance has no attribute 'extend'. Which version of python are you using?
<foo><bars><bar1/><bar2/></bars><bars><bar3/><bar4/></bars></foo>. It does not merge the lower levels as in <foo><bars><bar1/><bar2/><bar3/><bar4/></bars></foo>http://search.cpan.org/dist/XML-Twig/tools/xml_grep/xml_grep
xml_grep --pretty_print indented --wrap products --descr '' --cond "product" *.xml > combined.xml
products)product)Low-tech simple answer:
echo '<products>' > combined.xml
grep -vh '</\?products>\|<?xml' *.xml >> combined.xml
echo '</products>' >> combined.xml
Limitations:
combined.xml will be wiped out instead of getting included.Each of these limitations can be worked around, but not all of them easily.
Merging 2 trees includes the task to identify what is identical and what should be replaced. Unfortunately, this is not obvious. There is more semantic involved than what can be inferred from the source XML documents.
Consider the case where the first document has a middle level with several elements having the same tag, but different attributes. The second document adds an attribute to that middle level to an existing element, but also another child to it. One has to know the semantic.
<params>
...
<param><name>hello</name><value>world</value></param>
...
</params>
add/merge:
<params>
<param><name>hello</name><value>yellow submarine</value></param>
</params>
Another very helpful tool is yq, which aims to be jq for YAML, TOML and XML.
It can be installed via pip, the xml handling command is then called xq.
pip install yq
xq .products ?.xml --xml-output --xml-root=products > combined.xml
