Large XML parsing in Python

Question

I am a novice in python and have the following task on hand.

I have a large xml file like the one below:

<Configuration>
   <Parameters>
      <Component Name='ABC'>
         <Group Name='DEF'>
            <Parameter Name='GHI'>
               <Description>
                        Some Text
                    </Description>
               <Type>Integer</Type>
               <Restriction>
                  <Level>5</Level>
               </Restriction>
               <Value>
                  <Item Value='5'/>
               </Value>
            </Parameter>
            <Parameter Name='JKL'>
               <Description>
                        Some Text
                    </Description>
               <Type>Integer</Type>
               <Restriction>
                  <Level>5</Level>
               </Restriction>
               <Value>
                  <Item Value='5'/>
               </Value>
            </Parameter>
         </Group>
         <Group Name='MNO'>
            <Parameter Name='PQR'>
               <Description>
                        Some Text
                    </Description>
               <Type>Integer</Type>
               <Restriction>
                  <Level>5</Level>
               </Restriction>
               <Value>
                  <Item Value='5'/>
               </Value>
            </Parameter>
            <Parameter Name='TUV'>
               <Description>
                        Some Text
                    </Description>
               <Type>Integer</Type>
               <Restriction>
                  <Level>5</Level>
               </Restriction>
               <Value>
                  <Item Value='5'/>
               </Value>
            </Parameter>
         </Group>
      </Component>
   </Parameters>
</Configuration>

In this xml file I have to parse through the component "ABC" go to group "MNO" and then to the parameter "TUV" and under this I have to change the item value to 10. I have tried using xml.etree.cElementTree but to no use. And lxml dosent support on the server as its running a very old version of python. And I have no permissions to upgrade the version

I have been using the following code to parse and edit a relatively small xml:

def fnXMLModification(ArgStr):
argList = ArgStr.split()
strXMLPath = argList[0]
if not os.path.exists(strXMLPath):
        fnlogs("XML File: " + strXMLPath + " does not exist.\n")
        return False
try:
    import xml.etree.cElementTree as ET
    except ImportError:
        import xml.etree.ElementTree as ET
    f=open(strXMLPath, 'rt')
tree = ET.parse(f)
ValueSetFlag = False
AttrSetFlag = False
for strXPath in argList[1:]:
    strXPathList = strXPath.split("[")
    sxPath = strXPathList[0]
    if len(strXPathList)==3:
            # both present
        AttrSetFlag = True
            ValueSetFlag = True
        valToBeSet = strXPathList[1].strip("]")
        sAttr = strXPathList[2].strip("]")
        attrList = sAttr.split(",")
    elif len(strXPathList) == 2:
        #anyone present
            if "=" in strXPathList[1]:
                    AttrSetFlag = True
                    sAttr = strXPathList[1].strip("]")
                    attrList = sAttr.split(",")
            else:
            ValueSetFlag = True
            valToBeSet = strXPathList[1].strip("]")
    node = tree.find(sxPath)
    if AttrSetFlag:
        for att in attrList:
            slist = att.split("=")
                    node.set(slist[0].strip(),slist[1].strip())
        if  ValueSetFlag:
        node.text = valToBeSet
tree.write(strXMLPath)
    fnlogs("XML File: " + strXMLPath + " has been modified successfully.\n")
    return True

Using this function I am not able to traverse the current xml as it has lot of children attributes or sub groups.

Can you edit the question to include the code you tried and what about it did not work (e.g., an exception or output different from what you expected)? — Eric Renouf
– Eric Renouf, Commented Sep 22, 2015 at 12:43
Hi, I have added the code which i am currently using. Thanks — sKen
– sKen, Commented Sep 22, 2015 at 13:02
When I see a question about parsing a "large" XML file, I expect the question to be parsing something that won't fin in memory. This isn't such (or if it is, you need to specify it more clearly: Your existing answers are DOM-based and thus need to store the entire tree in RAM!), and people who are searching for help with genuinely large files aren't likely to find this question and its answers useful, as such large files require different techniques. I'd suggest changing the title. (Perhaps "complex" would be the better word?) — Charles Duffy
– Charles Duffy, Commented Sep 22, 2015 at 13:38

Vivek Sable · Accepted Answer · 2015-09-22 13:24:54Z

import statement

import xml.etree.cElementTree as ET

Parse content by fromstring method.

root = ET.fromstring(data)

Iterate according our requirement and get target Item tag and change value of Value attribute

for component_tag in root.iter("Component"):
    if "Name" in component_tag.attrib and component_tag.attrib['Name']=='ABC':
        for group_tag in component_tag.iter("Group"):
            if "Name" in group_tag.attrib and group_tag.attrib['Name']=='MNO':
                #for value_tag in group_tag.iter("Value"):
                for item_tag in group_tag.findall("Parameter[@Name='TUV']/Value/Item"):
                    item_tag.attrib["Value"] = "10"

We can use Xpath to get target Item tag

for item_tag in root.findall("Parameters/Component[@Name='ABC']/Group[@Name='MNO']/Parameter[@Name='TUV']/Value/Item"):
    item_tag.attrib["Value"] = "10"

Use tostring method to get content.

data = ET.tostring(root)

Collectives™ on Stack Overflow

Large XML parsing in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related