How can i use XmlReader in PowerShell to stream big/huge XML files?

Question

I have an XML of couple of gigabytes. There are no spaces in the XML.

So I wrote a little C# code to split in single files (which has some additional code to perform some stuff e.g. randomizing while testing)

using (XmlReader MyReader = XmlReader.Create(@"d:\xml\test.xml"))
            {
                while (MyReader.Read())
                {
                    switch (MyReader.NodeType)
                    {
                        case XmlNodeType.Element:
                            if (MyReader.Name == "Customer")
                            {
                                XElement el = XElement.ReadFrom(MyReader) as XElement;
                                if (el != null)
                                {
                                    custNumber = (string)el.Element("CustNumber");
                                    output = @"d:\xml\output\" + custNumber;

                                    File.WriteAllText(output, el.ToString());
                                }                                    
                            }
                            break;
                    }
                }
            }

I then parse the resulting files with PowerShell, basically because I find it easier to work with on the server while specs can change and I can on the fly change the script.

So... what is the easiest way to convert the above to PowerShell also, putting [.Net here] before everything ? would I have to read byte for byte just in the case it has "<cust" on one line and "omer>" on the next?

Wouter · Accepted Answer · 2014-11-13 15:56:51Z

12

This should be pretty close to what you wanted to do in Powershell:

$f = [System.Xml.XmlReader]::create("d:\xml\test.xml")

while ($f.read())
{
    switch ($f.NodeType)
    {
        ([System.Xml.XmlNodeType]::Element) # Make sure to put this between brackets
        {
            if ($f.Name -eq "Customer")
            {
                $e = [System.Xml.Linq.XElement]::ReadFrom($f)

                if ($e -ne $null)
                {
                    $custNumber = [string] $e.Element("CustNumber")

                    $e.ToString() | Out-File -Append -FilePath ("d:\xml\output\"+$e.ToString())
                }
            }
            break
        }
    }
}

edited Nov 13, 2014 at 15:56

Wouter

2,0273 gold badges30 silver badges34 bronze badges

answered Nov 8, 2014 at 19:05

Micky Balladelli

10k2 gold badges36 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

edelwater Over a year ago

Had to move the switch to a if ($f.NodeType -eq [Syste,,,) and interestingly in PowerShell had to use .Value of the element contrary to C#.

Wouter Over a year ago

@edelwater: adding brackets around the switch option fixed it.

Collectives™ on Stack Overflow

How can i use XmlReader in PowerShell to stream big/huge XML files?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related