How to remove duplicate nodes using XSLT?

Question

If system finds a duplicate , it should eliminate both nodes altogether. I have original xml as:

<?xml version="1.0" encoding="utf-8"?>
<RESPONSE>
  <RECORD>
    <COUNTRY_ID>2111</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2121</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2211</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2221</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3111</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3121</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3211</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3221</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2111</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2121</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2211</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2221</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2311</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>2321</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3111</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3121</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3211</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3221</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3311</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3321</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4111</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4121</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4211</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4221</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4311</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4321</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
</RESPONSE>

and the desired output should be below:

<?xml version="1.0" encoding="utf-8"?>
<RESPONSE>
<RECORD>
    <COUNTRY_ID>2311</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
<RECORD>
    <COUNTRY_ID>2321</COUNTRY_ID>
    <CITY id="2:">0</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
<RECORD>
    <COUNTRY_ID>3311</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>3321</COUNTRY_ID>
    <CITY id="3:">30</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4111</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4121</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="1:14404388">LA11CFTXX</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4211</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4221</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="2:14048042">LAZ1129YY</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4311</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="1:">1</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
  <RECORD>
    <COUNTRY_ID>4321</COUNTRY_ID>
    <CITY id="4:">100</CITY>
    <LOCATION id="3:14048629">LAZ11B91AA</LOCATION>
    <REPID id="2:">2</REPID>
    <FLIGHTID id="1:1091513">ALK</FLIGHTID>
  </RECORD>
</RESPONSE>

can we able to write an xsl to do this?

thanks

Can you define duplicate? Maybe a more compact sample would be better. — Dirk Vollmar
– Dirk Vollmar, Commented Jun 20, 2009 at 23:37

Jacob · Accepted Answer · 2009-06-20 23:52:42Z

2

Try this:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:template match="/">
    <xsl:apply-templates select="RESPONSE"/>
  </xsl:template>

  <xsl:template match="RESPONSE">
    <RESPONSE>
      <xsl:apply-templates select="RECORD"/>
    </RESPONSE>
  </xsl:template>

  <xsl:template match="RECORD">
    <xsl:variable name="this" select="."/>
    <xsl:variable name="country" select="COUNTRY_ID/text()"/>
    <xsl:if test="not(../RECORD[COUNTRY_ID/text() = $country and generate-id($this) != generate-id(.)])">
      <xsl:copy-of select="."/>
    </xsl:if>
  </xsl:template>

</xsl:stylesheet>

This is all assuming that when you say "duplicate" you mean that COUNTRY_ID has the same content. My XSLT appears to generate your desired output.

answered Jun 20, 2009 at 23:52

Jacob

79k24 gold badges159 silver badges245 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mark Schultheiss · Accepted Answer · 2009-06-21 00:07:58Z

1

This select will give you the set of when there is a duplicate COUNTRY_ID. (> 1)

select="/RESPONSE/RECORD/COUNTRY_ID/*[count(.|key('x',name())[1])>1]">

Use that set to then exclude those from your select.

I did not run a test, but it should give you the set to work with.

answered Jun 21, 2009 at 0:07

Mark Schultheiss

34.3k13 gold badges75 silver badges116 bronze badges

Collectives™ on Stack Overflow

How to remove duplicate nodes using XSLT?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related