How to filter a select nodeset with a PHP function?

Something that works for me is to return an instance of DOMDocumentFragment instead of an array. So to try it on your example, I saved your input as foo.xml. Then I made foo.xslt look like this:

<xsl:stylesheet version="1.0" xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
        xmlns:php="http://php.net/xsl">
    <xsl:template match="/">
        <xsl:call-template name="listing" />
    </xsl:template>
    <xsl:template match="name">
        <bar> <xsl:value-of select="text()" /> </bar>
    </xsl:template>
    <xsl:template name="listing">
        <foo>
            <xsl:for-each select="php:function('all_but_first', /p/name)">
                <xsl:apply-templates />
            </xsl:for-each>
        </foo>
    </xsl:template>
</xsl:stylesheet>

(This is mostly just your example with a xsl:stylesheet wrapper to invoke it.) And the real heart of the matter, foo.php:

<?php

function all_but_first($nodes) {
    if (($nodes == null) || (count($nodes) == 0)) {
        return ''; // Not sure what the right "nothing" return value is
    }
    $returnValue = $nodes[0]->ownerDocument->createDocumentFragment();
    array_shift($nodes);
    shuffle($nodes);
    foreach ($nodes as $node) {
        $returnValue->appendChild($node);
    }
    return $returnValue;
};

$xslDoc = new SimpleXMLElement('./foo.xslt', 0, true);
$xmlDoc = new SimpleXMLElement('./foo.xml', 0, true);

$proc = new XSLTProcessor();
$proc->registerPHPFunctions();
$proc->importStylesheet($xslDoc);
$buffer = $proc->transformToXML($xmlDoc);
echo $buffer;

?>

The important part being the call to ownerDocument->createDocumentFragment() to make the object that gets returned from the function.