According to Sebastian, I should use the @requires annotation to do that. It cannot be done with groups, because I cannot exclude them automatically depending on php version.
Btw. it does not help because I run always into parse errors by version 5.3, because of using yield and ::class...
He suggests to move the version dependent code to another files and use this:
<testsuite name="My Test Suite">
<directory suffix="Test.php" phpVersion="5.3.0" phpVersionOperator=">=">/path/to/files</directory>
<file phpVersion="5.3.0" phpVersionOperator=">=">/path/to/MyTest.php</file>
</testsuite>
The file should not be under the /path/to/files directory, unless you want it to be included...
Finally I added 2 new suffixes for tests related to higher php version:
<testsuite name="unit tests">
<directory suffix="Test.php" phpVersion="5.3.0" phpVersionOperator=">=">test/unit</directory>
<directory suffix="Test54.php" phpVersion="5.4.0" phpVersionOperator=">=">test/unit</directory>
<directory suffix="Test55.php" phpVersion="5.5.0" phpVersionOperator=">=">test/unit</directory>
</testsuite>