Java annotation processor does not get executed during compilation

I am currently trying to implement a small fun library, that auto-generates fixture classes on compile time by using anootation processing: https://github.com/floydkretschmar/fixturize/tree/main/fixturize

The project structure is quite a simple mono-repo

|-- fixturize
    |
    |--fixturize-core
    |--fixturize-playground

fixturize-core defines both the relevant annotations as well as the annotation processor:

@SupportedAnnotationTypes("de.floydkretschmar.fixturize.annotations.Fixture")
@SupportedSourceVersion(SourceVersion.RELEASE_21)
@AutoService(AbstractProcessor.class)
public class FixtureProcessor extends AbstractProcessor {
...
}

As you can see I am using auto-service to generate META-INF/services/javax.annotation.processing.Processor which seems to work. At least the file exists as part of the generated classes. This is the build.gradle file I am using for fixturize-core

plugins {
    id "java"
    id "io.freefair.lombok" version "8.6"
}

ext {
    nashornVersion = "15.4"
    guavaVersion = "33.2.1-jre"
    autoServiceVersion = "1.1.1"

    junitVersion = "5.10.0"
    assertJVersion = "3.25.1"
    mockitoVersion = "5.12.0"
    compileTestingVersion = "0.21.0"
    lombokVersion = "1.18.34"
}

dependencies {
    implementation "com.google.auto.service:auto-service:$autoServiceVersion"
    annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion"

    implementation "com.google.guava:guava:$guavaVersion"
    implementation "org.openjdk.nashorn:nashorn-core:$nashornVersion"

    testImplementation platform("org.junit:junit-bom:$junitVersion")
    testImplementation "org.junit.jupiter:junit-jupiter"
    testImplementation "org.assertj:assertj-core:$assertJVersion"
    testImplementation "org.mockito:mockito-junit-jupiter:$mockitoVersion"
    testImplementation "com.google.testing.compile:compile-testing:$compileTestingVersion"
    testImplementation "org.projectlombok:lombok:$lombokVersion"
}

test {
    useJUnitPlatform()
}

Now in fixturize-playground I want to use the the library in the exact way, I would use it in any other project: import it, register the annotation processor and then get autogenerated fixture classes for all domain objects annotated with @Fixture. So this is what my build.gradle looks like:

plugins {
    id "java"
    id "io.freefair.lombok" version "8.6"
}

tasks.withType(JavaCompile) {
    doFirst {
        println "AnnotationProcessorPath for $name is ${options.getAnnotationProcessorPath().getFiles()}"
    }
}

compileJava {
    options.annotationProcessorPath = configurations.annotationProcessor
}

dependencies {
    implementation project(":fixturize-core")
    annotationProcessor project(":fixturize-core")

    testImplementation platform('org.junit:junit-bom:5.10.0')
    testImplementation 'org.junit.jupiter:junit-jupiter'
}

test {
    useJUnitPlatform()
}

And my annotated class:

import de.floydkretschmar.fixturize.annotations.Fixture;

@Fixture
public class Test {
    private int field;
}

When running ./gradlew clean build no class gets generated and the log in the compile task shows that my custom annotation processor is not part of the annoation processor path. I am at a bit of a loss, and i was hoping someone has an idea, what i am doing wrong.