4

I am working with flink and kafka, i got this error.

java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
    at kafka.consumer.FetchRequestAndResponseMetrics.<init>(FetchRequestAndResponseStats.scala:32)
    at kafka.consumer.FetchRequestAndResponseStats.<init>(FetchRequestAndResponseStats.scala:46)
    at kafka.consumer.FetchRequestAndResponseStatsRegistry$$anonfun$2.apply(FetchRequestAndResponseStats.scala:59)
    at kafka.consumer.FetchRequestAndResponseStatsRegistry$$anonfun$2.apply(FetchRequestAndResponseStats.scala:59)
    at kafka.utils.Pool.getAndMaybePut(Pool.scala:61)
    at kafka.consumer.FetchRequestAndResponseStatsRegistry$.getFetchRequestAndResponseStats(FetchRequestAndResponseStats.scala:63)
    at kafka.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:39)
    at kafka.javaapi.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:34)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.getPartitionsForTopic(FlinkKafkaConsumer.java:695)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.<init>(FlinkKafkaConsumer.java:281)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer082.<init>(FlinkKafkaConsumer082.java:49)
    at com.inndata.flinkkafka.ReadFromKafka.main(ReadFromKafka.java:19)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505)
    at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403)
    at org.apache.flink.client.program.Client.runBlocking(Client.java:248)
    at org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:866)
    at org.apache.flink.client.CliFrontend.run(CliFrontend.java:333)
    at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1192)
    at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1243)

i am using flink - 1.0.3 version and Kafka- kafka_2.11-0.8.2.1 and scala - 2.11.5, these are the jars i am using in my buildpath :

asm-4.0.jar,commons-codec-1.6.jar,commons-exec-1.1.jar,commons-fileupload-1.2.1.jar,commons-io-2.4.jar,commons-lang-2.5.jar,commons-logging-1.1.3.jar,commons-logging-api-1.1.jar,curator-test-3.2.0.jar,disruptor-2.10.1.jar,flink-annotations-1.0.3.jar,flink-clients_2.10-1.0.3.jar,flink-clients_2.10-1.0.3-tests.jar,flink-core-1.0.3.jar,flink-dist_2.10-1.0.3.jar,flink-java-1.0.3.jar,flink-optimizer_2.10-1.0.3.jar,flink-python_2.10-1.0.3.jar,flink-runtime_2.10-1.0.3.jar,flink-test-utils_2.10-1.0.3.jar,guava-11.0.2.jar,hadoop-client-2.6.0.jar,hadoop-mapreduce-client-core-2.6.0.jar,hamcrest-all-1.3.jar,jetty-6.1.26.jar,jetty-util-6.1.26.jar,json-simple-1.1.1.jar,junit-4.11.jar,log4j-1.2.17.jar,logback-classic-1.0.13.jar,logback-core-1.0.13.jar,slf4j-api-1.7.7.jar,slf4j-log4j12-1.7.7.jar,zkclient-0.3.jar,zookeeper-3.4.6.jar,flink-connector-kafka-0.10.2.jar,flink-streaming-java-0.10.2.jar,kafka-clients-0.8.2.1.jar,kafka_2.11-0.8.2.1-test.jar,kafka_2.11-0.8.2.1-sources.jar,kafka_2.11-0.8.2.1-scaladoc.jar,kafka_2.11-0.8.2.1-javadoc.jar,kafka_2.11-0.8.2.1.jar,scala-xml_2.11-1.0.2.jar,scala-parser-combinators_2.11-1.0.2.jar,scala-library-2.11.5.jar,metrics-core-2.2.0.jar

I tried searching for this issue, but i can't find relevent jar.Please help me to fix this.

Improve this question
5
  • 3
    Most of the NoSuchMethodErrors are due to the fact that at development/compile time you used one of version of jar and runtime you are using another version of jar (which doesn't contain the method with exact syntax you are looking for). Check jar versions between compile time & run time. Commented Jun 30, 2016 at 15:18
  • stackoverflow.com/questions/25089852/… Commented Jun 30, 2016 at 15:19
  • There are _2.10 and _2.11 scala libs in the classpath, those are not compatible Commented Jun 30, 2016 at 15:21
  • Possible duplicate of java.lang.NoSuchMethodError in Flink Commented Oct 25, 2017 at 12:45
  • @kosa How do I check that? Commented Jan 28, 2019 at 9:39

3 Answers 3

Reset to default
4

There are a few jars that might be the cause of the issue. scala-library-2.11.5.jar suggests you are using Scala 2.11, this seems consistent with kafka_2.11-0.8.2.1.jar.

But, also in your CP, I see flink-clients_2.10-1.0.3.jar and flink-runtime_2.10-1.0.3.jar among others. These are scala libraries that were compiled against Scala 2.10. That's the problem.

Binaries in Scala are not compatible across major versions. You need to find the 2.11 version of these jars if they exist or compile them to 2.11 yourself.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

  1. Navigate to Scala folder under your project folder, e.g. src -> test -> scala

  2. Right click on Scala directory and choose mark directory as Test Sources Root

  3. Now mark mark src directory as Sources Root

Improve this answer

Comments

0

Facing same issue in my codebase as well when trying to trigger a simulation using Runner class, but when triggering using the below maven command this looks fine,

mvn gatling:test -Dgatling.simulationClass=simulations.simulationClassName

this is just because of version mismatch

things to check before running it using Runner class(Worked for me)

  1. Check if java is correctly configured and reflect the version (1.8.0_251 in my case)
  2. Check if maven is correctly set (3.6.3 pointing to the same java version as above)
  3. If you're using IntelliJ, make sure you've added scala SDK in global libraries(2.12.8 in my case)
  4. Once all configured rebuild your project
  5. Trigger your simulation using Runner now, should work.
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.