Kafka JDBC Source Connector is not reading data from mysql with sql query?

Question

I have created a local database for testing purpose on MySQL. When I'm trying to read the data from the database, it's reading fine. When I use the SQL query in connector configuration, it's just running but not reading any data from the database. Any idea what might cause this?

Here is my connector config:

name=just-like
acks=0 
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
connection.url=jdbc:mysql://localhost:3306/DBtest?user=**&password=**
connection.user=**  
connection.password=**  
table.whitelist=test
mode=timestamp  
timestamp.column.name=createddate  
query= select column_name from table_name  
validate.non.null=False 
topic.prefix=sql-

Database Schema

enter image description here

Can you show your table schema and current data in the table? Please create a minimal reproducible example — OneCricketeer
– OneCricketeer, Commented Nov 14, 2018 at 15:33
You may want to review your configuration, because as posted it seems to be invalid (multiple properties on a single line), which might well be the cause of your problem (but given you botched the initial formatting of your post, I can't be sure). — Mark Rotteveel
– Mark Rotteveel, Commented Nov 14, 2018 at 16:24
Please have a look on question again. i have added required details. — DP0808
– DP0808, Commented Nov 15, 2018 at 8:35
Rather than just "select column_name from table_name", can you please show the actual query? And table.whitelist=test wouldn't read from table_name, so that seems wrong. And then localhost:3306/DBtest user=**&password=** has a space in it, so I don't think that is correct — OneCricketeer
– OneCricketeer, Commented Nov 17, 2018 at 0:36
I have tried as you suggested but as soon as i put sql command it stopped working and when i remove query it's working fine and pulling data but it's fetching all columns from a table and i want to access few. Please let me know. — DP0808
– DP0808, Commented Nov 19, 2018 at 10:02

Muhammad Arslan Akhtar · Accepted Answer · 2019-05-28 03:29:02Z

1

We shouldn't use the tag table.whitelist with the custom query. If you want to use the query tag, I suggest removing the table.whitelist tag. You can't use them both together.

answered May 28, 2019 at 3:29

Muhammad Arslan Akhtar

5578 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Kafka JDBC Source Connector is not reading data from mysql with sql query?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related