Debezium connector fails connecting to MariaDB, error 'LOAD DATA LOCAL INFILE but functionality is disabled at client'

Question

We're using Debezium along with Kafka-Connect to ingest data from several MariaDB-database instances. The connectors run successfully for most databases, but for one the connector fails with the following error:

    at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:241)
    at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:218)
    at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:848)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.sql.SQLException: Server asked for stream in response to LOAD DATA LOCAL INFILE but functionality is disabled at client by 'allowLoadLocalInfile' being set to 'false'.
    at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
    at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
    at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
    at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:814)
    at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:764)
    ... 3 more

The server-side variable local_infile is set ON, so on the serverside everything should be fine. I guess a solution could be to somehow pass the client-side option allowLoadLocalInfile in the connection-string, but I can't see how you can configure that in Debezium. Also I'd find it rather strange this error pops up now, while other connectors don't have this problem. Does anyone recognize this error while using Debezium, know the cause or even have a solution?

Answer 1

I've found the solution. In the connector configuration, e.g. the json file to register/update the connector I've changed the setting: From this:

"min.row.count.to.stream.results": "0"

to this:

"min.row.count.to.stream.results": "1000"

When the property is set to 0, results are streamed. Somehow for a rather large table this resulted into the mentioned error. LOAD DATA LOCAL INFILE but functionality is disabled at client by 'allowLoadLocalInfile' being set to 'false'

I don't know the exact reason why this fails for a large table at mariadb, but at least the problem is solved.