Open Source

The MySQL connector parsed DDL with a grammar from Positive Technologies that is no longer maintained, and the ANTLR team planned to deprecate it to avoid having several flavours of MySQL grammar.

I migrated the connector to the Oracle MySQL grammar, which the ANTLR project maintains, and then fixed the DDL parsing edge cases that surfaced. I also wrote up the migration on the Debezium blog.

Debezium had no support for graph databases, even though Neo4j is widely adopted and has shipped a Change Data Capture API since version 5.13.

I am building the integration in both directions, and each step has a design document, an implementation and a runnable example. A single message transform maps Debezium change events from a relational database into Neo4j’s create, update and delete format. A second converter turns Neo4j change events into the Debezium envelope, so a Neo4j source can feed any Debezium sink such as the JDBC sink. The next step is a Neo4j source connector.

This is the ANTLR side of the Debezium MySQL migration. The connector now depends on the Oracle MySQL grammar in ANTLR’s shared grammars-v4 repository, so every gap in that grammar becomes a failure to parse a user’s schema.

While migrating the connector I found cases where the grammar rejected valid MySQL syntax, and I fixed them upstream instead of patching them in Debezium, so every project that uses the grammar benefits.

SeaTunnel’s MySQL, Oracle, PostgreSQL and SQL Server CDC connectors were all on Debezium 1.9.8. Jumping straight to 3.x would have been too risky, because every connector would have had to change at once.

I started by refactoring the build so that each connector can depend on its own Debezium version.