Skip to content

Various bugfixes - #121

Closed
nickchomey wants to merge 3 commits into
surrealdb:mainfrom
nickchomey:various-fixes
Closed

Various bugfixes#121
nickchomey wants to merge 3 commits into
surrealdb:mainfrom
nickchomey:various-fixes

Conversation

@nickchomey

Copy link
Copy Markdown

This is purely vibe coded. I didnt even look at the code. You probably only want to look at it as reference. But I am now able to connect mysql and mariadb, and they sync some basic data.

It fixes 3 issues:

Fix 1: MySQL 8.0 caching_sha2_password RSA auth (wire.rs)

  • negotiate_tls: For --tls-mode preferred, skip TLS entirely instead of trying and failing on self-signed certs. The RSA-based full auth path handles the password exchange.
  • Auth handler (0x04): Implemented the non-TLS caching_sha2_password full auth flow: request the RSA public key from MySQL, XOR the null-terminated password with SHA256(scramble), RSA-OAEP encrypt with SHA-1, and send.
  • Handshake parser: Fixed scramble length parsing — auth_plugin_data_len includes the trailing NUL byte, so we now subtract 1 to get the real 20-byte SCRAMBLE_LENGTH instead of reading 21 bytes.

Fix 2: MySQL zero dates (0000-00-00) (reverse.rs + binlog.rs)

  • reverse.rs: Wrapped extract_date, extract_time, extract_datetime calls in the full_sync path to return null values instead of crashing when MySQL emits zero dates (e.g., WordPress auto-drafts with post_date_gmt = 0000-00-00 00:00:00).
  • binlog.rs: Same treatment for the binlog stream path.

Fix 3: Composite primary keys (rows.rs + surreal2-sink/src/rows.rs)

  • Added UniversalValue::Array handling to universal_value_to_surreal_id — tables with composite primary keys (like wp_term_relationships with (object_id, term_taxonomy_id)) now get a string record ID like "4_2" instead of crashing.

I think you are definitely going to want to test all db sources with extreme (max, min, null, invalid) values for all data types. This is something I spent a significant amount of time doing when I initially built my own mysql/mariadb -> surreal stuff with Conduit then Benthos. I did it all via code in a private repo, so I dont have a sample SQL file that I can just share as a starting point with all the extreme values. I hope this helps

@nickchomey

nickchomey commented Jul 20, 2026

Copy link
Copy Markdown
Author

I see now that --tls-mode required doesn't work still. preferred and disabled work though. It probably made a bad hack. Also, the tls issues were with mysql 8.0. mariadb 10.11 worked. No idea about other versions. I dont have time to explore all of these things

@mumoshu

mumoshu commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@nickchomey We got a major refactor related to #118 (#123) that conflicted with all those changes, so I've addressed all in #125 #126 #127 #132.
Hoping it works out of the box on the next change!
Thank you very much for all your feedback!

@mumoshu mumoshu closed this Jul 23, 2026
@nickchomey

Copy link
Copy Markdown
Author

You work fast!

However, I doubt I'm going to test the referenced changes, as I have enough experience with CDC now (particularly with mysql/mariadb) to know it's very tricky, so I have to figure that there's still all sorts of edge cases that are not being handled yet and won't be found til people stumble on and report them.

Conversely, Debezium has a decade of scrutiny and improvements behind it, that come with being the gold standard for CDC. The mariadb test suite alone takes 25 min to run (I did so yesterday while adding v12.3 support). And it will have a simple native binary available soon. Plus, benthos makes it easy for me to do custom processing and custom surrealdb output connector.

But I'm happy to provide feedback/brainstorm in #118 if you have any questions on use cases etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants