Skip to content

Lex MySQL backslash escapes consistently in string literals - #2790

Merged
manticore-projects merged 1 commit into
JSQLParser:masterfrom
minleejae:fix/mysql-string-literal-escapes-20261013
Oct 13, 2026
Merged

manticore-projects merged 1 commit into
JSQLParser:masterfrom
minleejae:fix/mysql-string-literal-escapes-20261013

Conversation

@minleejae

@minleejae minleejae commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Lex string literals with backslash escapes the way MySQL reads them. This applies with Dialect.MYSQL, Dialect.MARIADB or withBackslashEscapeCharacter(true). Today:

  • SELECT 'it\'s ''x''' FROM t silently splits into the string 'it\'s ' and an alias 'x'''.
  • 'a\0b\'c', '\Z\'', 'a''b\'c' and 'x\%\'y' fail with a lexical error.
  • The double-quoted "it\"s" fails as well.

MySQL 8.4, MySQL 8.0 and MariaDB 11.4 read each of these as a single string.

Current design (As-is)

  • Single-quoted literals have two pattern alternatives:
    • a backslash-aware one that knows only \n \t \b \r \f \\ \" \' and no doubled quote;
    • a plain one without backslash escapes.
  • The token action patches the longest match with \'/\'' heuristics.
  • Double-quoted strings reuse the quoted-identifier token, which has no backslash escapes.

Target design (To-be)

  • Let the backslash-aware alternatives of both quote styles read any escaped character and doubled quotes.
  • In the token action, trim the match to the closing quote under the rules in effect. One helper serves both quote styles:
    • A backslash escapes the next character only when backslash escapes are enabled, or in a PostgreSQL E'' string.
    • Otherwise the literal ends at the first quote that is not doubled.
  • These keep their previous tokens: standard strings, LIKE ... ESCAPE '\', Oracle q'...' literals, and quoted identifiers such as "col\".

Validation

  • New tests cover:
    • mixed escapes in MySQL;
    • PostgreSQL E'' and standard strings;
    • the unchanged default reading;
    • double-quoted strings in the MySQL, MariaDB, BigQuery and Databricks presets;
    • each feature switch alone and both together;
    • quoted identifiers.
  • Java 17 offline Gradle spotlessApply check: 10,096 tests, 10,071 passed, 25 skipped, zero failures/errors. The grammar choice conflict check passes. The full suite also passes unchanged against the reworked tokens.
  • Ran 32 statements on MySQL 8.4, MySQL 8.0, MariaDB 11.4 and PostgreSQL 18.3, in original and rendered form. All succeed both ways. "a\""b", which MySQL rejects, is rejected by the parser too.

Integration validation

Applied #2788, #2789, #2790, #2791 and #2792 together on master (166f0a27):

With backslash escapes enabled, as in the MySQL and MariaDB presets,
the lexer recognized only a few escapes and no doubled quote in the
same literal. 'it\'s ''x''' silently split into a string and an alias,
'a\0b\'c', '\Z\'' and 'a''b\'c' failed, and the double-quoted "it\"s"
ended at the escaped quote.

Let both quote styles read any backslash escape and doubled quotes,
then trim the token to the closing quote of the rules in effect. A
backslash only escapes with backslash escapes enabled or in PostgreSQL
E'' strings, so standard strings, LIKE ... ESCAPE '\' and quoted
identifiers such as "col\" are unchanged.

Signed-off-by: minleejae <mmj9808@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants