Skip to content

Reserved keywords per dialect #2786

Closed as not planned
Closed as not planned

Description

@guizmaii

Problem

Reserved keywords are global. Every token outside NonReservedWord() is reserved for every dialect, so a keyword added for one dialect's syntax becomes unusable as a name everywhere:

  • ClickHouse's PREWHERE and SETTINGS, Exasol's FILE, SCRIPT, CSV, FBV and VERIFY, MySQL's XOR and DISTINCTROW, Oracle's MINUS, Salesforce's INCLUDES/EXCLUDES ...
  • DuckDB does not know any of these words; queries routinely use them as column names (settings, file, script, ...).

withDialect(...) cannot help today: it only sets lexer features and a few semantic checks, and reservation is decided by the token range (MIN_NON_RESERVED_WORD ... MAX_NON_RESERVED_WORD).

Making each such word non-reserved globally is not an option either: in ClickHouse FROM t PREWHERE ... must stay a clause, not an alias.

Proposal

Let each Dialect own its reserved-keyword rules, in both directions, with today's global rules as the default:

  1. KeywordPolicy#isName(word, position): whether a word is an unquoted name at a position (object name, function name, label after AS or ., bare alias), since databases reserve keywords differently in each (PostgreSQL's categories, used by DuckDB, are the reference).
  2. Dialect#getKeywordPolicy(): null keeps the default rules.
  3. The parser consults the policy wherever it decides "is this a name?", rejecting what the dialect reserves and accepting what it does not; constructs led by such words step aside when the word is a name.

To keep JavaCC's FIRST sets clean, keywords are accepted through dedicated productions behind semantic lookaheads, and words consumed by a syntactic lookahead before any name (MySQL's SELECT modifiers) stay as they are.

Possible follow-ups

  • Generated keywords.rst per dialect.
  • Sets for other dialects (PostgreSQL's reserved list, BigQuery's, ...).

#2787 implements it for DuckDB, checked against what DuckDB itself parses for each of its 489 keywords.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions