Repository navigation
BigQuery: Allow tokens after a hyphenated identifier ending in a number - #2620
LucaCappelletti94 wants to merge 4 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2620 +/- ##
=======================================
Coverage 81.99% 82.00%
=======================================
Files 42 42
Lines 35049 35050 +1
Branches 35049 35050 +1
=======================================
+ Hits 28740 28741 +1
Misses 2744 2744
Partials 3565 3565 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
127a284 to
e739b34
Compare
e739b34 to
66c2776
Compare
s5dsn-eqee
left a comment
There was a problem hiding this comment.
Looks good. Please add a few more tests and a fuzz seed.
Test example:
// Punctuation butted onto the trailing number is fine.
bigquery().one_statement_parses_to("SELECT * FROM foo-1;", "SELECT * FROM foo-1");
bigquery().verified_stmt("SELECT * FROM (SELECT 1 FROM foo-1)");
bigquery().one_statement_parses_to("CREATE TABLE t-1(a INT64)", "CREATE TABLE t-1 (a INT64)");
// A quoted word glued onto the number is rejected like a bare word.
let err = bigquery()
.parse_sql_statements("SELECT * FROM foo-1`x`")
.unwrap_err();
assert_eq!(
ParserError::ParserError(
"Expected: whitespace following hyphenated identifier, found: `x`".to_string(),
),
err
);| if requires_whitespace { | ||
| let token = self.next_token(); | ||
| if !matches!(token.token, Token::EOF | Token::Whitespace(_)) { | ||
| let token = self.peek_token_no_skip(); |
|
Added the tests and seed |
099818a to
b9a5c36
Compare
sabir-akhadov-localstack
left a comment
There was a problem hiding this comment.
One non-blocking comment inline.
| let token = self.next_token(); | ||
| if !matches!(token.token, Token::EOF | Token::Whitespace(_)) { | ||
| let token = self.peek_token_no_skip(); | ||
| if matches!(token.token, Token::Word(_) | Token::Number(_, false)) { |
There was a problem hiding this comment.
foox and `foo-bar`x parse with alias x on main and here, so pinning foo-1x`` as an error is inconsistent; a quoted word cannot glue into the identifier, so Token::Word(Word { quote_style: None, .. }) would make the numeric segment behave like the word segment.
There was a problem hiding this comment.
Added red tests and fixed
b9a5c36 to
94641a8
Compare
b6b1808 to
0e08e7f
Compare
0e08e7f to
a907231
Compare
A BigQuery identifier without quotes that ends in a number, such as
foo-123, had to be followed by whitespace or end of input.SELECT * FROM foo-1, bar-2therefore failed withwhitespace following hyphenated identifier, even though the comma is clearly not part of the name. The parser now peeks at the next token instead of consuming it and only rejects a word or number butted directly onto the trailing number, the case that would glue into the identifier as infoo-123aand is still rejected. Any other token, such as a comma, parenthesis, or keyword, may now follow the numeric segment.