Repository navigation
fix(bigquery-jdbc): preserve microsecond precision for TIME in getString() and LocalTime conversions - #14715
fix(bigquery-jdbc): preserve microsecond precision for TIME in getString() and LocalTime conversions#14715keshavdandeva wants to merge 1 commit into
TIME in getString() and LocalTime conversions#14715Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the BigQuery JDBC driver to improve the handling of TIME data types, specifically focusing on preserving microsecond precision during conversions. Key changes include introducing a new getRawElement and getRawAttribute abstraction to handle raw values consistently across different implementations, updating BigQueryTypeRegistry to correctly parse and format time values with microsecond precision, and adding comprehensive test cases to verify these improvements in arrays, structs, and standard result sets. The review comments suggest using legacy Calendar manipulation instead of java.time APIs for java.sql.Time conversions to better handle Daylight Saving Time and avoid potential precision loss.
…String()` and LocalTime conversions
f860ebd to
f2e2624
Compare
b/570669274
Summary
Fixes sub-second truncation for BigQuery
TIMEvalues (including nestedARRAY<TIME>andSTRUCT<TIME>) when accessed as strings orjava.time.LocalTime, while keepingjava.sql.Timestrictly compliant with the JDK and JDBC specifications (HH:mm:ssformat, millisecond backing).Problem
BigQuery
TIMEsupports microsecond precision (HH:mm:ss.SSSSSS). Previously:BigQueryTypeRegistry.createTimeDescriptor()converted rawTIMEvalues (Stringfrom REST JSON orLongmicroseconds from Arrow Storage Read API) toStringandLocalTimeby routing throughjava.sql.Time. Becausejava.sql.Timeonly stores milliseconds andjava.sql.Time.toString()is specified by the JDK to return"hh:mm:ss", fractional seconds were truncated or dropped.BigQueryBaseResultSet.getObject(int, Class<T>)calledgetObject(columnIndex)(producing a truncatedjava.sql.Time) before converting toLocalTime.classorString.class.BigQueryBaseArray.toString()andBigQueryBaseStruct.toString()formattedTIMEelements/attributes viajava.sql.Time.toString(), truncating fractional seconds to"HH:mm:ss"when callingResultSet.getString()onARRAY<TIME>orSTRUCT<TIME>columns.Solution
BigQueryTemporalUtility&BigQueryTypeRegistry:TIME_FORMATTER(HH:mm:sswith up to 9 optional fractional digits) inBigQueryTemporalUtility.BigQueryTypeRegistry.createTimeDescriptor()to convert rawStringandLong(epoch microseconds) directly toLocalTimeandStringwithout intermediate boxing intojava.sql.Time.BigQueryBaseResultSet(getObject(int, Class<T>)):abstract Object getObjectInternal(int columnIndex)onBigQueryBaseResultSet(implemented by the existinggetObjectInternalmethods inBigQueryJsonResultSetandBigQueryArrowResultSet).StandardSQLTypeName.TIMEwith target typeLocalTime.classorString.class, convert directly fromgetObjectInternal(columnIndex)to preserve full microsecond precision.BigQueryBaseArray&BigQueryBaseStruct(toString()):abstract Object getRawElement(int index)onBigQueryBaseArrayandabstract Object getRawAttribute(int index)onBigQueryBaseStruct, implemented in their JSON and Arrow subclasses.BigQueryBaseArray.toString()andBigQueryBaseStruct.toString()to formatStandardSQLTypeName.TIMEelements/attributes from their raw values viaBigQueryTypeRegistry.convert(..., StandardSQLTypeName.TIME, String.class, null).Array.getResultSet().getString(2)also preserves full microsecond precision forARRAY<TIME>asBigQueryJsonResultSetandBigQueryArrowResultSetread from the underlying raw values.Behavior Matrix
TIMEARRAY<TIME>STRUCT<TIME>rs.getString(...)"14:30:00.123456""[14:30:00.123456, ...]""{\"t\":\"14:30:00.123456\"}"rs.getObject(..., LocalTime.class)LocalTime(microsecond precision)rs.getArray(...).getResultSet().getString(2)"14:30:00.123456"rs.getTime(...)/getArray()/getAttributes()java.sql.Time(JDK spec compliant)java.sql.Time[]Object[]withjava.sql.TimeTesting
BigQueryTypeRegistryTest,BigQueryJsonArrayOfPrimitivesTest,BigQueryArrowArrayOfPrimitivesTest,BigQueryJsonStructTest, andBigQueryArrowStructTest.testTimeMicrosecondPrecisionInArrayAndStructinITBigQueryJDBCTestcovering both REST JSON and Storage Read (Arrow) pipelines.