[Fix][Connector-V2] Fix Parquet INT96 mixed-case field matching - #11067
Merged
davidzollo merged 1 commit intoJun 18, 2026
Merged
Conversation
DanielLeens
reviewed
Jun 12, 2026
DanielLeens
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the fix. I reviewed the latest head from the Parquet read path down to the new regression test.
What This PR Solves
- User pain: mixed-case configured field paths do not match Parquet INT96 timestamp fields reliably, which breaks expected field mapping.
- Fix approach: normalize the relevant path components consistently with Locale.ROOT before matching configured paths against Parquet schema field names.
- One-line summary: I did not find a new code-side blocker in the Parquet matching logic on the latest head; the remaining blocker is the current red Build signal.
Runtime Chain Rechecked
Parquet read path
-> configured field path list
-> normalize relevant path tokens with Locale.ROOT
-> Parquet schema walk
-> normalize schema field names the same way
-> match mixed-case configured path to INT96 timestamp field reliably
Findings
I did not find a new runtime-path blocker in the current Parquet field-matching logic.
Issue 1: the top-level Build is still red, and the visible failures do not point back to this Parquet path specifically
- Location: GitHub Build check
80821242970 - Evidence I confirmed:
the current run includes failing/cancelled jobs such as unit-test (Windows), connector-file-sftp-it, and other broad matrix churn rather than a focused Parquet regression signal. - Why this matters:
the code-side fix itself looks coherent, but the PR still should not merge while the top-level Build is red. - Better fix:
get the current Build green on the latest head, then re-evaluate only if a targeted Parquet/file-reader failure remains. - Severity: High
- Already raised by others: No.
Test Coverage / Stability
- The added regression test for mixed-case field matching is well targeted.
- Stability rating: Stable.
- The changed path is deterministic and not timing-sensitive.
Merge Conclusion
Conclusion: can merge after fixes
- Blocking items
- Issue 1: get the current Build green on this head.
- Suggested follow-up
- None from the Parquet code path I reviewed.
Overall, the matching fix itself looks good on the latest head. The remaining blocker is the CI gate, not a new code-side issue.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of this pull request
Fixes #10928.
When
parquet_avro_write_timestamp_as_int96 = true, Parquet schema field names are normalized to lowercase. However, the INT96 matching set kept the original SeaTunnel field names, such ascreateTime.For mixed-case timestamp columns, this made schema generation and value conversion disagree:
createtimeand created a normal timestamp fieldcreateTimeand wrote an INT96GenericData.FixedvalueThat could fail while writing Parquet rows.
Does this PR introduce any user-facing change?
No. It only fixes Parquet INT96 writing for mixed-case field names.
How was this patch tested?
./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base -Dtest=ParquetWriteStrategyTest#testParquetWriteInt96WithMixedCaseTimestampColumn test./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base -Dtest=ParquetWriteStrategyTest test./mvnw spotless:apply./mvnw -q -DskipTests verify./mvnw -pl seatunnel-connectors-v2/connector-file/connector-file-base test