Skip to content

GH-3715: Validate dictionary IDs while decoding - #3718

Open
arnabnandy7 wants to merge 1 commit into
apache:masterfrom
arnabnandy7:fix/dictionary-id-bounds
Open

GH-3715: Validate dictionary IDs while decoding#3718
arnabnandy7 wants to merge 1 commit into
apache:masterfrom
arnabnandy7:fix/dictionary-id-bounds

Conversation

@arnabnandy7

Copy link
Copy Markdown

Rationale for this change

Dictionary-encoded data pages may contain an ID that exceeds the valid range of the associated dictionary. Currently, this causes an ArrayIndexOutOfBoundsException when the dictionary implementation accesses its backing array, obscuring the actual data corruption.

What changes are included in this PR?

  • Validate decoded dictionary IDs against the dictionary's maximum ID.
  • Throw a descriptive ParquetDecodingException when an ID is outside the valid range.
  • Apply validation to typed value reads and readValueDictionaryId().
  • Add a regression test covering an out-of-range dictionary ID.

Are these changes tested?

Yes. A unit test constructs a dictionary-encoded page containing an out-of-range ID and verifies that reading it throws a descriptive ParquetDecodingException.

./mvnw -pl parquet-column -Dtest=TestDictionary test

Are there any user-facing changes?

Yes. Corrupt dictionary-encoded data now produces a descriptive ParquetDecodingException instead of exposing an internal ArrayIndexOutOfBoundsException. Valid Parquet files are unaffected.

Closes #3715

Signed-off-by: Arnab Nandy <arnab_nandy7@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[parquet-java] Intermittent out-of-bounds dictionary ID when reading dictionary-encoded Parquet pages

1 participant