Skip to content

Fix TS_2DIFF float/double page compatibility#796

Open
hongzhi-gao wants to merge 5 commits intodevelopfrom
fix/cpp-ts2diff
Open

Fix TS_2DIFF float/double page compatibility#796
hongzhi-gao wants to merge 5 commits intodevelopfrom
fix/cpp-ts2diff

Conversation

@hongzhi-gao
Copy link
Copy Markdown
Contributor

@hongzhi-gao hongzhi-gao commented Apr 30, 2026

Summary

  • Fix C++ TS_2DIFF float/double page layout to match Java overflow-page encoding.
  • Fix Java table-read time decoder selection to use actual time chunk encoding (instead of global default), preventing mis-decoding on PLAIN time chunks.
  • Harden googletest zip handling in CMake: reject/remove empty or invalid zip files and only accept valid ZIP magic before enabling tests.

Encoding Layout

Legacy raw block (old C++ path):
[ TS_2DIFF inner block ]

Java-compatible overflow page (fixed C++ path):
[ outer_magic(varint=2147483646) ]
[ count(varint) ]
[ bitmap_under ]
[ bitmap_over ]
[ inner_max_point(varint=2) ]
[ TS_2DIFF inner block payload ]

Test Input

Float values: 3.123456768E20f, NaN
Expected encoded hex (Java baseline):

FE FF FF FF 07 02 00 03 02 00 00 00 01 00 00 00 00 1E 38 8A AA 61 87 75 56

Verification

  • C++ encoded output matched the Java baseline byte-for-byte.
  • Java table query can read both C++-generated and Java-generated files

…idation.

Align C++ float/double TS_2DIFF flush/read behavior with Java overflow-page layout, and prevent 0-byte/corrupt googletest archives from being treated as successful downloads during test configuration.
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 30, 2026

Codecov Report

❌ Patch coverage is 67.00508% with 65 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.67%. Comparing base (5e2130f) to head (13bb80e).

Files with missing lines Patch % Lines
cpp/src/encoding/ts2diff_decoder.h 55.55% 36 Missing ⚠️
cpp/src/encoding/ts2diff_encoder.h 74.10% 29 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #796      +/-   ##
===========================================
- Coverage    62.74%   62.67%   -0.08%     
===========================================
  Files          706      707       +1     
  Lines        42842    42858      +16     
  Branches      6326     6376      +50     
===========================================
- Hits         26882    26860      -22     
- Misses       14968    15006      +38     
  Partials       992      992              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add optional TSFILE_OPTIMIZATION_FLAGS overrides in project/examples CMake while removing hardcoded optimization settings so library builds follow common CMake integration behavior.
Use Java-compatible default maxPoint handling for float/double TS_2DIFF encode/decode while preserving legacy raw-block compatibility, and add a byte-level regression test for the documented float/NaN hex sequence.
Use the actual time chunk encoding instead of global default when building time decoders in table read paths, preventing wrong TS_2DIFF decoding on PLAIN time chunks.
Drop nonessential explanatory comments in TS_2DIFF codec changes and remove unused defaultTimeDecoder from AbstractChunkReader to keep the PR focused for review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants