New features: chunking and compression#93

Open

khou2020 wants to merge 30 commits into

masterfrom

chunk

Commits on Feb 16, 2026

add config option to enable chunking feature

khou2020
authored and
wkliao
committed
add chunk drivere

khou2020
authored and
wkliao
committed
add ncmpi_var_get/set_chunk and ncmpi_var_get/set_filter

khou2020
authored and
wkliao
committed
enable chunk driver when chunking hint is set

khou2020
authored and
wkliao
committed
rename default filter hint to nc_chunk_default_filter

khou2020
authored and
wkliao
committed
add documentation about chunked I/O driver

khou2020
authored and
wkliao
committed
bugfix:chunk driver build condition is ENABLE_CHUNKING

khou2020
authored and
wkliao
committed
always build chunk related APIs

khou2020
authored and
wkliao
committed
bug fix: add ncchunkio to DIST_DIR

khou2020
authored and
wkliao
committed
bugfix: add new m4 scripts to extra dist

khou2020
authored and
wkliao
committed
bugfix: add ncchkioi_profile_timers.m4 to extra_dist

khou2020
authored and
wkliao
committed
bugfix: add nchunk io to m4 include

khou2020
authored and
wkliao
committed
define ENABLE_COMPRESSION if --enable-zlib or --enable-sz is set
wkliao
committed
add example of using chunking and compression
wkliao
committed
run chunk_compress.c only when compression is enabled
wkliao
committed
Remove unused variable, initialize err to NC_NOERR
wkliao
committed
add chunking example, examples/C/chunk_io.c
wkliao
committed
Add verbose to print which processes perform compression/decompression
wkliao
committed
comment out unused printf
wkliao
committed
test: add examples/C/chunk_2D.c which uses 2D checkerboard partitioning pattern
wkliao
committed
examples/C/chunk_2D.c one wait_all for all nonblocking reads
wkliao
committed
chunk_2D.c: fix use of NX by mistake
wkliao
committed
use MPI_COMM_SELF to call MPI_Pack and MPI_Unpack
wkliao
committed
bug fix: sbuf index, array of size nsend
wkliao
committed
chunk: fix printing expected number of records
wkliao
committed
chunk_2D.c add partitioning along time for decompression
wkliao
committed
bug fix: all ranks participate wait call whenin collective data mode
wkliao
committed
add test program for chunking nonblocking API call
wkliao
committed
new example program for chunking/compression feature: chunk_compress_FLDS.c
wkliao
committed
rebase master branch
wkliao
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New features: chunking and compression#93

New features: chunking and compression#93
khou2020 wants to merge 30 commits into
masterfrom
chunk

Commits on Feb 16, 2026

add config option to enable chunking feature

add chunk drivere

add ncmpi_var_get/set_chunk and ncmpi_var_get/set_filter

enable chunk driver when chunking hint is set

rename default filter hint to nc_chunk_default_filter

add documentation about chunked I/O driver

bugfix:chunk driver build condition is ENABLE_CHUNKING

always build chunk related APIs

bug fix: add ncchunkio to DIST_DIR

bugfix: add new m4 scripts to extra dist

bugfix: add ncchkioi_profile_timers.m4 to extra_dist

bugfix: add nchunk io to m4 include

define ENABLE_COMPRESSION if --enable-zlib or --enable-sz is set

add example of using chunking and compression

run chunk_compress.c only when compression is enabled

Remove unused variable, initialize err to NC_NOERR

add chunking example, examples/C/chunk_io.c

Add verbose to print which processes perform compression/decompression

comment out unused printf

test: add examples/C/chunk_2D.c which uses 2D checkerboard partitioning pattern

examples/C/chunk_2D.c one wait_all for all nonblocking reads

chunk_2D.c: fix use of NX by mistake

use MPI_COMM_SELF to call MPI_Pack and MPI_Unpack

bug fix: sbuf index, array of size nsend

chunk: fix printing expected number of records

chunk_2D.c add partitioning along time for decompression

bug fix: all ranks participate wait call whenin collective data mode

add test program for chunking nonblocking API call

new example program for chunking/compression feature: chunk_compress_FLDS.c

rebase master branch