Skip to content

Reduce object body memory copies#36

Closed
louisgls wants to merge 1 commit intomainfrom
codex/reduce-s3proxy-memory
Closed

Reduce object body memory copies#36
louisgls wants to merge 1 commit intomainfrom
codex/reduce-s3proxy-memory

Conversation

@louisgls
Copy link
Copy Markdown
Collaborator

Summary

  • avoid cloning successful S3 response bodies in the raw-response middleware
  • preallocate request and response body buffers when Content-Length is known
  • drop plaintext/ciphertext references earlier after encryption/decryption to reduce peak live memory
  • add unit coverage for fixed-size body reads and short-body errors

Context

This is a focused upstream version of the memory work discussed in #28 and in the k3s-argocd-cluster/s3proxy fork. The fork contains additional unrelated features such as request caching, B2/no-tagging support, chart edits, and workflow changes; this PR intentionally keeps only the object-body memory reduction.

Unlike #28, the raw-response middleware still stores the error body in metadata after reading it, so forwarded S3 error responses keep their raw XML body.

Validation

  • go test ./...
  • CGO_ENABLED=0 GOOS=linux go build -o /tmp/s3proxy-memory ./s3proxy/cmd

@louisgls louisgls force-pushed the codex/reduce-s3proxy-memory branch from 84c5f40 to 50200d7 Compare April 22, 2026 13:46
@louisgls louisgls closed this Apr 22, 2026
@louisgls louisgls deleted the codex/reduce-s3proxy-memory branch April 22, 2026 13:47
@louisgls louisgls changed the title [codex] Reduce object body memory copies Reduce object body memory copies Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant