Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[submodule "duckdb_mooncake"]
path = duckdb_mooncake
url = https://github.com/Mooncake-Labs/duckdb_mooncake.git
[submodule "moonlink"]
path = moonlink
url = https://github.com/Mooncake-Labs/moonlink.git
[submodule "pg_duckdb"]
path = pg_duckdb
url = https://github.com/duckdb/pg_duckdb.git
[submodule "ivy_moonlink"]
path = ivy_moonlink
url = https://github.com/IvorySQL/ivy_moonlink.git
[submodule "ivy_duckdb"]
path = ivy_duckdb
url = https://github.com/IvorySQL/ivy_duckdb.git
[submodule "ivy_duckdb_mooncake"]
path = ivy_duckdb_mooncake
url = https://github.com/IvorySQL/ivy_duckdb_mooncake.git
2 changes: 1 addition & 1 deletion .vscode/c_cpp_properties.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
{
"name": "Dev Container",
"includePath": [
"${workspaceFolder}/pg_duckdb/**",
"${workspaceFolder}/ivy_duckdb/**",
"/home/vscode/.pgrx/18.*/src/**"
]
}
Expand Down
4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ pg18 = ["pgrx/pg18"]

[dependencies]
bincode = "2"
moonlink_rpc.path = "moonlink/src/moonlink_rpc"
moonlink_service = { path = "moonlink/src/moonlink_service", optional = true }
moonlink_rpc.path = "ivy_moonlink/src/moonlink_rpc"
moonlink_service = { path = "ivy_moonlink/src/moonlink_service", optional = true }
native-tls = "0.2"
pgrx = "0.16.1"
postgres.git = "https://github.com/Mooncake-Labs/rust-postgres.git"
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ RUN cargo install --locked cargo-pgrx@0.16.1 \
WORKDIR pg_mooncake

COPY Cargo.toml Makefile pg_mooncake.control .
COPY moonlink moonlink
COPY ivy_moonlink ivy_moonlink
COPY src src

RUN make package
Expand Down
14 changes: 7 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ PG_VERSION ?= pg18
export PG_CONFIG := $(shell cargo pgrx info pg-config $(PG_VERSION))
MAKEFLAGS += --no-print-directory

.PHONY: help clean duckdb_mooncake format install package pg_duckdb run test
.PHONY: help clean ivy_duckdb_mooncake format install package ivy_duckdb run test

help:
@echo "Usage: make <COMMAND> [OPTIONS]"
@echo ""
@echo "Commands:"
@echo " run Build and run pg_mooncake for development"
@echo " install Build and install pg_mooncake"
@echo " pg_duckdb Build and install pg_duckdb"
@echo " ivy_duckdb Build and install ivy_duckdb"
@echo " package Build an installation package for release"
@echo " format Format the codebase"
@echo " test Run all tests"
Expand All @@ -22,8 +22,8 @@ help:
clean:
@cargo clean

duckdb_mooncake:
@$(MAKE) -C duckdb_mooncake GEN=ninja OVERRIDE_GIT_DESCRIBE=v1.4.1
ivy_duckdb_mooncake:
@$(MAKE) -C ivy_duckdb_mooncake GEN=ninja OVERRIDE_GIT_DESCRIBE=v1.4.1

format:
@cargo fmt
Expand All @@ -35,10 +35,10 @@ install:
package:
@cargo pgrx package

pg_duckdb:
@$(MAKE) -C pg_duckdb install -j$(shell nproc)
ivy_duckdb:
@$(MAKE) -C ivy_duckdb install -j$(shell nproc)

run: pg_duckdb
run: ivy_duckdb
@cargo pgrx run

test:
Expand Down
57 changes: 18 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,60 +1,44 @@
<div align=center>
<h1 align=center>pg_mooncake 🥮</h1>
<h4 align=center>Real-time analytics on Postgres tables</h4>
<h1 align=center>ivy_mooncake 🥮</h1>
<h4 align=center>Real-time analytics on Postgres tables (IvorySQL distribution)</h4>

[![][docs-shield]][docs-link]
[![][license-shield]][license-link]
[![][slack-shield]][slack-link]
[![][x-shield]][x-link]
</div>

## Overview

**pg_mooncake** is a Postgres extension that creates a columnstore mirror of your Postgres tables in [Iceberg][iceberg-link], enabling fast analytics queries with sub-second freshness:
**ivy_mooncake** is the IvorySQL distribution of [pg_mooncake][upstream-link], a Postgres extension that creates a columnstore mirror of your Postgres tables in [Iceberg][iceberg-link], enabling fast analytics queries with sub-second freshness:
- **Real-time ingestion** powered by [moonlink][moonlink-link] for streaming and batched INSERT/UPDATE/DELETE.
- **Fast analytics** accelerated by [DuckDB][pgduckdb-link], ranking top 10 on [ClickBench][clickbench-link].
- **Postgres-native** allowing you to query a columnstore table just like a regular Postgres table.
- **Iceberg-native** making your data readily accessible by other query engines.

## Installation

### Option 1: Docker

For new users, we recommend using the Docker image to get started quickly:
```bash
docker run --name mooncake --rm -e POSTGRES_PASSWORD=password mooncakelabs/pg_mooncake
```
This fork tracks IvorySQL-maintained branches of `pg_duckdb`, `moonlink`, and `duckdb_mooncake` (`ivy_duckdb`, `ivy_moonlink`, `ivy_duckdb_mooncake`). The repository name is `ivy_mooncake`, but the extension it installs is unchanged — `pg_mooncake` — so SQL-side compatibility with upstream is preserved.

This will start Postgres with pg_mooncake preinstalled. You can then connect to it using `psql` with the default user `postgres`:
```bash
docker exec -it mooncake psql -U postgres
```

### Option 2: From Source
## Installation

To build pg_mooncake, first install [Rust][rust-install], [pgrx][pgrx-install], and [the build tools for DuckDB][duckdb-install].
To build ivy_mooncake, first install [Rust][rust-install], [pgrx][pgrx-install], and [the build tools for DuckDB][duckdb-install].

Then, clone the repository:
Then, clone the repository with submodules:
```bash
git clone --recurse-submodules https://github.com/Mooncake-Labs/pg_mooncake.git
git clone --recurse-submodules https://github.com/IvorySQL/ivy_mooncake.git
```

To build and install for Postgres versions 14-18, run:
```bash
cargo pgrx init --pg18=$(which pg_config) # Replace with your Postgres version
make pg_duckdb # Skip if pg_duckdb is already installed
make ivy_duckdb # Skip if ivy_duckdb is already installed
make install PG_VERSION=pg18
```

Finally, add pg_mooncake to `shared_preload_libraries` in your `postgresql.conf` file and enable logical replication:
Finally, add `pg_mooncake` to `shared_preload_libraries` in your `postgresql.conf` file and enable logical replication:
```ini
duckdb.allow_community_extensions = true
shared_preload_libraries = 'pg_duckdb,pg_mooncake'
wal_level = logical
```

For a complete walkthrough, refer to our [Dockerfile][dockerfile-link].

## Quick Start

First, create the pg_mooncake extension:
Expand Down Expand Up @@ -91,26 +75,21 @@ Finally, query `trades_iceberg` to see that it reflects the up-to-date state of
SELECT avg(price) FROM trades_iceberg WHERE symbol = 'AMZN';
```

## Contributing
Note: The repository is renamed to `ivy_mooncake` only at the GitHub-fork level. Inside the repo, the extension, the cdylib, and all SQL-level identifiers (`pg_mooncake` extension, `mooncake.*` schema, `USING mooncake` access method, `INSTALL mooncake FROM community`) are intentionally unchanged from upstream so SQL written against `pg_mooncake` continues to work.

## Upstream

pg_mooncake is an open-source project maintained by [Mooncake Labs][mooncake-link] and licensed under the [MIT License][license-link]. We'd love your help to make it even better! Join [our Slack][slack-link], participate in [discussions][discussions-link], open [issues][issues-link] to report bugs or suggest features, contribute code and documentation, or help us improve the project in any way. All contributions are welcome! 🥮
ivy_mooncake is a fork of [Mooncake-Labs/pg_mooncake][upstream-link] under the [MIT License][license-link].

[clickbench-link]: https://www.mooncake.dev/blog/clickbench-v0.1
[discussions-link]: https://github.com/Mooncake-Labs/pg_mooncake/discussions
[dockerfile-link]: https://github.com/Mooncake-Labs/pg_mooncake/blob/main/Dockerfile
[docs-link]: https://docs.mooncake.dev/
[docs-shield]: https://img.shields.io/badge/docs-mooncake?logo=readthedocs&logoColor=white
[duckdb-install]: https://duckdb.org/docs/stable/dev/building/overview.html#prerequisites
[iceberg-link]: https://iceberg.apache.org/
[issues-link]: https://github.com/Mooncake-Labs/pg_mooncake/issues
[license-link]: https://github.com/Mooncake-Labs/pg_mooncake/blob/main/LICENSE
[license-link]: ./LICENSE
[license-shield]: https://img.shields.io/badge/License-MIT-blue
[mooncake-link]: https://mooncake.dev/
[moonlink-link]: https://github.com/Mooncake-Labs/moonlink
[pgduckdb-link]: https://github.com/duckdb/pg_duckdb
[moonlink-link]: https://github.com/IvorySQL/ivy_moonlink
[pgduckdb-link]: https://github.com/IvorySQL/ivy_duckdb
[pgrx-install]: https://github.com/pgcentralfoundation/pgrx?tab=readme-ov-file#getting-started
[rust-install]: https://www.rust-lang.org/tools/install
[slack-link]: https://join.slack.com/t/mooncake-devs/shared_invite/zt-2sepjh5hv-rb9jUtfYZ9bvbxTCUrsEEA
[slack-shield]: https://img.shields.io/badge/Mooncake%20Devs-purple?logo=slack
[x-link]: https://x.com/mooncakelabs
[x-shield]: https://img.shields.io/twitter/url?label=%40mooncakelabs&url=https%3A%2F%2Fx.com%2Fmooncakelabs
[upstream-link]: https://github.com/Mooncake-Labs/pg_mooncake
1 change: 1 addition & 0 deletions ivy_duckdb
Submodule ivy_duckdb added at 5eb63f
1 change: 1 addition & 0 deletions ivy_moonlink
Submodule ivy_moonlink added at bc37b3
1 change: 0 additions & 1 deletion moonlink
Submodule moonlink deleted from c1a99e
1 change: 0 additions & 1 deletion pg_duckdb
Submodule pg_duckdb deleted from 7daa8e
2 changes: 1 addition & 1 deletion rust-toolchain.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[toolchain]
channel = "1.88.0"
channel = "1.91.1"
components = ["clippy", "rustfmt"]