wal2mongo
使用逻辑解码捕获MongoDB JSON格式的CDC变更
仓库
HighgoSoftware/wal2mongo
https://github.com/HighgoSoftware/wal2mongo
源码
wal2mongo-1.0.7.tar.gz
wal2mongo-1.0.7.tar.gz
概览
| 扩展包名 | 版本 | 分类 | 许可证 | 语言 |
|---|---|---|---|---|
wal2mongo | 1.0.7 | ETL | Apache-2.0 | C |
| ID | 扩展名 | Bin | Lib | Load | Create | Trust | Reloc | 模式 |
|---|---|---|---|---|---|---|---|---|
| 9640 | wal2mongo | 否 | 是 | 否 | 否 | 否 | 否 | - |
| 相关扩展 | mongo_fdw wal2json decoderbufs decoder_raw documentdb pglogical test_decoding pgoutput |
|---|
版本
| 类型 | 仓库 | 版本 | PG 大版本 | 包名 | 依赖 |
|---|---|---|---|---|---|
| EXT | PIGSTY | 1.0.7 | 1817161514 | wal2mongo | - |
| RPM | PIGSTY | 1.0.7 | 1817161514 | wal2mongo_$v | - |
| DEB | PIGSTY | 1.0.7 | 1817161514 | postgresql-$v-wal2mongo | - |
| OS / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
| el8.x86_64 | PIGSTY MISS | PIGSTY MISS | PGDG 1.0.7 el8.x86_64.pg16 : wal2mongo_16 wal2mongo_16-1.0.7-1PGDG.rhel8.x86_64.rpm
| PGDG 1.0.7 el8.x86_64.pg15 : wal2mongo_15 wal2mongo_15-1.0.7-1PGDG.rhel8.x86_64.rpm
| PGDG 1.0.7 el8.x86_64.pg14 : wal2mongo_14 wal2mongo_14-1.0.7-1PGDG.rhel8.x86_64.rpm
|
| el8.aarch64 | PIGSTY MISS | PIGSTY MISS | PGDG 1.0.7 el8.aarch64.pg16 : wal2mongo_16 wal2mongo_16-1.0.7-1PGDG.rhel8.aarch64.rpm
| PGDG 1.0.7 el8.aarch64.pg15 : wal2mongo_15 wal2mongo_15-1.0.7-1PGDG.rhel8.aarch64.rpm
| PGDG 1.0.7 el8.aarch64.pg14 : wal2mongo_14 wal2mongo_14-1.0.7-1PGDG.rhel8.aarch64.rpm
|
| el9.x86_64 | PIGSTY MISS | PIGSTY MISS | PGDG 1.0.7 el9.x86_64.pg16 : wal2mongo_16 wal2mongo_16-1.0.7-1PGDG.rhel9.x86_64.rpm
| PGDG 1.0.7 el9.x86_64.pg15 : wal2mongo_15 wal2mongo_15-1.0.7-1PGDG.rhel9.x86_64.rpm
| PGDG 1.0.7 el9.x86_64.pg14 : wal2mongo_14 wal2mongo_14-1.0.7-1PGDG.rhel9.x86_64.rpm
|
| el9.aarch64 | PIGSTY MISS | PIGSTY MISS | PGDG 1.0.7 el9.aarch64.pg16 : wal2mongo_16 wal2mongo_16-1.0.7-1PGDG.rhel9.aarch64.rpm
| PGDG 1.0.7 el9.aarch64.pg15 : wal2mongo_15 wal2mongo_15-1.0.7-1PGDG.rhel9.aarch64.rpm
| PGDG 1.0.7 el9.aarch64.pg14 : wal2mongo_14 wal2mongo_14-1.0.7-1PGDG.rhel9.aarch64.rpm
|
| el10.x86_64 | PIGSTY MISS | PIGSTY MISS | PGDG 1.0.7 el10.x86_64.pg16 : wal2mongo_16 wal2mongo_16-1.0.7-3PGDG.rhel10.x86_64.rpm
| PGDG 1.0.7 el10.x86_64.pg15 : wal2mongo_15 wal2mongo_15-1.0.7-3PGDG.rhel10.x86_64.rpm
| PGDG 1.0.7 el10.x86_64.pg14 : wal2mongo_14 wal2mongo_14-1.0.7-3PGDG.rhel10.x86_64.rpm
|
| el10.aarch64 | PIGSTY MISS | PIGSTY MISS | PGDG 1.0.7 el10.aarch64.pg16 : wal2mongo_16 wal2mongo_16-1.0.7-3PGDG.rhel10.aarch64.rpm
| PGDG 1.0.7 el10.aarch64.pg15 : wal2mongo_15 wal2mongo_15-1.0.7-3PGDG.rhel10.aarch64.rpm
| PGDG 1.0.7 el10.aarch64.pg14 : wal2mongo_14 wal2mongo_14-1.0.7-3PGDG.rhel10.aarch64.rpm
|
| d12.x86_64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 d12.x86_64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~bookworm_amd64.deb
| PIGSTY 1.0.7 d12.x86_64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~bookworm_amd64.deb
| PIGSTY 1.0.7 d12.x86_64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~bookworm_amd64.deb
|
| d12.aarch64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 d12.aarch64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~bookworm_arm64.deb
| PIGSTY 1.0.7 d12.aarch64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~bookworm_arm64.deb
| PIGSTY 1.0.7 d12.aarch64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~bookworm_arm64.deb
|
| d13.x86_64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 d13.x86_64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~trixie_amd64.deb
| PIGSTY 1.0.7 d13.x86_64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~trixie_amd64.deb
| PIGSTY 1.0.7 d13.x86_64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~trixie_amd64.deb
|
| d13.aarch64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 d13.aarch64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~trixie_arm64.deb
| PIGSTY 1.0.7 d13.aarch64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~trixie_arm64.deb
| PIGSTY 1.0.7 d13.aarch64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~trixie_arm64.deb
|
| u22.x86_64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 u22.x86_64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~jammy_amd64.deb
| PIGSTY 1.0.7 u22.x86_64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~jammy_amd64.deb
| PIGSTY 1.0.7 u22.x86_64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~jammy_amd64.deb
|
| u22.aarch64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 u22.aarch64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~jammy_arm64.deb
| PIGSTY 1.0.7 u22.aarch64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~jammy_arm64.deb
| PIGSTY 1.0.7 u22.aarch64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~jammy_arm64.deb
|
| u24.x86_64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 u24.x86_64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~noble_amd64.deb
| PIGSTY 1.0.7 u24.x86_64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~noble_amd64.deb
| PIGSTY 1.0.7 u24.x86_64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~noble_amd64.deb
|
| u24.aarch64 | PIGSTY MISS | PIGSTY MISS | PIGSTY 1.0.7 u24.aarch64.pg16 : postgresql-16-wal2mongo postgresql-16-wal2mongo_1.0.7-1PIGSTY~noble_arm64.deb
| PIGSTY 1.0.7 u24.aarch64.pg15 : postgresql-15-wal2mongo postgresql-15-wal2mongo_1.0.7-1PIGSTY~noble_arm64.deb
| PIGSTY 1.0.7 u24.aarch64.pg14 : postgresql-14-wal2mongo postgresql-14-wal2mongo_1.0.7-1PIGSTY~noble_arm64.deb
|
构建
您可以使用 pig build 命令构建 wal2mongo 扩展的 DEB 包:
pig build pkg wal2mongo # 构建 DEB 包
安装
您可以直接安装 wal2mongo 扩展包的预置二进制包,首先确保 PGDG 和 PIGSTY 仓库已经添加并启用:
pig repo add pgsql -u # 添加仓库并更新缓存
使用 pig 或者是 apt/yum/dnf 安装扩展:
pig install wal2mongo; # 当前活跃 PG 版本安装
pig ext install -y wal2mongo -v 16 # PG 16
pig ext install -y wal2mongo -v 15 # PG 15
pig ext install -y wal2mongo -v 14 # PG 14
dnf install -y wal2mongo_16 # PG 16
dnf install -y wal2mongo_15 # PG 15
dnf install -y wal2mongo_14 # PG 14
apt install -y postgresql-16-wal2mongo # PG 16
apt install -y postgresql-15-wal2mongo # PG 15
apt install -y postgresql-14-wal2mongo # PG 14
此扩展不需要执行
CREATE EXTENSION语句
用法
一个逻辑解码输出插件,将 PostgreSQL WAL 变更格式化为 MongoDB 兼容命令,实现从 PostgreSQL 到 MongoDB 的数据复制。
配置
在 postgresql.conf 中:
wal_level = logical
max_replication_slots = 10
使用 SQL 函数
-- 创建逻辑复制槽
SELECT * FROM pg_create_logical_replication_slot('w2m_slot', 'wal2mongo');
-- 执行 DML 操作
CREATE TABLE books (id SERIAL PRIMARY KEY, title VARCHAR(100), author VARCHAR(100));
INSERT INTO books (id, title, author) VALUES (123, 'My Book', 'Author');
-- 查看变更(MongoDB 格式)
SELECT * FROM pg_logical_slot_peek_changes('w2m_slot', NULL, NULL);
-- 输出:db.books.insertOne( { id:123, title:"My Book", author:"Author" } )
-- 消费变更
SELECT * FROM pg_logical_slot_get_changes('w2m_slot', NULL, NULL);
-- 删除槽
SELECT pg_drop_replication_slot('w2m_slot');
使用 pg_recvlogical
pg_recvlogical -d postgres --slot w2m_slot --create-slot -P wal2mongo
pg_recvlogical -d postgres --slot w2m_slot --start -f -
复制到 MongoDB
输出可以直接在 MongoDB shell 中应用:
// 复制 pg_logical_slot_get_changes 的输出
db.books.insertOne( { id:123, title:"My Book", author:"Author" } )
或保存为 .js 文件并导入:
mongo < changes.js
输出格式
- INSERT:
db.<table>.insertOne( { <columns> } ) - UPDATE:
db.<table>.updateOne( { <key> }, { $set: { <changes> } } ) - DELETE:
db.<table>.deleteOne( { <key> } )
表需要主键或副本标识才能捕获 UPDATE/DELETE 操作。