pgspider_ext
使用多种FDW访问远程数据库服务器
仓库
pgspider/pgspider_ext
https://github.com/pgspider/pgspider_ext
源码
pgspider_ext-1.3.0.tar.gz
pgspider_ext-1.3.0.tar.gz
概览
| 扩展包名 | 版本 | 分类 | 许可证 | 语言 |
|---|---|---|---|---|
pgspider_ext | 1.3.0 | FDW | PostgreSQL | C |
| ID | 扩展名 | Bin | Lib | Load | Create | Trust | Reloc | 模式 |
|---|---|---|---|---|---|---|---|---|
| 8540 | pgspider_ext | 否 | 是 | 否 | 是 | 否 | 是 | - |
| 相关扩展 | plproxy wrappers multicorn postgres_fdw citus mysql_fdw oracle_fdw mongo_fdw |
|---|
版本
| 类型 | 仓库 | 版本 | PG 大版本 | 包名 | 依赖 |
|---|---|---|---|---|---|
| EXT | PGDG | 1.3.0 | 1817161514 | pgspider_ext | - |
| RPM | PGDG | 1.3.0 | 1817161514 | pgspider_ext_$v | - |
| DEB | PIGSTY | 1.3.0 | 1817161514 | postgresql-$v-pgspider-ext | - |
| OS / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
| el8.x86_64 | PGDG MISS | PGDG 1.3.0 el8.x86_64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel8.x86_64.rpm
| PGDG 1.3.0 el8.x86_64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel8.x86_64.rpm
| PGDG 1.3.0 el8.x86_64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel8.x86_64.rpm
| PGDG MISS |
| el8.aarch64 | PGDG MISS | PGDG 1.3.0 el8.aarch64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel8.aarch64.rpm
| PGDG 1.3.0 el8.aarch64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel8.aarch64.rpm
| PGDG 1.3.0 el8.aarch64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel8.aarch64.rpm
| PGDG MISS |
| el9.x86_64 | PGDG MISS | PGDG 1.3.0 el9.x86_64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel9.x86_64.rpm
| PGDG 1.3.0 el9.x86_64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel9.x86_64.rpm
| PGDG 1.3.0 el9.x86_64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel9.x86_64.rpm
| PGDG MISS |
| el9.aarch64 | PGDG MISS | PGDG 1.3.0 el9.aarch64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel9.aarch64.rpm
| PGDG 1.3.0 el9.aarch64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel9.aarch64.rpm
| PGDG 1.3.0 el9.aarch64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel9.aarch64.rpm
| PGDG MISS |
| el10.x86_64 | PGDG MISS | PGDG 1.3.0 el10.x86_64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel10.x86_64.rpm
| PGDG 1.3.0 el10.x86_64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel10.x86_64.rpm
| PGDG 1.3.0 el10.x86_64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel10.x86_64.rpm
| PGDG MISS |
| el10.aarch64 | PGDG MISS | PGDG 1.3.0 el10.aarch64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel10.aarch64.rpm
| PGDG 1.3.0 el10.aarch64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel10.aarch64.rpm
| PGDG 1.3.0 el10.aarch64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel10.aarch64.rpm
| PGDG MISS |
| d12.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 d12.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~bookworm_amd64.deb
| PIGSTY 1.3.0 d12.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~bookworm_amd64.deb
| PIGSTY 1.3.0 d12.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~bookworm_amd64.deb
| PIGSTY MISS |
| d12.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 d12.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~bookworm_arm64.deb
| PIGSTY 1.3.0 d12.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~bookworm_arm64.deb
| PIGSTY 1.3.0 d12.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~bookworm_arm64.deb
| PIGSTY MISS |
| d13.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 d13.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~trixie_amd64.deb
| PIGSTY 1.3.0 d13.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~trixie_amd64.deb
| PIGSTY 1.3.0 d13.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~trixie_amd64.deb
| PIGSTY MISS |
| d13.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 d13.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~trixie_arm64.deb
| PIGSTY 1.3.0 d13.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~trixie_arm64.deb
| PIGSTY 1.3.0 d13.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~trixie_arm64.deb
| PIGSTY MISS |
| u22.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 u22.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~jammy_amd64.deb
| PIGSTY 1.3.0 u22.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~jammy_amd64.deb
| PIGSTY 1.3.0 u22.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~jammy_amd64.deb
| PIGSTY MISS |
| u22.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 u22.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~jammy_arm64.deb
| PIGSTY 1.3.0 u22.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~jammy_arm64.deb
| PIGSTY 1.3.0 u22.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~jammy_arm64.deb
| PIGSTY MISS |
| u24.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 u24.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~noble_amd64.deb
| PIGSTY 1.3.0 u24.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~noble_amd64.deb
| PIGSTY 1.3.0 u24.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~noble_amd64.deb
| PIGSTY MISS |
| u24.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 u24.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~noble_arm64.deb
| PIGSTY 1.3.0 u24.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~noble_arm64.deb
| PIGSTY 1.3.0 u24.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~noble_arm64.deb
| PIGSTY MISS |
构建
您可以使用 pig build 命令构建 pgspider_ext 扩展的 DEB 包:
pig build pkg pgspider_ext # 构建 DEB 包
安装
您可以直接安装 pgspider_ext 扩展包的预置二进制包,首先确保 PGDG 仓库已经添加并启用:
pig repo add pgdg -u # 添加 PGDG 仓库并更新缓存
使用 pig 或者是 apt/yum/dnf 安装扩展:
pig install pgspider_ext; # 当前活跃 PG 版本安装
pig ext install -y pgspider_ext -v 17 # PG 17
pig ext install -y pgspider_ext -v 16 # PG 16
pig ext install -y pgspider_ext -v 15 # PG 15
dnf install -y pgspider_ext_17 # PG 17
dnf install -y pgspider_ext_16 # PG 16
dnf install -y pgspider_ext_15 # PG 15
apt install -y postgresql-17-pgspider-ext # PG 17
apt install -y postgresql-16-pgspider-ext # PG 16
apt install -y postgresql-15-pgspider-ext # PG 15
创建扩展:
CREATE EXTENSION pgspider_ext;
用法
PGSpider 扩展将 PostgreSQL 变为分布式查询引擎,通过创建虚拟分区表,透明地并行查询多个远程节点的数据。
设置子服务器
首先,使用各自的 FDW 为每个数据源创建服务器:
CREATE EXTENSION pgspider_ext;
CREATE EXTENSION postgres_fdw;
CREATE SERVER pgsrv1 FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host '127.0.0.1', port '5433', dbname 'postgres');
CREATE SERVER pgsrv2 FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host '127.0.0.1', port '5434', dbname 'postgres');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv1
OPTIONS (user 'user', password 'pass');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv2
OPTIONS (user 'user', password 'pass');
创建子外部表
CREATE FOREIGN TABLE t1_pg1_child (i int, t text)
SERVER pgsrv1 OPTIONS (table_name 't1');
CREATE FOREIGN TABLE t1_pg2_child (i int, t text)
SERVER pgsrv2 OPTIONS (table_name 't1');
创建 PGSpider 分区表
创建 PGSpider 服务器和带有额外分区键列的分区父表:
CREATE SERVER spdsrv FOREIGN DATA WRAPPER pgspider_ext;
CREATE USER MAPPING FOR CURRENT_USER SERVER spdsrv;
CREATE TABLE t1 (i int, t text, node text)
PARTITION BY LIST (node);
CREATE FOREIGN TABLE t1_pg1 PARTITION OF t1
FOR VALUES IN ('node1') SERVER spdsrv;
CREATE FOREIGN TABLE t1_pg2 PARTITION OF t1
FOR VALUES IN ('node2') SERVER spdsrv
OPTIONS (child_name 't1_pg2_child');
默认情况下,PGSpider 使用 [父表名]_child 模式查找子表。使用 child_name 指定不同的名称。
跨所有节点查询
SELECT * FROM t1;
i | t | node
----+---+-------
10 | a | node1
11 | b | node1
20 | c | node2
21 | d | node2
查询会自动并行分发到所有子节点。WHERE 子句和聚合函数会下推到子节点:
SET enable_partitionwise_aggregate TO on;
SELECT count(*), node FROM t1 GROUP BY node;
注意: 仅支持 SELECT 操作;不支持 INSERT、UPDATE 和 DELETE。