v3.1: PG 17 as default, Better Supabase & MinIO, ARM & U24 support

With the release of PostgreSQL 17.2 earlier this week, Pigsty promptly followed up with its v3.1 release.

In this version, PostgreSQL 17 has been promoted as the default major version, and nearly 340 PostgreSQL extensions are now available out of the box.

Additionally, Pigsty 3.1 introduces the ability to self-hosting Supabase with a simple command, improves best practices for using MinIO the object storage.

It also supports the ARM64 arch, and the newly released Ubuntu 24.04 LTS. And provide a way to simplify configuration management across different OS distro and PG major versions.

Finally, pigsty also provides a range of ready-to-use scenario-based templates in this release.


Self-Hosting Supabase

The complete tutorial for self-hosting supabase

Supabase is an open-source Firebase alternative, a Backend as a Service (BaaS).

Supabase wraps PostgreSQL kernel and vector extensions, alone with authentication, realtime subscriptions, edge functions, object storage, and instant REST and GraphQL APIs from your postgres schema. It let you skip most backend work, requiring only database design and frontend skills to ship quickly.

Currently, Supabase may be the most popular open-source project in the PostgreSQL ecosystem, boasting over 74,000 stars on GitHub. And become quite popular among developers, and startups, since they have a generous free plan, just like cloudflare & neon.

Supabase’s slogan is: “Build in a weekend, Scale to millions”. It has great cost-effectiveness in small scales (4c8g) indeed. But there is no doubt that when you really grow to millions of users, some may choose to self-hosting their own Supabase —— for functionality, performance, cost, and other reasons.

That’s where Pigsty comes in. Pigsty provides a complete one-click self-hosting solution for Supabase. Self-hosted Supabase can enjoy full PostgreSQL monitoring, IaC, PITR, and high availability capability,

You can run the latest PostgreSQL 17(,16,15,14) kernels, (supabase is using the 15 currently), alone with 340 PostgreSQL extensions out-of-the-box. Run on mainstream Linus OS distros with production grade HA PostgreSQL, MinIO, Prometheus & Grafana Stack for observability, and Nginx for reverse proxy.

Since most of the supabase maintained extensions are not available in the official PGDG repo, we have compiled all the RPM/DEBs for these extensions and put them in the Pigsty repo: pg_graphql, pg_jsonschema, wrappers, index_advisor, pg_net, vault, pgjwt, supautils, pg_plan_filter,

Everything is under your control, you have the ability and freedom to scale PGSQL, MinIO, and Supabase itself. And take full advantage of the performance and cost advantages of modern hardware like Gen5 NVMe SSD.

All you need is prepare a VM with several commands and wait for 10 minutes….

 curl -fsSL https://repo.pigsty.io/get | bash
./bootstrap          # install deps (ansible)
./configure -c supa  # use supa config template (IMPORTANT: CHANGE PASSWORDS!)
./install.yml        # install pigsty, create ha postgres & minio clusters 
./supabase.yml       # launch stateless supabase containers with docker compose

asciicast


PostgreSQL 17

PostgreSQL 17 has some exciting new features, but the most notable improvement is its write performance. I tested it on a 128c physical machine and found that it performed well as expected under pigsty’s default OLTP configuration without any tuning.

perf.png

pgbench & sysbench results on a 128c BM with PG 17.2

For example, the max LSN Rate (WAL throughput) is around 110 MiB/s @ PostgreSQL 14, while it can reach 180 MiB/s @ PostgreSQL 17. Of course, this is the software bottleneck, not the hardware. We can see a significant improvement in write performance, which is crucial for many OLTP workloads.

The detailed performance comparison will be published in the upcoming blog post, so stay tuned!


340 Extensions

One of the standout features of Pigsty 3.1 is its support for 340 PostgreSQL extensions. This impressive number comes even after carefully pruning over a dozen “obsolete extensions”.

To achieve this milestone, I’ve built a YUM/APT repository that offers pre-packaged RPM/DEB files for the following combinations of operating systems and PostgreSQL versions:

  • Operating Systems: EL 8/9, Ubuntu 22.04/24.04, Debian 12
  • PostgreSQL Versions: PG 12–17
Code OS Distro x86_64 PG17 PG16 PG15 PG14 PG13 PG12
EL9 RHEL 9 / Rocky9 / Alma9 el9.x86_64
EL8 RHEL 8 / Rocky8 / Alma8 / Anolis8 el8.x86_64
U24 Ubuntu 24.04 (noble) u24.x86_64
U22 Ubuntu 22.04 (jammy) u22.x86_64
D12 Debian 12 (bookworm) d12.x86_64

Currently, the repository provides packages for the x86_64 architecture. ARM64 and other architectures are under development, and are currently only offered to advanced users upon request.

More importantly, I maintain an Extension Directory that meticulously documents metadata, compatibility across OS/DB versions, and usage guidelines for every extension, helping users quickly locate the extensions they need.

ext-repo.png

Using the Repository

Pigsty’s extension repository integrates seamlessly with native OS package managers and is openly shared. You’re not required to use Pigsty to access these extensions. Simply add the repository to your existing system or Dockerfile, then install extensions with standard commands like yum or apt install.

Notably, the popular open-source project postgresql-cluster has already adopted this repository. It integrates with the repository as part of its installation workflow to distribute extensions to its users.

postgresql-cluster.png

Rust Extensions

The PostgreSQL ecosystem is seeing a growing number of Rust-based extensions developed using the pgrx framework. Pigsty’s repository currently includes 23 Rust extensions.

If you know of a promising extension, feel free to recommend it! I’ll do my best to include them in the repository. If you’re a PostgreSQL extension developer, I’m happy to provide assistance with packaging and distribution to ensure a smooth final delivery process to all end users.


Ubuntu 24.04 Support

Ubuntu 24.04 “Noble” has been out for six months now, and some users have started using it in production. Pigsty 3.1 formally adds support for Ubuntu 24.04 as a primary platform.

That said, being a relatively new system, Ubuntu 24.04 still has a few gaps compared to 22.04. For instance:

  • Extensions like citus and topn are unavailable system-wide.
  • timescaledb_toolkit lacks pre-built binaries for Ubuntu 24.04 on x86_64.

Despite these minor exceptions, most extensions are fully compatible with Ubuntu 24.04, making it a strong candidate for Pigsty’s supported platforms.

Sunsetting Ubuntu 20.04

As part of this update, Ubuntu 20.04 “Focal” is being retired from Pigsty’s list of primary supported systems, even though its official EOL is in May 2025. This decision stems from challenges like outdated software (e.g., PostGIS) and dependency issues, which made supporting it increasingly impractical. I’m more than happy to see it phased out.

However, you can still technically use Pigsty on Ubuntu 20.04, and subscription customers will continue to receive support for this version.

Current Supported Platforms

Pigsty now supports the following major operating systems:

  • EL 8/9
  • Ubuntu 22.04 and Ubuntu 24.04
  • Debian 12

For these platforms, Pigsty provides the latest packages and the full set of PostgreSQL extensions.

Code OS Distro x86_64 PG17 PG16 PG15 PG14 PG13 PG12 Arm64 PG17 PG16 PG15 PG14 PG13 PG12
EL9 RHEL 9 / Rocky9 / Alma9 el9.x86_64 el9.arm64
EL8 RHEL 8 / Rocky8 / Alma8 / Anolis8 el8.x86_64 el8.arm64
U24 Ubuntu 24.04 (noble) u24.x86_64 u24.arm64
U22 Ubuntu 22.04 (jammy) u22.x86_64 u22.arm64
D12 Debian 12 (bookworm) d12.x86_64 d12.arm64
D11 Debian 11 (bullseye) d12.x86_64 d11.arm64
U20 Ubuntu 20.04 (focal) d12.x86_64 u20.arm64
EL7 RHEL7 / CentOS7 / UOS … d12.x86_64 el7.arm64

= Primary Support; = Optional; = EOL, Extended Commercial Support


ARM64 Support

The ARM architecture has been steadily gaining ground, particularly in the cloud computing sector, where ARM servers are capturing an increasing market share. Users have been requesting ARM support for Pigsty for over two years. In fact, Pigsty previously supported ARM when adapting for “domestic systems,” but version 3.1 marks the first time ARM64 support is included in the open-source release.

Currently, ARM64 support is in Beta. All the core features are functional, and everything works end-to-end, but the actual performance and reliability need more real-world testing and user feedback.

Supported Features

Most of Pigsty’s key functionalities have been adapted, including the Grafana/Prometheus stack, which now has ARM-compatible packages. The remaining gap lies in PostgreSQL extensions, particularly the 140 extensions maintained by Pigsty. These are still in progress for ARM64 support. However, if you rely on extensions already provided by PGDG(e.g., postgis, pgvector), you should encounter no issues.

Currently, the ARM version runs smoothly on EL9, Debian 12, and Ubuntu 22.04.

  • EL8: Missing some official PGDG packages.
  • Ubuntu 24.04: A few missing extensions.

Due to these limitations, ARM64 support is not yet recommended for these two platforms.

Future Plans

I plan to run ARM in a pilot phase over the next one or two minor releases. Once the extension ecosystem is complete, ARM64 support will be marked as GA. In the meantime, I welcome feedback from anyone testing Pigsty’s ARM version. Your input will help refine and stabilize the implementation.


Simplified Config

Another significant improvement in Pigsty v3.1 is the simplification of config management. Handling package variations across different OS distros and PG major versions has always been a headache.

The Problem

Different OS distributions often have subtle differences in package names and available software, requiring Pigsty to generate separate configuration files for each distribution. This approach quickly leads to combinatorial explosion. For instance, Pigsty offers config templates for over a dozen scenarios. If each template must support 5–7 operating systems, the total number of configurations becomes unmanageable.

The Solution: Indirection

As the adage in computer science goes: “Any problem can be solved by adding a layer of indirection.” Version 3.1 addresses this issue by introducing a new configuration file, package_map, which defines package aliases. For each OS distribution, a node_id/vars file translates these aliases into the specific package names required by the OS.

config.png

How It Works

Take the Supabase self-hosting template as an example. It uses dozens of PostgreSQL extensions. Users only need to specify the extension names; details like CPU architecture, OS version, PostgreSQL version, or package names are handled internally.

pg_extensions: # extensions to be installed on this cluster
- supabase   # essential extensions for supabase
- timescaledb postgis pg_graphql pg_jsonschema wrappers pg_search pg_analytics pg_parquet plv8 duckdb_fdw pg_cron pg_timetable pgqr
- supautils pg_plan_filter passwordcheck plpgsql_check pgaudit pgsodium pg_vault pgjwt pg_ecdsa pg_session_jwt index_advisor
- pgvector pgvectorscale pg_summarize pg_tiktoken pg_tle pg_stat_monitor hypopg pg_hint_plan pg_http pg_net pg_smtp_client pg_idkit

Here’s an example: If you want to install PostgreSQL 16 and its extensions, you no longer need to manually specify the real package name in repo_packages and pg_extension (you can still do that!) the download or installation lists with PG16-specific packages. Instead, you just modify a single parameter: pg_version: 16, all settled!

config.png


Infra Improvement

Beyond functional improvements, Pigsty continues to enhance its underlying infrastructure. For instance, in version 3.0, we introduced support for alternative PostgreSQL kernels such as Babelfish (MSSQL compatibility), IvorySQL (Oracle compatibility), and PolarDB (a domestic PG-compatible kernel).

Previously, users needed to install these kernels from external repositories online. With Pigsty v3.1, the official repository now directly includes mirrors of Babelfish/WiltonDB, IvorySQL, PolarDB, and similar “exotic” PostgreSQL kernels. This means installation is now much simpler—just use the pre-configured templates, and you can set up these alternative kernels with a single command, no extra configuration required.

Additionally, Pigsty maintains Prometheus and Grafana repositories for both YUM and APT package managers, supporting AMD and ARM architectures. These repositories are updated in real-time to track the latest versions of these observability tools. In this release:

  • Prometheus was upgraded to its major v3 version.
  • VictoriaLogs officially released its v1.0 version.

If you rely on these monitoring tools, Pigsty’s repositories can be a valuable resource.


Improvements to MinIO

Let’s talk about MinIO, an open-source object storage solution. Pigsty leverages MinIO for PostgreSQL backups and as the underlying storage service for Supabase. Our goal with MinIO has always been to reduce deployment complexity to the bare minimum —— “Deploy in minutes, Scale to millions.”

When we first adopted MinIO internally, it was still in its 0.x days. At the time, we used it to store 25 PB of data, but since MinIO didn’t support online scaling, we had to break this into 7–8 independent clusters. Today, MinIO has come a long way. While it still doesn’t support directly modifying disk/node counts online, it now offers storage pool expansion. This allows smooth scaling by adding new storage pools, migrating data, and retiring old ones without downtime.

minio.png

For v3.1, I re-read MinIO’s latest documentation and updated the best-practice templates and SOPs to reflect its new features. These include:

  • Single node, single disk
  • Single node, multi-disk
  • Multi-node, multi-disk
  • Multi-storage-pool deployments
  • Handling disk failures and node failures
  • Expand/Shrink cluster
  • Using VIP and HAProxy for high-availability access

All procedures are documented and can be executed with just a few commands.

The Case for MinIO

Object storage is foundational for modern cloud infrastructure. MinIO, as a leading open-source object storage solution, delivers excellent performance and features. Most importantly, it is cloud-neutral, giving you independence from cloud vendor lock-in.

MinIO is also a compelling alternative to cloud object storage. Consider the scenario outlined in “DHH: Our cloud-exit savings will now top ten million over five years”:

  • Their cloud infrastructure included 10 PB of object storage.
  • The standard cost: $3M/year, reduced to $1.3M/year with saving plans
  • By contrast, a dedicated storage server with 1.2 PB of capacity costs just $20,000.
  • With OSS MinIO, 3-copies, DC, power, operations, the 5year TCO is still less than the one-year S3 cost.

If your business heavily relies on object storage, a self-hosted MinIO deployment combined with Cloudflare for external access could unlock significant cost savings and efficiency improvements. For many use cases, it’s an optimization worth serious consideration.


v3.1.0

Features

  • PostgreSQL 17 as the default major version (17.2)
  • Ubuntu 24.04 noble support
  • ARM64 support (el9, debian12, ubuntu 22.04)
  • New playbook supabase.yml for quick self-hosting supabase
  • MinIO Enhancement, best, practice, conf template, dashboards,…
  • Allow using -v to specify PG major version during configure
  • A series of out-of-the-box configuration templates and documentation.
  • Now install the pgvector extension by default
  • Simplify the repo packages configuration with package map alias
  • Setup WiltonDB, IvorySQL, PolarDB repo mirror
  • Enable postgres checksum by default

Software Upgrades

  • PostgreSQL 17.2, 16.6, 15.10, 14.15, 13.18, 12.22
  • PostgreSQL Extension Upgrades: https://ext.pigsty.io
  • Patroni 4.0.4
  • MinIO 20241107 / MCLI 20241117
  • Rclone 1.68.2
  • Prometheus: 2.54.0 -> 3.0.0
  • VictoriaMetrics 1.102.1 -> 1.106.1
  • VictoriaLogs v0.28.0 -> 1.0.0
  • vslogcli 1.0.0
  • MySQL Exporter 0.15.1 -> 0.16.0
  • Redis Exporter 1.62.0 -> 1.66.0
  • MongoDB Exporter 0.41.2 -> 0.42.0
  • Keepalived Exporter 1.3.3 -> 1.4.0
  • DuckDB 1.1.2 -> 1.1.3
  • etcd 3.5.16 -> 3.5.17
  • tigerbeetle 16.8 -> 0.16.13

API Change

  • repo_upstream: Now has defaults per distro: roles/node_id/vars.
  • repo_packages: Now support using package_map alias.
  • repo_extra_packages: Now support missing default values, and using package_map alias.
  • pg_checksum: Now the default value is true.
  • pg_packages: Change to postgresql, wal2json pg_repack pgvector, patroni pgbouncer pgbackrest pg_exporter pgbadger vip-manager.
  • pg_extensions: Change to empty array [].
  • infra_portal: Now allow using path in the home server.

News: Pigsty v3.1 Released

Pigsty v3.1 is now live, closely following the release of PostgreSQL 17.2. This version marks PostgreSQL 17 as the default, featuring compatibility with the latest Ubuntu 24.04 LTS and initial ARM64 arch support. It also introduces nearly 340 ready-to-use PostgreSQL extensions right from the start.

A notable feature is the one-click setup for a self-hosted Supabase, which is built upon PostgreSQL. Pigsty v3.1 enables you to run Supabase on local-first HA PostgreSQL (ver14-17) alone with 300+ extensions on mainstream Linux distros without virtualization or containers, utilizing modern hardware to its full potential.

The update enhances its repository with new PostgreSQL-compatible kernels like Babelfish, IvorySQL, and PolarDB, and the OLAP / DuckDB race players such as pg_analytics, pg_duckdb, pg_mooncake, pg_parquet, and duckdb_fdw, now directly accessible for the pigsty repo.

Additionally, MinIO best practices have been refined to lower deployment barriers and allow expanding existing clusters, which are detailed in the new admin SOP.

Pigsty 3.1 simplifies configuration across various OS distros by standardizing scenario-based config templates. You can download and install kernels and extensions by specifying their name. and changing the significant versions with just one pg_version parameter.

With comprehensive support for new operating systems and architectural improvements, Pigsty 3.1 aims to deliver a more feature-rich and cost-effective OSS RDS PG solution. I hope this could help you easily enjoy the latest PostgreSQL features and extensions.

Last modified 2024-11-30: routine update (a3b78ef1)