PostgreSQL 19 and DuckDB 2.0

Data Engineer Upgrade Guide

PostgreSQL 19 Beta 4 adds REPACK and sequence replication, and DuckDB 2.0 is tentatively due October 21. What Data Engineer teams should test now.

Published: Proudly published by: Jeff Wahl11 min read

What this post covers

01

What DuckDB 1.5 Set Up for 2.0: Features and stability commitments from the 1.5.0 announcement

02

DuckDB 2.0 Timeline and the Slipped Date: Release calendar status versus the 1.5.0 September plan

03

JIT Off by Default: Query Performance Implications: Why the default changed and what to benchmark

04

What PostgreSQL 19 Beta 4 Contains: Release status, GA timing, and headline features from the announcement

05

Logical Replication of Sequence Values: Effect on CDC pipelines and database migration cutovers

06

REPACK and Online Table Maintenance: What the REPACK command changes for bloat management

07

Postgres for OLTP, DuckDB for Local Analytics: Where each engine fits in modern pipeline designs

08

What This Means for Data Engineer Interviews: Replication, maintenance, and engine-choice questions in system design rounds

PostgreSQL 19 is close to done. PostgreSQL 19 Beta 4 shipped September 24, 2026, and the project's announcement says a release candidate should land in early October with general availability possibly following in October.[1] The release team has since set RC1 for October 15 and a GA target of October 29, conditional on how the RC period goes.[2] For Data Engineers, 3 changes matter: a new REPACK command for online bloat cleanup, logical replication of sequence values, and JIT compilation switched off by default.

DuckDB 2.0 is also close, and it's late. DuckDB's own September 2 post says the 2.0 branch was cut and projects the final release for the second half of October.[3] The 1.5.0 announcement in March had pointed at September.[4] A 2-month slip on a major version is normal; plan around it.

My take: neither release changes what good data engineering looks like. Both change the specific answers you give in a system design round, and both will bite anyone who upgrades without reading the notes. Here's what's confirmed, what's still soft, and what to test.

Prepare for the interview
01 / Open invite
02min.

Know the patterns before the interviewer asks them.

a system design query, the same shape a screen would give you.
The diff against expected. Where ties broke. What you missed.
sandbox
1source → bronze → silver → gold
2 ingest : CDC + Kafka
3 transform : dbt + Airflow
4 serve : Snowflake
5
Execute your solution0.4s avg.
PayPalInterview question
Solve a problem

PostgreSQL 19 Beta 4: What Shipped and What Got Pulled

The Beta 4 announcement is worth reading for what it removed. The project reverted SQL/PGQ property graph queries, online enabling and disabling of data checksums, temporal FOR PORTION OF updates and deletes, and ALTER TABLE ... MERGE PARTITIONS and SPLIT PARTITIONS.[1] The same release includes REPACK fixes for crashes and incorrect behavior with invalid indexes and materialized views.[1]

If you read a PostgreSQL 19 preview blog in the spring and walked away excited about graph queries in core Postgres, update your notes. A beta feature list is a wish list with a deadline. I've watched candidates confidently describe features in interviews that got cut before GA, and the interviewer happened to follow the mailing lists. That's an awkward 30 seconds.

Here's what's still in and matters for pipelines:

  • REPACK, which replaces VACUUM FULL and CLUSTER and adds a CONCURRENTLY mode.[5]
  • Sequence synchronization in logical replication.[6]
  • JIT off by default, listed as a migration change in the release notes.[7]

One caveat on all of it: the release notes were still marked draft as of the research for this piece. Final wording can shift between RC and GA.

REPACK: Online Bloat Cleanup in Core PostgreSQL 19

Every DE who has supported an OLTP database has a bloat story. Mine involves a status table updated a few million times a day, a VACUUM FULL someone ran during business hours, and a very long Slack thread. The lesson was the same one everyone learns: reclaiming space in Postgres used to mean taking a lock that stops the world, or installing an extension.

Per the PostgreSQL 19 docs, REPACK does the work of VACUUM FULL and CLUSTER in one command. Without CONCURRENTLY, it holds an ACCESS EXCLUSIVE lock for the whole run. With CONCURRENTLY, that lock is taken only to swap the table and index files, and changes made during the rebuild are captured through logical decoding and applied before the swap.[5]

The Restrictions You Need to Know

The docs list real limits on the concurrent mode:[5]

  • It is not MVCC-safe.
  • It doesn't apply to partitioned tables, unlogged tables, or tables without a primary key or replica identity index.
  • It uses replication slots, and max_repack_replication_slots defaults to 5.
  • The final lock can still be noticeable if many changes pile up while REPACK waits for it.

That last one is the trap. In my experience, the dangerous part of any "online" operation is the brief exclusive lock at the end. A queued ACCESS EXCLUSIVE request waits behind your longest-running transaction, and every query that arrives after it waits too. Set a lock timeout so the operation fails fast instead of taking production down with it.

-- Fail fast if the final swap lock can't be acquired
SET lock_timeout = '5s';
-- Rebuild the table online; check the PG 19 docs for exact option syntax
REPACK (CONCURRENTLY, VERBOSE) orders;

REPACK vs pg_repack

Fleets on PostgreSQL 18 and earlier still depend on the pg_repack extension. Its docs say it takes no exclusive lock during the work but takes one at the end, cancels conflicting queries after a 60-second default wait, needs a primary key or a unique index on a NOT NULL column, and must be run by a superuser.[8]

OptionAvailabilityLockingKey requirement
VACUUM FULLAll versionsExclusive for the full runNone
pg_repackExtensionExclusive at the endPK or unique NOT NULL index; superuser
REPACK CONCURRENTLYPostgreSQL 19+Exclusive only for the file swapPK or replica identity index; not partitioned or unlogged

The practical upside is operational. Getting an extension approved on a managed database or through a security review can take longer than the bloat problem itself. A core command removes that conversation.

Sequence Replication: The PostgreSQL 19 Feature That Changes Cutovers

If you've done a logical-replication migration, you know the moment. Data is synced, lag is zero, you promote the new primary, and the first insert fails with a duplicate key because the subscriber's sequence is sitting at 1. Before PostgreSQL 19, the fix was a manual setval script per sequence, run at exactly the right moment, or avoiding sequences entirely.

PostgreSQL 19 docs say sequence values can now be synchronized from publisher to subscriber during CREATE SUBSCRIPTION, ALTER SUBSCRIPTION ... REFRESH PUBLICATION, and ALTER SUBSCRIPTION ... REFRESH SEQUENCES.[6]

Read the next part twice. Per the same docs, ongoing sequence changes are not streamed. The subscriber keeps the last synced value until you refresh, so you must run REFRESH SEQUENCES before a switchover. The publisher must also run PostgreSQL 19 or later.[6]

Key takeaway: PostgreSQL 19 makes sequence sync a single command; it doesn't make it automatic. If REFRESH SEQUENCES isn't in your cutover runbook right before promotion, you still have the duplicate-key problem.
-- Run on the subscriber after writes on the publisher are stopped,
-- immediately before promoting the subscriber
ALTER SUBSCRIPTION migration_sub REFRESH SEQUENCES;

2 more things for pipeline people. First, sequence sync is separate from row-level change streams. CDC tools like Debezium emit events for inserts, updates, and deletes; a sequence advancing is not a row change, so don't expect your downstream consumers to see it. Second, the publication defaults still apply: the default publish set is insert, update, delete, truncate, so TRUNCATE replicates unless you exclude it. I've seen at least one popular interview-prep site get that backwards; the PostgreSQL docs are the authority.

The release notes also say logical replication can now be enabled without a server restart when wal_level is set to replica.[7] If you've ever had to schedule a maintenance window just to flip wal_level before a migration could start, you'll appreciate that one.

JIT Off by Default: Check Your Analytical Queries

PostgreSQL 19 changes the default of jit to off. The release notes say JIT was previously triggered by optimizer cost estimates, which were judged unreliable, and that sites running many large analytical queries must enable it manually.[7] The commit message reportedly adds that JIT can hurt very fast queries and that most large deployments already disable it.

This is a sane default. JIT's compilation overhead lands on short OLTP queries that happened to cross a cost threshold; the speedup lands on long CPU-bound scans. Most Postgres boxes are OLTP. The people who benefit from JIT are the ones running reports directly against Postgres, and that's exactly who will see a regression after upgrading.

If that's you, here's the test plan I'd run (my recommendation, not a published benchmark; I couldn't find a credible PostgreSQL 19 JIT benchmark):

  • Run your analytical query set on 18 and 19, with jit on and off, and compare total and per-query runtimes.
  • Check the JIT section of EXPLAIN ANALYZE to confirm whether compilation actually ran.
  • Measure OLTP latency too; that's the workload the change targets, and you may get a free win.
  • If analytics need JIT, set it per session or per role with SET jit = on rather than flipping it globally.

When a query slows down after an upgrade, the first stop is the release notes' migration section. Boring advice. Saves days.

DuckDB 2.0 Release Date: What's Confirmed

The DuckDB 2.0 timeline has more moving parts than it should, so here's each piece with its source.

  • March 9, 2026: DuckDB 1.5.0 "Variegata" shipped with a reworked CLI and built-in VARIANT and GEOMETRY types. The announcement names v1.4 LTS and v1.5 as the supported lines and says the next release, planned for September, will be v2.0.[4]
  • August 17, 2026: the v2.0 preview describes 2.0, codenamed "Cyanoptera," with a new SQL parser, a new default storage format, a reworked C API, and "a small number of carefully chosen breaking changes."[9]
  • September 2, 2026: DuckDB split off the 2.0 branch, ending new feature work, made alpha Python and CLI clients available, and projected the final release for the second half of October.[3]
  • Release calendar: one reading of the calendar lists 2.0.0 for October 21, 2026, marked tentative; another cached version shows only "Fall 2026."[10]

So: October 21 is plausible and consistent with DuckDB's own "second half of October," but the calendar itself says planned dates are tentative. As of October 14, I found no 2.0.0 release announcement. Treat October 21 as a target, not a ship date.

The Storage Format Change Is What Matters

A new default storage format is the part embedded-analytics teams should care about. DuckDB's FAQ documents that newer versions can always read files written by older versions, while older versions reading newer files is best-effort only.[11] Since v1.2.0, a STORAGE_VERSION option lets you write files that older releases can open.[11]

That asymmetry is the whole upgrade risk. If one service upgrades to 2.0 and writes files that a 1.5 consumer has to read, you've built yourself an incident. Pin the storage version until every reader is upgraded:

-- Write a database file that older DuckDB readers can still open
ATTACH 'shared_extract.duckdb' AS shared (STORAGE_VERSION 'v1.2.0');

You'll see claims online that existing .duckdb files won't open in 2.0. I couldn't trace that to any DuckDB source, and it contradicts the documented compatibility direction. Ignore it until DuckDB says otherwise.

Patch cadence is the other stability signal: 1.5.0 in March, 1.5.4 in June (which fixed a VARIANT cast that read the wrong rows under a filter), 1.5.5 in July.[4] If your production app embeds DuckDB, pin the version and stay on an LTS line until 2.0 has a patch release or 2 behind it. A wrong-rows bug in a new type is exactly why you let other people find the edge cases first.

Analysts Are Slowing the Store Down

> Analytics queries against the production database are slowing the live application. Move analytics onto its own warehouse fed from the database's change log, while a merchant dashboard shows new orders within fifteen minutes on a path of its own.

Source
Transform
Storage
Quality
Consumer
Queue
Bronze
Silver
Gold
Custom
Pipeline Architecture
Sketch the architecture

Add a node from the toolbar or right-click the canvas

Drag from a right port to a left port to wire data flow

Where Postgres and DuckDB Fit in a Data Engineering Stack

The pattern I keep seeing in real stacks: Postgres runs OLTP, CDC carries changes out, DuckDB handles local analytics on extracts and files, and a warehouse covers shared concurrent workloads. Neither of these releases changes that shape. They change the details at the seams.

DuckDB already talks to Postgres 2 ways, and the distinction comes up in interviews:

  • The postgres extension keeps DuckDB as the client. ATTACH ... (TYPE postgres) connects read-write by default, with a READ_ONLY option.
  • pg_duckdb puts DuckDB's engine inside the Postgres process, routing analytics queries to it.

DuckDB's own docs note that Postgres's client-server protocol is slow, which is why the team invested in optimizing the scanner. Either way, don't point heavy analytics at your production primary. Use a replica. (Most companies don't have big data; they have medium data and one reporting query that runs at 9am and locks up checkout.)

If you want the fundamentals behind that split, the OLAP vs OLTP breakdown covers why the 2 workloads fight over the same box.

What These Releases Mean for Data Engineer Interviews

Interviewers don't need you to memorize release notes. They do notice when you know what changed and can say how confident you are about it. Here's where each change shows up.

Maintenance rounds. "A busy table is bloated; reclaim space without downtime." Give the 3 paths from the table above, lead with version, then name the restrictions and the final-lock risk. Mentioning lock_timeout unprompted is the kind of detail that signals you've actually run this in production.

Migration and failover rounds. "Walk me through a logical replication cutover." Cover sequence state at promotion. Say that PostgreSQL 19 adds REFRESH SEQUENCES but doesn't stream sequence changes, and that before 19 you'd script setval. Expect a follow-up on UUIDs or range allocation for multi-writer designs. This is core Data Engineering system design material, and it's pipeline architecture rather than load balancer trivia.

Performance debugging. "Analytical queries got slower after an upgrade." Release notes first, then EXPLAIN ANALYZE. The JIT default change is a perfect example answer. Brush up on reading query plans alongside your SQL interview questions practice; the plan is where every regression story starts.

Engine choice. "When would you use DuckDB over a warehouse?" "DuckDB is single-writer" has been the standard answer. DuckDB's concurrency docs describe multi-process writes through a remote protocol that's maturing toward 2.0, while the LTS docs still say multi-process writing isn't supported. Date-stamp anything you say about DuckDB concurrency: "as of 1.5, per the docs."

The meta-skill across all 4: separate shipped from planned. Say "per DuckDB's August preview, 2.0 brings a new default storage format; details may change before release." That sentence tells the interviewer you read primary sources and don't overclaim. Concepts transfer; version trivia expires. Bloat, lock contention, replication state, and cost-based execution are the concepts, and these releases are just new examples of them.

Saying it cleanly under pressure takes practice. A mock interview is the best way I've found to find out whether your cutover explanation holds up when someone pushes back.

What to Do and Watch Next

  • October 15: PostgreSQL 19 RC1 is scheduled.[2] Stand up a test instance and run your analytical query set with JIT on and off.
  • October 29: the PostgreSQL 19 GA target, conditional on the RC.[2] Read the final release notes before you upgrade anything, especially the migration section.
  • Second half of October: DuckDB 2.0, with October 21 as the tentative calendar date.[3] Check the official changelog for the breaking changes before touching production.
  • Now: add REFRESH SEQUENCES to any cutover runbook targeting 19, inventory which services read shared DuckDB files, and pin storage versions where readers lag writers.

Neither upgrade is urgent. Postgres x.0 releases have earned a patch-release waiting period for most shops, and DuckDB's own track record says the same about new types and formats. Test now, upgrade later, and spend the time in between on the concepts these features exercise. The practice problems are a good place to keep the SQL side sharp while the dust settles.

The tools will change again in 18 months. Bloat, stale sequences, and surprise query regressions will still be there waiting for you.

References

  1. PostgreSQL Global Development Group, "PostgreSQL 19 Beta 4 Released!," September 24, 2026. postgresql.org
  2. PostgreSQL release team, message to pgsql-hackers on PostgreSQL 19 RC1 and GA dates, September 28, 2026. postgresql.org
  3. DuckDB Labs, "Try DuckDB v2.0-alpha," September 2, 2026. duckdb.org
  4. DuckDB, "Announcing DuckDB 1.5.0," March 9, 2026. duckdb.org
  5. PostgreSQL Global Development Group, "REPACK," PostgreSQL 19 documentation. postgresql.org
  6. PostgreSQL Global Development Group, "Replicating Sequences," PostgreSQL 19 documentation. postgresql.org
  7. PostgreSQL Global Development Group, "PostgreSQL 19 Release Notes" (draft). postgresql.org
  8. pg_repack project, "pg_repack documentation," version 1.3.4. pgxn.org
  9. DuckDB Labs, "A Preview of DuckDB v2.0," August 17, 2026. duckdb.org
  10. DuckDB, "Release Calendar." duckdb.org
  11. DuckDB, "Storage (Internals)," DuckDB documentation. duckdb.org

PostgreSQL 19DuckDB 2.0PostgreSQL 19 new featuresdata engineer PostgreSQLDuckDB release date
02 / Why practice

Try the actual problems

  1. 01

    Reading a solution is not the same as writing one

    Every engineer who has frozen on a query they had read a dozen times knows the gap. The only preparation that closes it is producing the answer yourself, under time, before the interview does it for you

  2. 02

    76% of hiring managers reject on the coding task, not the resume

    From HackerRank's 2024 Developer Skills Report. Candidates who look strong on paper still fail the live screen if they haven't done timed, executable practice

  3. 03

    System design comes down to the calls you defend out loud

    Ingestion, batch vs streaming, the bronze/silver/gold layers, idempotency, backfill and replay. Sketching the pipeline and naming the failure modes is the signal, not the boxes