PostgreSQL Shared Buffers: A Production DBA Guide

A PostgreSQL server had just returned from planned operating-system maintenance. Recovery completed cleanly, connections were available, and the database log showed nothing unusual. Yet application latency was several times higher than before the restart, read IOPS had increased, and the first recommendation was to increase shared_buffers.

That may help, but it is not a diagnosis. The PostgreSQL cache is cold after a restart. A shared-buffer miss may be served by the operating-system page cache or may reach storage. A query can also have an excellent cache-hit ratio and still be slow because it processes millions of cached blocks. On the write side, dirty buffers bring WAL, the background writer, client backends, and the checkpointer into the same performance path.

This article explains what PostgreSQL keeps in shared buffers, how cache hits and misses should be interpreted, and what happens when pages are modified. It also covers practical sizing, monitoring SQL, common production failures, and the operational differences Oracle DBAs should expect when moving from the SGA buffer cache to PostgreSQL.

Why Buffer Sizing Becomes a Production Problem

shared_buffers defines a fixed-size shared memory area allocated when PostgreSQL starts. All databases and normal backends in the same PostgreSQL cluster use the same pool. It normally contains pages from permanent relations, including heap tables, indexes, materialized views, and system catalogs.

Most installations use an 8 KB database block size. A 4 GB setting therefore provides approximately 524,288 block-sized buffer slots. PostgreSQL caches pages, not complete query results. Finding a page in memory does not remove the CPU work required for MVCC visibility checks, filtering, joins, aggregation, row locking, or result transfer.

The setting is also not PostgreSQL's complete memory budget. Sorts and hash tables generally use work_mem. Temporary tables use session-local temp_buffers. WAL waiting to be written uses wal_buffers. The planner setting effective_cache_size allocates no memory at all; it only estimates the cache likely to be available.

What Really Happens During a Read

When an execution plan requests a table or index block, the buffer manager checks whether that block is already in shared buffers. On a cache hit, the backend pins the buffer and works with the cached page. On a miss, PostgreSQL must find a reusable slot and request the page through the I/O path.

A PostgreSQL read is not automatically a physical storage read. Linux may already have the file page in its page cache. PostgreSQL statistics generally cannot distinguish a page supplied by kernel memory from one retrieved from the storage device. That is why a low PostgreSQL hit ratio and low storage IOPS can exist at the same time.

1. The executor requests a heap or index block.
2. The buffer manager searches the shared buffer mapping.
3. On a hit, PostgreSQL uses the cached page.
4. On a miss, it selects an unused or replaceable buffer.
5. A dirty victim may need to be written before reuse.
6. The requested page is loaded and the executor continues processing.

How PostgreSQL Chooses a Victim Buffer

Once unused slots are exhausted, PostgreSQL uses a clock-sweep style replacement algorithm. Buffers have a usage count. Repeatedly used pages receive some protection, while pages with a low usage count become stronger replacement candidates. A pinned page cannot be replaced while a backend is actively using it.

Large scans use limited buffer-access strategies for several operations so that one scan does not automatically occupy the entire pool. Even with that protection, repeated reporting scans can displace useful OLTP pages or saturate storage. The useful signal is not simply a low hit ratio. It is a hot page being evicted and then read again shortly afterward.

Dirty Pages, WAL, and Data-File Writes

An INSERT, UPDATE, or DELETE normally changes a page in shared buffers. Once the in-memory page differs from its current data-file copy, the buffer is dirty. PostgreSQL does not need to write that page to the data file at transaction commit.

For a normal WAL-logged change, PostgreSQL generates the required WAL record, modifies the shared page, and marks it dirty. Commit normally waits for the required WAL to become durable. The data page can be written later because crash recovery can replay WAL if the instance fails before the page reaches disk.

Writer	Role in Production	DBA Signal
Background writer	Writes selected dirty buffers in advance to improve the supply of clean reusable buffers.	Cleaning volume and repeated stops at the configured limit.
Checkpointer	Writes and synchronizes pages required for a checkpoint recovery boundary.	Requested checkpoints, write time, sync time, and storage latency.
Client backend	Can write a dirty victim when it needs a reusable buffer and no clean one is ready.	Writes occurring directly in the application path.

A dirty page is not the same as an uncommitted page. One page can contain row versions created by several transactions. MVCC metadata determines visibility. WAL and the synchronization path protect durability.

Too Small and Too Large Both Create Problems

Condition	Typical Symptoms	What to Validate
shared_buffers too small	Repeated reads of hot relations, rising evictions, backend writes, and unstable peak latency.	Whether evicted pages are reused, storage latency, OS cache behavior, and available memory.
shared_buffers too large	Reduced OS cache, swap risk, memory pressure during concurrency peaks, and a larger dirty working set.	Full memory exposure, container limits, connection count, query memory, WAL, and checkpoints.

The PostgreSQL documentation suggests 25% of RAM as a reasonable starting point for a dedicated server with at least 1 GB of memory. It also notes that values above roughly 40% are unlikely to work better for most workloads because PostgreSQL relies on the operating-system cache. These are starting points, not universal targets.

Do not size from total database capacity. A 5 TB database may have a 20 GB active working set. A 200 GB reporting database may scan almost everything. The access pattern and reuse interval matter more than allocated size.

A Practical Production Scenario

A pattern I have seen during platform migrations is a production host inheriting a lower-environment configuration. In one representative case, a server with 64 GB of RAM was running with only 512 MB of shared buffers. Query plans were unchanged, but p95 latency rose sharply during the morning peak.

Problem: Hot order and customer indexes were repeatedly read. pg_stat_io showed frequent evictions, storage telemetry confirmed physical reads, and client-backend writes increased during peak concurrency. The server still had safe memory headroom and no swap activity.

Root cause: The PostgreSQL cache was sized for a test workload, while the reusable OLTP working set was several gigabytes. The OS cache absorbed some misses but could not protect the workload when concurrent reporting started.

Fix: We replayed a representative workload with 8 GB and 16 GB settings, reviewed peak memory exposure, and adjusted max_wal_size to spread checkpoint-related writes. The final setting was applied during a planned restart.

Result: Repeated reads and evictions fell, backend writes became uncommon during the peak, and latency stabilized. The lesson was not that 25% is always correct. The improvement came from proving that a reusable working set was being displaced while the host had safe memory headroom.

SQL and Commands for Buffer Analysis

Start with configuration, database-level cache activity, and cluster I/O. Capture differences over a representative interval because cumulative counters can hide a short production incident.

SHOW shared_buffers;
SHOW block_size;

SELECT name, setting, unit, context, pending_restart FROM pg_settings WHERE name IN ( 'shared_buffers', 'effective_cache_size', 'work_mem', 'max_wal_size', 'checkpoint_timeout'
) ORDER BY name;

SELECT datname,
blks_read,
blks_hit,
round(
100.0 * blks_hit /
nullif(blks_hit + blks_read, 0), 2
) AS shared_buffer_hit_pct FROM pg_stat_database WHERE datname IS NOT NULL ORDER BY shared_buffer_hit_pct;

SELECT backend_type, object, context,
reads, hits, evictions, writes, writebacks FROM pg_stat_io WHERE object = 'relation' ORDER BY evictions DESC NULLS LAST;

pg_stat_io is available from PostgreSQL 16. PostgreSQL 17 introduced pg_stat_checkpointer and moved checkpoint-related fields out of pg_stat_bgwriter. Version-aware monitoring matters. Also, pg_stat_bgwriter has never been the view for calculating the cache-hit ratio.

Query-Level and Cache Inspection

EXPLAIN (ANALYZE, BUFFERS, WAL, TIMING OFF)
SELECT order_id, customer_id, order_status
FROM orders
WHERE customer_id = 1001;

CREATE EXTENSION IF NOT EXISTS pg_buffercache;

SELECT count(*) AS total_buffers,
count(*) FILTER (WHERE relfilenode IS NOT NULL) AS used_buffers,
count(*) FILTER (WHERE isdirty) AS dirty_buffers,
round(avg(usagecount)::numeric, 2) AS avg_usage_count FROM pg_buffercache;

EXPLAIN (ANALYZE) executes the SQL. Do not run production DML casually to obtain buffer statistics. A plan reporting only shared hits can still be expensive if it touches hundreds of thousands of cached blocks. Logical I/O consumes CPU and can create contention.

Operating-System Correlation and Configuration Review

free -h
vmstat 1 10
iostat -xz 1 10

# Example values for review, not a universal recommendation

shared_buffers = '16GB'
effective_cache_size = '44GB'
max_wal_size = '32GB'
checkpoint_timeout = '15min'
checkpoint_completion_target = 0.9

Check swap, run queue, device utilization, throughput, and read/write latency. Increasing shared buffers requires a restart. A substantial increase should be reviewed with WAL volume, max_wal_size, checkpoint frequency, recovery-time expectations, and disk-space planning.

Oracle Buffer Cache vs PostgreSQL

Oracle DBAs will recognize the basic design. Oracle's database buffer cache is a shared SGA component containing data blocks, while DBWn writes dirty buffers. PostgreSQL shared buffers serve a comparable purpose, but the sizing and write paths are not identical.

Area	PostgreSQL	Oracle
Primary cache	shared_buffers, shared across databases in one cluster.	Database buffer cache inside the SGA.
Durability	WAL protects changes before related data pages become durable.	Redo provides the corresponding instance-recovery protection.
Dirty-page writer	Background writer, checkpointer, and sometimes client backends.	DBWn writes dirty buffers and supports checkpoint progress.
Replacement	Clock-sweep behavior using usage counts and buffer pins.	LRU-based mechanisms using touch counts and internal lists.
Sizing model	Fixed at server start and deliberately leaves room for the OS cache.	DB_CACHE_SIZE can operate within SGA memory management such as ASMM.

SELECT name, display_value
FROM v$parameter
WHERE name IN (
  'db_cache_size',
  'sga_target',
  'memory_target'
);

SELECT name, block_size, current_size, buffers FROM v$buffer_pool;

SELECT name, physical_reads, db_block_gets, consistent_gets FROM v$buffer_pool_statistics;

In both products, the hit ratio can mislead. An Oracle query performing excessive consistent gets and a PostgreSQL query performing excessive shared hits can both be slow without generating many physical reads. Logical I/O is not free.

Common Failure and Troubleshooting Patterns

High Reads Immediately After Restart

The cache is cold. Check whether read latency and IOPS fall as the workload warms the pool. Do not resize from the first few minutes unless warm-up itself breaks the service objective.

Excellent Hit Ratio but Slow SQL

Review EXPLAIN (ANALYZE, BUFFERS). Look for excessive block access, poor joins, missing predicates, repeated nested-loop reads, and rows removed by filters.

Evictions Rise While the Host Has Free Memory

This can support testing a larger pool, but first confirm that the evicted pages are needed again. A one-time sequential scan may not benefit from a permanent increase.

Backend Writes Increase During Traffic Peaks

Review buffer pressure, background-writer activity, checkpoint timing, and storage latency together. More aggressive background cleaning may reduce backend work but can increase steady write traffic.

Temporary Files Remain High After a Cache Increase

Shared buffers do not store sort and hash spill files. Review work_mem, plan shape, concurrency, and temporary-file logging instead of increasing the wrong memory area.

The Server Starts Swapping

Revisit the full memory budget immediately. Include connections, parallel workers, autovacuum, per-operation memory, extensions, agents, and container limits. A larger cache is not an improvement if active PostgreSQL processes are paged out.

Lessons From the Field

The first common mistake is treating 25% of RAM as a finished design. It is only a starting point. The useful cache size depends on reuse, concurrency, the OS cache, storage, and the memory needed outside the buffer pool.

The second mistake is tuning from cumulative ratios. Capture deltas during the incident window. Separate restart warm-up, reporting, autovacuum, backup activity, and business traffic. One percentage covering all of them is rarely actionable.

The third mistake is increasing memory before reviewing SQL. Cached block access still consumes CPU. A missing index or poor join remains expensive even if every required page is already in memory.

Finally, watch the write side. Backend writes, requested checkpoints, dirty-buffer volume, and storage sync latency can explain application spikes that a cache-hit ratio cannot. The real target is stable latency with predictable memory and I/O behavior at peak load.

Quick Takeaways

• Shared buffers cache relation pages and hold the normal working pages modified by DML.
• A shared-buffer miss is not proof of physical storage I/O.
• Dirty buffers can be written by the background writer, checkpointer, or a client backend.
• A high cache-hit ratio does not prove that a query is efficient.
• Size for reusable working data and the full memory budget, not total database size.
• Review WAL and checkpoint behavior when substantially increasing the pool.
• Correlate PostgreSQL statistics with operating-system and storage telemetry.

Frequently Asked Questions

Should shared_buffers always be 25% of RAM?

No. Use it as a starting point for a dedicated server, then validate working-set reuse, OS memory, concurrency, checkpoints, and storage performance. Managed platforms may calculate or restrict the value differently.

Why is the hit ratio low when storage latency is normal?

The operating-system page cache may be satisfying PostgreSQL reads. The workload may also contain expected sequential or bulk reads. Compare PostgreSQL counters with device I/O.

Will increasing shared_buffers eliminate disk reads?

No. Cold starts, one-time scans, changing access patterns, and datasets larger than memory still generate reads. Poor SQL can also touch more blocks than any sensible cache should retain.

Can the setting be changed without a restart?

No. It is a server-start parameter. Check pg_settings.pending_restart to confirm whether a configured change is waiting for restart.

Which view should be used for cache hits?

Use pg_stat_database for database-level hits, pg_statio_* for relations, EXPLAIN (BUFFERS) for a query, and pg_stat_io for cluster I/O by backend type and context.

Conclusion

Shared buffers sit at the center of PostgreSQL's normal table and index I/O path. They reduce repeated reads, hold pages changed by DML, and connect directly to WAL, checkpoints, background writing, and crash recovery. That makes shared_buffers one of the first parameters DBAs inspect, but it should not become the automatic answer to every performance problem.

An undersized pool can repeatedly evict useful pages and expose applications to storage latency. An oversized pool can starve the operating system, increase total memory risk, and complicate write behavior. A cache hit avoids a PostgreSQL file read, but it does not make query processing free. A cache miss may still be served from kernel memory. These details explain why a cache-hit percentage alone is weak tuning evidence.

Start with a sensible allocation, then measure the workload that matters. Capture short-interval changes in reads, hits, evictions, backend writes, checkpoints, and storage latency. Review which relations and SQL statements are responsible. Test changes under realistic concurrency, leave headroom for the OS and per-session memory, and compare the same metrics after restart and cache warm-up.

Review your current setting, prove whether hot pages are being displaced, and confirm the finding outside PostgreSQL before scheduling a restart. Measure, test, change, and monitor. Do not size the buffer cache from folklore.

Have you handled an incident caused by buffer churn, backend writes, or an oversized memory setting? Share the evidence you used and whether resizing the pool actually fixed the problem.

References

• PostgreSQL Documentation: Resource Consumption
• PostgreSQL Documentation: Cumulative Statistics
• PostgreSQL Documentation: pg_buffercache
• Oracle Database Documentation: Buffer Cache