os.pread

Q: What is the difference between os.pread and os.read?

os.read(fd, n) reads from the current position and advances it. os.pread(fd, n, offset) reads from offset and leaves the position where it was — so threads sharing one fd do not move each other's position.

Q: Does shutil.copyfile use sendfile or copy_file_range?

In Python 3.12 and 3.13, shutil.copyfile uses os.sendfile on Linux (falling back to a normal read/write loop if it fails) and the macOS fcopyfile call on macOS; Windows uses a readinto loop. os.copy_file_range is not used by shutil in these versions — call it yourself if you want reflinks or server-side copies.

Q: What is os.sendfile used for?

Sending a file to a socket (or on Linux, another fd) without copying the bytes into Python. It returns the number of bytes sent, 0 at end of file, so loop. socket.socket.sendfile is the higher-level wrapper and works on every platform.

None of these exist on Windows. pread, pwrite, readv, writev: Unix. sendfile: Unix (not WASI). preadv, pwritev: Linux 2.6.30+, FreeBSD, OpenBSD, AIX. splice: Linux 2.6.17+; copy_file_range: Linux 4.5+; the RWF_* and SPLICE_F_* flags: Linux. Every call returns a byte count that can be smaller than you asked for.

os functionPython 3.3+ (preadv / pwritev 3.7+, copy_file_range 3.8+, splice 3.10+)

Common call

os.pread(fd, 4096, offset)

Returns

bytes (pread) or a byte count (the rest)

Replaces

lseek + read, which moves the shared file position

Watch out

Unix only — and counts can be short: loop

os.pread(fdfd — An open file descriptor.type: int · required, nn — pread: maximum number of bytes to read.type: int · required, offsetoffset — Byte position to read or write at; the fd position is left alone.type: int · required, /)

→ bytes

Parameters

Name	Type	Required	Description
fd	int	yes	An open file descriptor.
n	int	yes	pread: maximum number of bytes to read.
offset	int	yes	Byte position to read or write at; the fd position is left alone.
buffers	sequence	yes	readv / preadv: mutable buffers (bytearray, memoryview) filled in order. writev / pwritev: bytes-like objects written in order.
flags	int	no (0)	preadv: RWF_HIPRI, RWF_NOWAIT. pwritev: RWF_DSYNC, RWF_SYNC, RWF_APPEND (Linux 4.6+ for flags).
out_fd, in_fd, offset, count	int	yes	sendfile: copy count bytes from in_fd (starting at offset, or the current position when offset is None on Linux) to out_fd.
src, dst, count, offset_src=None, offset_dst=None	int	yes	splice / copy_file_range: copy count bytes between two fds inside the kernel. splice needs a pipe on one side.

Return value

bytes — At most n bytes read from offset; b"" at end of file. The fd position is unchanged.

Common patterns

Threads reading one file at different offsets

pread does not touch the shared position, so threads can share one fd without a lock.

import os
from concurrent.futures import ThreadPoolExecutor
fd = os.open('data.bin', os.O_RDONLY)
try:
    with ThreadPoolExecutor() as pool:
        blocks = list(pool.map(lambda off: os.pread(fd, 4096, off), range(0, 65536, 4096)))
finally:
    os.close(fd)

Header + body in one system call

writev gathers several buffers without concatenating them in Python first.

import os
header = len(body).to_bytes(4, 'big')
os.writev(fd, [header, body])

Copy a file inside the kernel (Linux)

copy_file_range can be short; loop until it returns 0.

import os
src = os.open('in.bin', os.O_RDONLY)
dst = os.open('out.bin', os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
try:
    while os.copy_file_range(src, dst, 1 << 30):
        pass
finally:
    os.close(src)
    os.close(dst)

Serve a file over a socket

socket.sendfile wraps os.sendfile and falls back to send() where it is missing.

with open('index.html', 'rb') as f:
    conn.sendfile(f)

Examples

1. What pread does, spelled out with lseek

import os from pathlib import Path Path('f.bin').write_bytes(b'hello world') def pread_like(fd, n, offset): saved = os.lseek(fd, 0, os.SEEK_CUR) os.lseek(fd, offset, os.SEEK_SET) try: return os.read(fd, n) finally: os.lseek(fd, saved, os.SEEK_SET) fd = os.open('f.bin', os.O_RDONLY | getattr(os, 'O_BINARY', 0)) try: os.lseek(fd, 2, os.SEEK_SET) result = (pread_like(fd, 5, 6), os.lseek(fd, 0, os.SEEK_CUR)) finally: os.close(fd) result

Returns

(b'world', 2)

2. How readv fills its buffers (pure Python model)

def readv_like(data, buffers): pos = 0 for buf in buffers: chunk = data[pos:pos + len(buf)] buf[:len(chunk)] = chunk pos += len(chunk) return pos a, b = bytearray(5), bytearray(3) (readv_like(b'hello world', [a, b]), a, b)

Returns

(8, bytearray(b'hello'), bytearray(b' wo'))

3. writev writes the buffers back to back

import os from pathlib import Path buffers = [b'ab', b'cd'] fd = os.open('out.bin', os.O_WRONLY | os.O_CREAT | getattr(os, 'O_BINARY', 0)) try: n = os.write(fd, b''.join(buffers)) finally: os.close(fd) (n, Path('out.bin').read_bytes())

Returns

(4, b'abcd')

4. shutil.copyfile: the portable fast copy

import shutil from pathlib import Path Path('src.bin').write_bytes(b'payload' * 1000) shutil.copyfile('src.bin', 'dst.bin') Path('dst.bin').read_bytes() == Path('src.bin').read_bytes()

Returns

True

5. socket.sendfile exists everywhere

import socket hasattr(socket.socket, 'sendfile')

Returns

True

6. The cross-device error splice and copy_file_range can raise

import errno errno.errorcode[errno.EXDEV]

Returns

'EXDEV'

Pitfalls

1. Seek-and-read moves the shared position

lseek + read changes where the next read starts — for every thread using that fd. pread (or saving and restoring the position) leaves it alone.

lseek + read

import os
from pathlib import Path
Path('f.bin').write_bytes(b'hello world')
fd = os.open('f.bin', os.O_RDONLY | getattr(os, 'O_BINARY', 0))
try:
    os.lseek(fd, 6, os.SEEK_SET)
    os.read(fd, 5)
    nxt = os.read(fd, 5)
finally:
    os.close(fd)
nxt

b''

pread (or restore)

import os
from pathlib import Path
Path('f.bin').write_bytes(b'hello world')
def pread(fd, n, offset):
    if hasattr(os, 'pread'):
        return os.pread(fd, n, offset)
    saved = os.lseek(fd, 0, os.SEEK_CUR)
    os.lseek(fd, offset, os.SEEK_SET)
    try:
        return os.read(fd, n)
    finally:
        os.lseek(fd, saved, os.SEEK_SET)
fd = os.open('f.bin', os.O_RDONLY | getattr(os, 'O_BINARY', 0))
try:
    pread(fd, 5, 6)
    nxt = os.read(fd, 5)
finally:
    os.close(fd)
nxt

b'hello'

2. Ignoring the byte count

os.write, writev, pwrite, sendfile, splice and copy_file_range all return how much they actually moved, which can be less than you passed (here: a 1 MiB write into a non-blocking pipe). Loop on the remainder.

one call

import os
r, w = os.pipe()
os.set_blocking(w, False)
data = b'x' * (1024 * 1024)
try:
    n = os.write(w, data)
finally:
    os.close(r)
    os.close(w)
n == len(data)

False

loop on the rest

import os
r, w = os.pipe()
os.set_blocking(r, False)
os.set_blocking(w, False)
data = b'x' * (1024 * 1024)
view = memoryview(data)
got = bytearray()
try:
    while view or len(got) < len(data):
        if view:
            try:
                view = view[os.write(w, view):]
            except BlockingIOError:
                pass
        try:
            got += os.read(r, 1 << 16)
        except BlockingIOError:
            pass
finally:
    os.close(r)
    os.close(w)
len(got) == len(data)

True

When to use

Use it

Several threads reading or writing one fd at different offsets (pread / pwrite)
Record formats where a header and a body live in separate buffers (writev / readv)
Copying large files or serving files to sockets without passing the bytes through Python (sendfile, copy_file_range, splice)

Reach for something else

Code that must run on Windows → lseek + read with a lock, or file objects
Plain file copies → shutil.copyfile / shutil.copy2 already pick the fast path
Sockets → socket.socket.sendfile instead of os.sendfile

Notes

CPython impl

Wrappers over the C calls in Modules/posixmodule.c. Lib/shutil.py (3.12 and 3.13) uses os.sendfile for shutil.copyfile on Linux and fcopyfile on macOS; Windows copies with readinto

Availability

pread, pwrite, readv, writev: Unix (3.3+). preadv, pwritev: Linux 2.6.30+, FreeBSD 6.0+, OpenBSD 2.7+, AIX 7.1+ (3.7+); flags need Linux 4.6+. sendfile: Unix, not WASI (3.3+). splice: Linux 2.6.17+ with glibc 2.5+ (3.10+). copy_file_range: Linux 4.5+ with glibc 2.27+ (3.8+). None on Windows

RWF_* flags

RWF_HIPRI (Linux 4.6+, O_DIRECT fds) and RWF_NOWAIT (Linux 4.14+) for preadv; RWF_DSYNC / RWF_SYNC (Linux 4.7+) and RWF_APPEND (Linux 4.16+, 3.10+) for pwritev: per-call versions of O_DSYNC, O_SYNC and O_APPEND

SPLICE_F_*

SPLICE_F_MOVE, SPLICE_F_NONBLOCK and SPLICE_F_MORE (3.10+) are the flags argument of splice; values on Linux 1, 2, 4

Short counts

readv / preadv return the total bytes read, which can be less than the buffers hold; sendfile and splice return 0 at end of input

EXDEV

splice needs both files on the same file system; copy_file_range too on Linux kernels older than 5.3. Otherwise OSError with errno.EXDEV

FAQ

Does os.pread work on Windows? ▾

No. pread, pwrite, readv, writev and sendfile are Unix-only, preadv, pwritev, splice and copy_file_range are Linux (and some BSDs), and none of them exist in the Windows os module. That is why this page has no live demo and its examples show portable equivalents. On Windows use lseek + read under a lock, or file objects.

What is the difference between os.pread and os.read? ▾

Does shutil.copyfile use sendfile or copy_file_range? ▾

What is os.sendfile used for? ▾

os.pread

Parameters

Return value

Common patterns

Examples

Pitfalls

When to use

Notes

Related

FAQ