os.fsync

Q: What is the difference between flush() and os.fsync() in Python?

f.flush() moves data from the Python file object buffer to the operating system. os.fsync(fd) moves it from the OS cache to the disk. For a write that survives power loss you need both, in that order: f.flush(); os.fsync(f.fileno()). A with-block close flushes, but does not fsync.

Q: How do I truncate a file in Python?

os.truncate(path, size), os.ftruncate(fd, size), or f.truncate(size) on an open file object. A size larger than the file extends it with zero bytes. To empty a file, os.truncate(path, 0) or open(path, 'w').close().

Q: How do I lock a file in Python on Linux?

os.lockf(fd, os.F_LOCK, 0) waits for an exclusive lock, os.F_TLOCK fails at once with BlockingIOError (an OSError) if another process holds it, os.F_ULOCK releases it. The fd must be open for writing. fcntl.flock is the BSD-style alternative; both are advisory and Unix-only.

fsync makes a write survive a crash or power loss — but only what the OS already has: flush the Python buffer first. fsync, ftruncate and truncate work on Unix and Windows; fdatasync, posix_fallocate and posix_fadvise are Unix-only (not macOS); sync, lockf and the F_* / POSIX_FADV_* constants are Unix-only.

os functionPython 3.0+ (sync, truncate, posix_fallocate, posix_fadvise, lockf 3.3+; ftruncate / truncate on Windows 3.5+)

Common call

f.flush(); os.fsync(f.fileno())

Returns

None

Replaces

hoping the OS writes the cache out before a crash

Watch out

fsync does not flush the Python buffer — call f.flush() first

os.fsync(fdfd — An open file descriptor (f.fileno()). os.fsync also accepts any object with a fileno() method.type: int · required)

→ None

Parameters

Name	Type	Required	Description
fd	int	yes	An open file descriptor (f.fileno()). os.fsync also accepts any object with a fileno() method.
length	int	yes	ftruncate(fd, length) / truncate(path, length): the new size. Shorter cuts the file, longer pads it with zero bytes.
path	str \| bytes \| PathLike \| int	yes	truncate only: the file to resize (or an open fd).
offset, len	int	yes	posix_fallocate / posix_fadvise: the byte range the call applies to.
advice	int	yes	posix_fadvise: one of the POSIX_FADV_* constants.
cmd	int	yes	lockf: F_LOCK (wait for the lock), F_TLOCK (fail at once if locked), F_ULOCK (unlock) or F_TEST (check).

Return value

None — Returns once the OS reports the data is on the storage device.

Common patterns

Durable, atomic file replace

Write a temp file, flush, fsync, then os.replace it over the target: readers see the old or the new file, never half of one.

import os
def save(path, data: bytes):
    tmp = path + '.tmp'
    with open(tmp, 'wb') as f:
        f.write(data)
        f.flush()
        os.fsync(f.fileno())
    os.replace(tmp, path)

Exclusive lock on a file (Unix)

F_TLOCK fails immediately when another process holds the lock; F_LOCK would wait.

import os
fd = os.open('app.lock', os.O_RDWR | os.O_CREAT)
try:
    os.lockf(fd, os.F_TLOCK, 0)
except OSError:
    print('another instance is running')

Reserve disk space up front (Linux)

posix_fallocate makes sure the disk blocks for the whole range are allocated before you start writing.

import os
fd = os.open('big.dat', os.O_RDWR | os.O_CREAT)
try:
    os.posix_fallocate(fd, 0, 1024 ** 3)
finally:
    os.close(fd)

Tell the kernel you will read once, front to back (Linux)

Sequential read-ahead while streaming, then drop the pages from the cache.

import os
fd = os.open('huge.log', os.O_RDONLY)
try:
    os.posix_fadvise(fd, 0, 0, os.POSIX_FADV_SEQUENTIAL)
    while os.read(fd, 1 << 20):
        pass
    os.posix_fadvise(fd, 0, 0, os.POSIX_FADV_DONTNEED)
finally:
    os.close(fd)

Examples

1. flush, then fsync

import os with open('data.txt', 'w') as f: f.write('data') f.flush() os.fsync(f.fileno()) size = os.path.getsize('data.txt') size

Returns

2. truncate shrinks a file

import os from pathlib import Path Path('f.bin').write_bytes(b'hello world') os.truncate('f.bin', 5) Path('f.bin').read_bytes()

Returns

b'hello'

3. …and pads with zero bytes when growing

import os from pathlib import Path Path('f.bin').write_bytes(b'abc') os.truncate('f.bin', 6) Path('f.bin').read_bytes()

Returns

b'abc\x00\x00\x00'

4. ftruncate on an open descriptor

import os from pathlib import Path Path('f.bin').write_bytes(b'hello world') fd = os.open('f.bin', os.O_RDWR) try: os.ftruncate(fd, 4) size = os.fstat(fd).st_size finally: os.close(fd) size

Returns

5. truncate also takes an fd

import os from pathlib import Path Path('f.bin').write_bytes(b'abcdef') fd = os.open('f.bin', os.O_RDWR) try: os.truncate(fd, 0) finally: os.close(fd) (os.truncate in os.supports_fd, os.path.getsize('f.bin'))

Returns

(True, 0)

6. Missing file

import os try: os.truncate('ghost.txt', 0) except OSError as e: result = type(e).__name__ result

Returns

'FileNotFoundError'

Pitfalls

1. fsync without flush

Text written to a Python file object sits in its buffer. fsync only pushes what the OS already has, so nothing reaches the file. Flush first.

fsync only

import os
with open('a.txt', 'w') as f:
    f.write('data')
    os.fsync(f.fileno())
    size = os.path.getsize('a.txt')
size

flush + fsync

import os
with open('a.txt', 'w') as f:
    f.write('data')
    f.flush()
    os.fsync(f.fileno())
    size = os.path.getsize('a.txt')
size

2. ftruncate does not move the file position

After cutting a file to 0 bytes the fd still points at the old offset, so the next write leaves a gap of zero bytes. Seek back to 0.

truncate, write

import os
from pathlib import Path
fd = os.open('f.bin', os.O_RDWR | os.O_CREAT | getattr(os, 'O_BINARY', 0))
try:
    os.write(fd, b'hello')
    os.ftruncate(fd, 0)
    os.write(fd, b'hi')
finally:
    os.close(fd)
Path('f.bin').read_bytes()

b'\x00\x00\x00\x00\x00hi'

truncate, lseek, write

import os
from pathlib import Path
fd = os.open('f.bin', os.O_RDWR | os.O_CREAT | getattr(os, 'O_BINARY', 0))
try:
    os.write(fd, b'hello')
    os.ftruncate(fd, 0)
    os.lseek(fd, 0, os.SEEK_SET)
    os.write(fd, b'hi')
finally:
    os.close(fd)
Path('f.bin').read_bytes()

b'hi'

3. Using truncate to "cut to at most n bytes"

truncate sets the size exactly: a file shorter than n grows, padded with zeros. Take the smaller of the two sizes if you only want to shrink.

truncate(path, 8)

import os
from pathlib import Path
Path('s.txt').write_bytes(b'abc')
os.truncate('s.txt', 8)
os.path.getsize('s.txt')

min(size, 8)

import os
from pathlib import Path
Path('s.txt').write_bytes(b'abc')
os.truncate('s.txt', min(os.path.getsize('s.txt'), 8))
os.path.getsize('s.txt')

When to use

Use it

Data that must survive a crash: databases, journals, config files you replace
Resizing a file in place (ftruncate / truncate)
Large sequential reads or writes where kernel hints (posix_fadvise, posix_fallocate) help
Simple advisory locks between cooperating processes on Unix (lockf)

Reach for something else

Every small write → fsync is slow; batch writes and sync once
Resizing through a file object → f.truncate(size)
Cross-platform locking → a lock-file library or msvcrt.locking on Windows

Notes

CPython impl

Unix: fsync(2), fdatasync(2), sync(2), ftruncate(2), posix_fallocate(3), posix_fadvise(2), lockf(3). Windows: fsync calls the MS C runtime _commit() (docs.python.org)

Availability

fsync, ftruncate, truncate: Unix and Windows (ftruncate and truncate on Windows since 3.5). fdatasync, posix_fallocate, posix_fadvise: Unix, not macOS, not iOS. sync, lockf, F_LOCK / F_TLOCK / F_ULOCK / F_TEST, POSIX_FADV_*: Unix only

fdatasync

Like fsync but, per docs.python.org, does not force the update of metadata (such as timestamps)

Constants on Linux

F_ULOCK 0, F_LOCK 1, F_TLOCK 2, F_TEST 3; POSIX_FADV_NORMAL 0, RANDOM 1, SEQUENTIAL 2, WILLNEED 3, DONTNEED 4, NOREUSE 5 (values from Linux CPython 3.12 — use the names)

lockf

Advisory locks: a process that ignores lockf can still write to the file; only lockf callers are stopped. The fd must be open for writing (O_RDONLY gives OSError EBADF on Linux). The lock covers len bytes from the current position

FAQ

Does os.fsync work on Windows? ▾

Yes — os.fsync, os.ftruncate and os.truncate work on Unix and Windows (fsync calls _commit() there). fdatasync, posix_fallocate and posix_fadvise are Unix-only and not on macOS; sync and lockf are Unix-only. This page has no live demo; its examples only call the functions that work everywhere.

What is the difference between flush() and os.fsync() in Python? ▾

How do I truncate a file in Python? ▾

How do I lock a file in Python on Linux? ▾

os.fsync

Parameters

Return value

Common patterns

Examples

Pitfalls

When to use

Notes

Related

FAQ