Skip to main content

Memory Maps and Large Files

rg can use memory-mapped I/O (mmap) to search large files. Instead of reading a file byte-by-byte, mmap maps the file directly into the process's virtual memory space and lets the OS kernel manage paging.

When mmap Helps​

mmap shines when:

  • The file fits in RAM (the OS caches the pages after the first access)
  • You search the same large file repeatedly
  • The file is on a local SSD
# Force mmap mode
rg --mmap "error" /var/log/large-app.log

When to Disable mmap​

mmap can harm performance when:

  • Files are on a network filesystem (NFS, CIFS) — page faults cross the network
  • Files are on a slow spinning HDD — random page faults beat sequential reads
# Disable mmap (sequential read)
rg --no-mmap "error" /mnt/nfs/logs/app.log

Large File Defaults​

By default rg skips files larger than a configurable threshold. Check the current limit:

rg --max-filesize 2G "pattern" /data/   # search files up to 2GB

Practical Large-Log Workflow​

# 1. Stream only new lines in real-time (rg can't tail -f, use grep)
tail -f /var/log/app.log | grep --line-buffered "ERROR"

# 2. But for historical batch searching, rg is faster:
rg --mmap --threads 1 "ERROR" /var/log/app.log.1

# 3. For rotated gzip logs:
zcat /var/log/app.log.*.gz | rg "ERROR"

.ripgreprc Config File​

You can persist performance tuning in a config file:

# ~/.config/ripgrep/ripgreprc

# Always use up to 4 threads (safe for production)
--threads=4

# Enable mmap by default
--mmap

# Set file size limit to 500MB
--max-filesize=500M

Activate it by exporting the path:

export RIPGREP_CONFIG_PATH="$HOME/.config/ripgrep/ripgreprc"