Skip to main content

Short-Circuit Read

Short-circuit read allows a client co-located with a GooseFS worker to read block data directly from the worker's local storage via memory-mapped I/O, bypassing the gRPC data path entirely. This is the fastest read path and is used automatically when available.

How It Works​

  1. The client opens a file and resolves the block → worker mapping.
  2. If the client and worker are on the same host, the client attempts a short-circuit open.
  3. On success, the data payload is read via mmap of the worker's block file — no protobuf encoding or network transfer for the file bytes, though setup may use an OpenLocalBlock control RPC.
  4. On any recoverable failure (permission denied, file not found, etc.), the client transparently falls back to the gRPC read path.

Enabling​

Short-circuit is disabled by default. Enable via configuration:

# Environment variable
export GOOSEFS_SHORT_CIRCUIT_ENABLED=true
# goosefs-site.properties
goosefs.user.short.circuit.enabled=true
from goosefs import Config

# The Config builder inherits the setting from env / properties.
# No explicit Python-side toggle is needed.
cfg = Config("127.0.0.1:9200")

Capability Authorization​

On clusters with capability enforcement enabled, short-circuit reads require a valid capability from the master. If the capability is missing or invalid, the client falls back to gRPC — the read still succeeds, just slower.

When Short-Circuit Engages​

ConditionShort-circuit?
Client and worker on same host, SC enabled✅ Yes
Client and worker on different hosts❌ No (gRPC)
SC disabled in config❌ No (gRPC)
Capability required but not available❌ No (gRPC fallback)
Worker local file missing or unreadable❌ No (gRPC fallback)

The fallback is automatic and transparent — the application sees identical data regardless of which path is used.

Verifying Short-Circuit​

Call goosefs.enable_tracing(level="debug") near script startup to install the tracing subscriber, then set RUST_LOG for the short-circuit target:

RUST_LOG=goosefs_sdk::block::short_circuit=debug python your_script.py

Look for short-circuit open succeeded (engaged) vs falling back to gRPC (not engaged).