iReplayer: In-Situ and Identical Record-and-Replay for Multithreaded Applications

In one paragraph

Reproducing a multithreaded failure is hard because thread interleavings, system-call results, and memory layouts differ from run to run. Existing record-and-replay systems replay in a separate process, and most cannot guarantee an identical re-execution. iReplayer replays in situ, inside the original process, and identically: the same synchronization order, the same system-call results, the same heap addresses. Because the replay happens in the live process, debugging tools such as watchpoints and GDB can be attached directly, and the replay can be repeated as many times as needed. Recording costs about 3% on average, low enough to leave on in production.

Key ideas

• In-situ replay: re-execute in the same process instead of a new one. Addresses stay identical, and the replay can feed directly into diagnosis or online repair.
• Identical replay without ad-hoc synchronization: a data structure records the order of synchronizations and system calls cheaply and makes divergence easy to detect during replay.
• Record only what cannot be reconstructed: a custom heap removes the need to log allocations, file reads and writes are treated as revocable, and data races are handled at replay time rather than logged on every memory access.
• Epochs bounded by irrevocable system calls, so only the last epoch has to be replayed.

Results

• About 3% average recording overhead on PARSEC and on Memcached, Apache httpd, SQLite, pbzip2, Aget, and Pfscan.
• Identical re-executions verified by comparing synchronization order, system-call order, and the final heap image.
• In the same study, CLAP ran about 2.4× slower and RR about 17× slower on the same workloads.
• Three tools built on top: automatic detectors for heap overflows and use-after-free, and an interactive debugger integrated with GDB.