Buffer overruns

The instrumented kernel is both the very core of the system and the controller of the event buffers.

When the kernel is busy, it logs more events. The buffers fill more quickly, and the kernel requests that the buffers be flushed more often. The data-capture program handles each flush request; the kernel switches to the next buffer and tries to continue logging events. In an extremely busy system, the data-capture program may not be able to flush the buffers as quickly as the kernel fills them.

Suppose the kernel fills buffer 1 in the ring for CPU 0. At this point, the _NTO_TRACE_WAITBUFFER command issued by the data-capture thread that is bound to CPU 0 returns. This is the mechanism by which the kernel requests the data-capture program to flush the buffer. The program takes ownership of buffer 1 and the kernel marks the buffer as busy/in use.

After filling a buffer, the kernel moves to the next buffer in the same ring, buffer 2, and tries to write events to it. But if this buffer is also being flushed, then the kernel simply drops all events logged for that CPU until that next buffer becomes free. There's no attempt to skip ahead to find a free buffer. When buffer 2 becomes free again, the kernel will resume writing events but the timestamps in the event buffer slots will show a discontinuity.

Because there is one buffer ring per CPU, the event logging and capturing for other CPUs is not at all affected by the buffer overrun for CPU 0.

For more information on buffer overruns, see the Tutorials chapter.