LfD Logo

Comparative Analysis of Tracing Mechanisms in Linux for Real-Time Systems

Tracing mechanisms in real-time Linux systems enable detailed observation of application and kernel behavior while minimizing interference, aiming to capture system behavior as accurately as possible under production conditions with minimal observer effect. This thesis presents a comparative analysis of three widely used tracing frameworks, ftrace, LTTng, and eBPF, evaluating their usability, instrumentation workflow, features, performance impact, and suitability for tracing user-space applications. The study systematically measures the overhead introduced by each framework using a microbenchmark focused on User Statically-Defined Tracing (USDT). Additionally, the instrumentation workflow and feature set of each framework are examined to assess their practicality and integration usability. The impact on real-time performance is evaluated empirically using Cyclictest, the de facto standard benchmark for measuring real-time performance. The results indicate that LTTng has the least impact on real-time performance for user-space instrumentation due to its static instrumentation approach, while ftrace achieves the best stability and lowest overhead for kernel-space tracing. In contrast, eBPF introduces the highest performance impact in both kernel-space and user-space tracing. The study further highlights differences in usability, with ftrace offering the simplest setup, LTTng providing the most structured workflow for instrumenting user-space applications, and eBPF enabling advanced programming and scripting capabilities. The findings support developers and system designers in selecting a tracing framework that aligns with system constraints, enabling effective debugging and performance analysis while preserving real-time properties and ensuring accurate insights.