RPC Performance: Forest vs Lotus
The following measurements were taken while a Forest node and a Lotus node handled a similar stream of real RPC provider traffic during the same window. Because both nodes served an almost identical request stream, the differences below are attributable to the nodes rather than to uneven load. The "Comparable load" section documents the checks used to support that assumption.
Environment
The measurements cover Lotus 1.36.1-rc1 and Forest 0.33.8.
RPC latency
Average wall time per HTTP request was approximately 5 ms for Forest and 40-80 ms for Lotus over the measurement window. The Forest trace stayed roughly constant, while the Lotus trace varied as the request mix changed.
Per-flow latency
At the median (P50) the two nodes were close: approximately 6-7 ms for Forest and 8-12 ms for Lotus. The difference was larger in the tail. At P95, Forest stayed near 25 ms while Lotus stayed around 90 ms and reached approximately 450-470 ms under load. At P99, Forest held around 150 ms while Lotus ranged from approximately 500 ms to 1.6 s.
Amortized latency
The amortized view accounts for batched requests. It shows the same pattern as the per-flow view: comparable values at P50 and a larger difference at P95, P99, and on average.
Comparable load
The request rates and batch sizes reaching each node were close, which supports comparing the two nodes directly. HTTP request count averaged 44.9 req/s for Forest and 45.2 req/s for Lotus; batch-aware counts were 25.4 vs 25.9 req/s; and average batch size was approximately 1.28 vs 1.29. Both nodes therefore processed a comparable amount of work.
Concurrency followed from the latency difference: Forest kept fewer requests in flight at any instant (mean 0.39, peak 5) than Lotus (mean 1.90, peak 16).
Resource usage
While serving this workload, Forest used approximately 10% CPU and Lotus used approximately 40-80%, trending upward over the session. Forest's CPU trace was also less variable.
Memory usage was approximately 10% for Forest and approximately 25-30% for Lotus.
The absolute memory figures are higher than they appear here because of OS page caches, but the relative proportions between the two nodes still hold.