Latency and Determinism, and Which Is More Important for Your Application?
November 13, 2020
In applications where data flows need to take place in nanoseconds, the delay between when an instruction is received and when it executes, known as latency, is very important. But latency is different from determinism, which is knowing exactly when an instruction will execute. It’s an important distinction to be aware of when building embedded systems.
If your system will provide information to a pilot who needs to make decisions in only tens of milliseconds, you certainly want to reduce any delays in the information chain. However, in instances where users don’t need to make real-time decisions, such as while reviewing camera footage from a UAV fly-by, then some latency from an embedded network may not be an issue.
CPU vs GPU vs FPGA: Which Wins When It Comes to Latency?
It’s also imperative to choose the right processor type, whether a CPU, a GPU, or an FPGA, for your embedded system because each has its own considerations when it comes to latency and determinism. Increasingly, military system designers are putting the massive floating-point throughput and internal pipelining architecture of GPUs to good use, yet FPGA latency is still lower because traditional GPUs must use bus interconnects (i.e. PCIe lanes) to move data off-chip and then communicate with a CPU.
For some radar and signals intelligence (SIGINT) applications, the latency performance of dedicated GPUs is surely acceptable, yet for others it’s not. Some modern CPUs have started to embed GPUs into the same silicon device to improve efficiencies, while some FPGAs are used to send data directly to CPUs or GPUs. In either case, latency is reduced compared to a dedicated, externally connected GPU device. Latency is decreasing for newer GPUs as their bus speeds increase, but the latency of an FPGA is still more deterministic.
With an FPGA, it is feasible to have latency around one microsecond, whereas CPU latency may be around 50 microseconds. This is because FPGAs are a specialized architecture that do not depend on generic operating systems and generic buses.
Can an RTOS Improve Latency?
A traditional OS may struggle with determinism, since activities such as a garbage collection routine that operates in the background to clean up SSDs might take CPU time. Something that simple can cause a temporal drop in responsiveness.
But what if you use a real-time operating system (RTOS) rather than a traditional OS like Linux or Windows? This may help with determinism, but it doesn’t necessarily improve latency. In other words, you may have a better idea of how fast it will execute, but it may not necessarily execute faster.
Continue reading about latency, determinism, and other factors to consider when building an application in our white paper, Choosing the Best Processor for the Job.