The tunnel module in PulseAudio has handling to ensure we enforce a minimum latency in order to avoid underruns. It works well enough, but it has some room for improvement that could reduce the number of underruns experienced even further:
a) We don't look at the downstream sink at all when setting the limit. There's not much point trying to request a latency lower than what the downstream server will give us. Unfortunately the introspection API doesn't give us the proper information directly, so we may be forced to infer it.
b) We don't look at the currently configured latency when an underrun actually happens. If an underrun happened at with 10 ms configured latency, increasing the minimum from 2 to 4 ms is not going to help much.
There are also some small underruns happening without any server logging anything (noticed when solving bug 5902). Unsure what's going on there, but perhaps we can work around it by configuring a larger minimum latency when starting. Right now it is 0 ms and increasing it to just 10 ms seems to make the problem go away.
There is also:
c) Buffer latency should also probably be larger than the RTT (which is measured and known)