We've been sloppy handling the bi-directional nature of TCP throughout our entire chain handling VNC traffic. You should not just call close() on a socket but rather do shutdown(SHUT_WR) and wait for the other side to finish up. For our intermediate steps this behaviour also needs to be propagated properly. Similarily EPIPE indicates some kind of unclean socket closure that also needs to propagate correctly.
We've seen issues from stray EPIPE on bug 7552, but also bug 289 and bug 1560. These are likely (at least partly) caused by this poor handling.
We need to go through the components involved here:
OpenSSH already seems to have proper handling, but it should be verified.