High bandwidth high latency performance tuning
We recently upgraded our office Internet connection to 20Mbps fibre. Interestingly, for local traffic we were seeing excellent speeds, but international connections were disappointing (1 - 3 Mbps).
In NZ we have quite a high ping to the US (where we’re primarily connecting to), and this requires a high TCP window size to be able to have sufficient data in the pipe to take advantage of the full capacity.
Our office has both Mac OS X machines and Windows 7 machines. I’ve been unable to solve the issue on the Windows 7 machines, they are currently capped at 1 Mbps.
For Mac OS X machines it’s easy to change the appropriate parameters temporarily.
First your need to calculate the appropriate value for your TCP window. Take your network speed in Mbps (eg. 20Mbps) and your typical round-trip-time in seconds (eg. 0.3s) and multiply them then divide by 8 (eg. 20,000,000 * 0.3 / 8) to get a reasonable TCP buffer size. For my example this is 750,000 which is substantially more than the default 65,536.
Try this in the Terminal (substituting your TCP window size from the calculations above):
sudo sysctl -w net.inet.tcp.sendspace=750000 sudo sysctl -w net.inet.tcp.recvspace=750000
Test it to see if you can notice a difference. We could particularly see the difference in uploads. Then to permanently enable it, edit (or create) /etc/sysctl.conf (you’ll need to sudo this) and append:
net.inet.tcp.sendspace=750000
net.inet.tcp.recvspace=750000
Thanks to Layton Duncan for confirming this was possible on Mac OS X when I was losing faith on Windows 7.
I’m still looking for how to solve this issue on Windows 7. Suggestions welcome.
For reference reading:
For bandwidth testing I used iperf on Linux, Windows and Mac OS X (jperf).
