How bufferbloat kills your VOIP and other low-latency communications
So, if you use a VOIP phone, or Skype, or play real-time internet games, or do lots of video chats you may notice that sometimes your connection becomes "SLOOOW" in particular that there is a large lag between when you do something and when that thing registers on the internet. Symptoms of this include both sides of a conversation talking over each other for example.
One way to deal with this is QOS traffic shaping. That is, to define different queues for different packets. I've covered that before on this blog. But one thing that is typically recommended is that you QOS traffic shape only on the up-link, that is packets you're sending to the internet. Packets you receive from the internet, typically you'd just send them to their destination machine on your network as fast as possible.
Well, it turns out that this is bad advice. Yes, you should traffic shape your upstream packets, but, there is good reason to actually traffic shape your download packets as well. The reason is, Bufferbloat.
Somewhere on the other end of your cable service (or DSL or WISP or whatever) is a router that is sending your router packets. That router may have a large buffer in it. If something like a streaming video player on your network requests a big block of video data, the connection between Amazon or whatever and the router on the other end of your last-mile cable could be a lot faster than the cable connection you have. When you request all that video data, the remote router will get an initial flood of data and slam it all into a buffer to be sent to you. Once that buffer fills up, it will stop ACKing packets and Amazon will eventually slow down. The only problem? If you are expecting some voice packets to come in, they will have to wait in line at the remote router until all that video gets sent to you, and if that buffer is big, they will have to wait in line for maybe 1 to 3 SECONDS. That means several seconds of DEAD AIR on your VOIP phone (or several seconds of NO RESPONSE in your game).
The solution is to not let that remote router buffer up stuff. To do that, you have to ensure that your router drops packets on the floor quickly so that Amazon throttles its bandwidth back so that the remote buffer doesn't fill up. The main way you have to achieve this is to use QOS and throttle your download speed slightly below what your ISP offers you. Here's how:
Go to the dslreports speed test and run the test with your QOS turned off, and no-one using your internet connection. Run it two or three times and see how fast your uplink and downlink speeds are. Pick the smallest value you see out of 3 tries for upload and download and multiply by 0.95. These two numbers will be your uplink and downlink speed you will put into your QOS script.
Did the dslreports speed test show buffer bloat of several hundred to a thousand milliseconds? Mine did on download, and it was destroying my VOIP performance. I have about a 60 Mbps download connection, but if I had someone pulling video I could have 1-3 second cut-outs in my audio thanks to buffer bloat. Putting 58000 kbit/s in my download speed of my qos script dropped buffer bloat measurements from 1000+ ms to 40ms. Now, when my router detects that it's receiving packets faster than 58000 kbps it stops sending ACKs and the source will automatically slow down its send speed, this prevents a buffer from blowing up in size at my ISP's router, and now I can control my QOS with low-latency!