7800DXL Bufferbloat

[vanburen]

I have an 7800DXL, running on an ADSL2+ connection.

The 7800DXL suffers from significant bufferbloat on the upload buffer. This means that whenever I upload using the full upload bandwidth of my ADSL connection, the entire connection becomes unusable, as everything grinds to a halt.

This can be seen with a Netalyzer test on my connection which says this:

We estimate your uplink as having 3300 ms of buffering. This is quite high, and you may experience substantial disruption to your network performance when performing interactive tasks such as web-surfing while simultaneously conducting large uploads. With such a buffer, real-time applications such as games or audio chat can work quite poorly when conducting large uploads at the same time.

This shows that the 7800DXL is buffering upload traffic for over 3 seconds, which explains the poor performance of the connection. This is probably because you are using a basic FIFO queue for the upload buffer. This means time sensitive packets like TCP acks are being delayed for over 3 seconds. This is a problem that has been found on other network devices before called bufferbloat, more info here:

http://www.bufferbloat.net/projects/blo ... troduction

It would be great if Billion could offer FQ_Codel (http://en.wikipedia.org/wiki/CoDel) as an option to manage the upload buffer, as this would greatly alleviate the problem.

[davetaht]

Wow. That is a pretty astonishing amount of bufferbloat for DSL.

The only two modern dsl modems I know of that manage their underlying DSL buffering correctly are this http://www.traverse.com.au/geos21-dual- ... appliance (because I was there, 3 years ago, when david woodhouse fixed that driver!) and what free.fr is shipping in their revolution v6 box. Talks with *all* the folk making DSL firmware across the industry (lantiq and others in both the CPE and DSLAM markets) have met with blank looks, sadly, and no bloat-free updates for their hardware and firmware has been distributed to the field that I know of. (please inform me if you find one!)

Orange's bufferbloat is a lot less than yours, but still pretty bad.

http://burntchrome.blogspot.com/2014/05 ... r-dsl.html

I don't know what modem ipfire used to fix their bufferbloat with fq_codel - it was undoubtedly an older one that supported pause frames: http://planet.ipfire.org/post/ipfire-2- ... ufferbloat - but their results were quite phenomenal.

I know of a lot of people that hang on dearly to their older DSL modems because they "just work better" at lower speeds!

The best answer we have currently, lacking better DSL devices, is for people to put a cheap openwrt supported router in front of the modem, and doing software rate shaping up and down using the fq_codel enabled sqm-scripts. There are many other 3rd party firmwares that now support fq_codel, and sqm-scripts itself can run on any form of linux. A key feature of the sqm-scripts is the ability to do correct ATM/DSL framing compensation.

For more information about bufferbloat, we have collected tons of information and benchmarks on these g+ pages: https://plus.google.com/u/0/explore/bufferbloat

as well as on the http://www.bufferbloat.net website, and work is in progress to standardize the solutions now available in an ietf working group: https://tools.ietf.org/wg/aqm/ and mailing list. (There is also a bloat mailing list that is quite useful)

After the development of some new theory and algorithms 3 years ago We like to think that bufferbloat itself - a 50 year old network thorny research problem* is *cured* on ethernet, dsl, cable, and fiber with a combination of fair queueing and AQM technologies...

... but the good news does seem to be taking too long for fixes to reach the end-users, ISPs, and and hardware makers. You CAN easily reduce induced latencies for yourself now with the code and hardware in the field... and if you do... please help two friends fix their bufferbloat - and convince them to convince two more! - and perhaps that way, we'll eventually get to where the ISPs and vendors realize they can make their networks faster and more responsive, always, under all forms of traffic, with a mere firmware upgrade.

The cable industry has half a bufferbloat cure in their DOCSIS 3.1 standard, but wow - I typically only see a max of 2 seconds or so of bufferbloat on a typical cable modem. Here is an example of fixing a cable modem with the sqm-scripts - see the second chart for how bad it is on that technology, and how well it can be cured.

http://snapon.lab.bufferbloat.net/~cero ... sults.html

We at bufferbloat.net are moving on to fixing wifi, next, and are reduced to merely hoping that the knowledge of and solutions to the bufferbloat problem spread more widely, somehow, soon.

At least one fiber deployment I know of, also has some bufferbloat - but not a lot - also cured, with sqm-scripts + fq_codel.

http://zlkj.in/fiber-sqm

We (this includes big names you can find on wikipedia like vint cerf, van jacobson, kathie nichols, eric dumazet, esr, - and many many more obscure volunteers like myself) gave away all this theory and code for *free* in the hope that the industries would jump on them, and improve their services. Sigh.

[vanburen]

Hi Dave,

Great to see you over here on the Billion forums! I have followed your FQ_Codel work on the Ubiquiti forums, I have an edge router POE at the office which is using FQ_Codel. Thanks for all the hard work on the bufferbloat problem. Most users (including me before I learned about bufferbloat) will just assume its their internet connection bandwidth that is causing the slowdown, rather than the networking gear.

I use the Billion at home as it, has very good performance on my marginal ADSL line, I could stick an edge router light behind it, but would really like to keep it a one box setup at home...

Billion has a pretty solid rep for good home network equipment, so I'm really hope they take FQ_Codel on-board. I think it would be a great selling point for their equipment (i'd definitely recommend it more myself anyway).

I will have to watch some of your talks on WiFi.

[davetaht]

What a lot of the hardware companies don't see is that there may be no hardware changes needed, just better firmware, starting first with an incredibly simple BQL-like technique ( Byte queue limits documentation here: http://lwn.net/Articles/454390/ ) at the lowest driver layer - and then layering something like fq_codel on top instead of a pure fifo. fq_codel has been backported as far as linux 2.6.32....

and all all these companies (chipset makers, vendors, and ISPs) have to do, is ask the bufferbloat experts on the bloat and codel mailing lists what to do. Woodhouse debloated his DSL driver noted above in *half a day*. Based on that I figured we could have fixed all the main DSL firmwares in the world with a month of engineering effort, total.

A lot of DSL modems used to come with SFQ or SQF, and those were pretty good solutions too.

Ethernet is even easier to fix, except at higher rates (for example, the edgerouter lite peaked at 60mbits of shaping). Going faster does require a faster cpu and/or some hardware offload support.

[davetaht]

and thank you for your interest in our upcoming wifi work! Wifi had many more problems than just bufferbloat, identified in my talk at MIT here:

https://plus.google.com/u/0/10794217561 ... y8DpJy52PY

and the fixes are all interrelated, and it has taken a long time to sort out what had to be done - but I think the low hanging fruit is now fixable, as I described in a talk to the IEEE 802.11 wg here:

http://snapon.lab.bufferbloat.net/~d/ie ... rbloat.pdf

I hope to kick the make-wifi-fast project off the ground by mid-april, and I expect we will see the same 20x or more reductions in latency under load on wifi and enormous improvements in per-station behavior as we did on removing bufferbloat from all the other technologies.

*In under a year* - for those willing to support the project and jump on the bleeding edge.

[davetaht]

one last post before I go back to work...

Many other videos and papers besides mine are here. In particular, everyone involved with networking should see Van Jacobson's talk here:

http://www.bufferbloat.net/projects/cer ... oat-videos

And I also highly recommend Stephen Hemminger's highly entertaining talks, as well as (for a real backgrounder on fair queueing technologies) my talks at the university of modena.