Thursday, November 15, 2012

Measuring Internet Connection Throughput

[1]
[Updated Post Friday December 10, 2012]

I thought it would be helpful to share my results with everyone.  Figure 2 is chart showing hourly trending for my Internet speed over the last 2 weeks.

Figure 2: Hourly Trend of Broadband Speed
My Internet connection to my ISP is 50Mbps.  The chart shows I'm receiving slightly less than 1/2 my expected bandwidth.  I changed the chart around to produce bandwidth results in megabits per second (e.g., Mbps).  Megabits is how my ISP advertises their connections so it's less error prone for me to think of my results in terms of Mbps.  At little Excel magic yields the chart in Figure 2.  Now that measurements and charting are in good shape I'm thinking about my data and results.  I have come to the conclusion it's best if I rerun the tests connected directly to my ISPs access point.

In my home network my *NIX host, running the testing, connects to a HP Procurve 1700-8 managed switch, then to a Netgear Powerline AV500 Adapter running as a bridge, then to my Asus RT-N56U router/firewall, and finally to the ISPs access point.  It seems likely to me the WAN connection should be the most rate limited of all the gear.  However, to eliminate all possibility of error, I'm going to connect the *NIX host directly to the ISP access point and rerun the tests.  Sigh.

[Updated Post Friday November 23, 2012]

I made a few improvements I thought I would share.  The original program code had interleaving series data in rows.  There's nothing wrong with the data but it's hard to graph in Excel.  I improved the program by moving series data to columns.  Once I get all the data into Excel I delete all the columns except: DATE, TIME, ELAPSED-sjc01-10, ELAPSED-sjc01-100, ELAPSED-wdc01-10, ELAPSED-dal01-10.  To graph, I select all the data including column headings and choose the graph I want.  Following are the improved files.

Binary - ispspeedcheck2.jar
Code - ISPSpeedCheck2.zip

If you run the tool, following are a few lines of sample data you can expect to see in the new CSV file.

DATE,TIME,sjc01-10,PAYLOADSZ-sjc01-10,ELAPSED-sjc01-10,sjc01-100,PAYLOADSZ-sjc01-100,ELAPSED-sjc01-100,wdc01-10,PAYLOADSZ-wdc01-10,ELAPSED-wdc01-10,dal01-10,PAYLOADSZ-dal01-10,ELAPSED-dal01-10
11/20/2012,7:14:09 PM,sjc01-10,11536384,8145,sjc01-100,104874307,51949,wdc01-10,11536384,6383,dal01-10,11536384,5972
11/20/2012,8:15:14 PM,sjc01-10,11536384,5295,sjc01-100,104874307,46618,wdc01-10,11536384,6157,dal01-10,11536384,5537
11/20/2012,9:16:25 PM,sjc01-10,11536384,8588,sjc01-100,104874307,49301,wdc01-10,11536384,6273,dal01-10,11536384,5595

The following is a chart I generated with a few hours of data.

Updated Figure 1: Broadband Bandwidth Hourly Trend
Keep in mind if your going to run the new jar you should update your command line like the following.
java -Dfile=./ispspeedtest2.csv -jar ./ispspeedcheck2.jar

I'm interested to see some real data.  Happy Thanksgiving!  --Milton

[Original Post]

Many of us at one time or another wonder if we are really receiving our full Internet bandwidth.  There are a number of ways we can get this information.  I will cover a few of them as well as provide a small Java executable along with source code so you can experiment on your own.

If your only looking for a quick easy test for your Internet connection you don't need to write a program.  Open your browser to speedtest.net and give them a try.  I like speedtest.net and it's easy to use.  In my case, I wanted an hour by hour trend of my Internet connection throughput.  To get a trends on speedtest would be bothersome since I would have to run the test manually once each hour.  Great tool but you need a little something more if your interested in trends.  This monkey has better buttons to push so I decided on a Java project.  Moving on, following are some of the questions and points I considered when thinking about my broadband bandwidth.

Am I receiving my full Internet bandwidth?
Broadband is prickly business.  On the one hand, Internet Service Providers(ISPs) advertise and sell their connections by bandwidth.  On other hand, they don't make any guarantees about the bandwidth they provide.  As a consumer, what am I really receiving?  The impact to my wallet is guaranteed, that happens regularly each month, but my bandwidth is variable.  To be an ISP is a great line of business.  ISPs can command a constant monthly price for a product they may deliver in more or less in quantity -- at their sole discretion.  Imagine a grocery store operating on the same principle.  You pull up to the checkout counter with a full shopping cart of groceries or 1/2 a cart and it makes no difference.  It's all the same price.  Ok, I feel a rant coming on so I'll stop now but you get my point and source of my curiosity.

Paying for additional bandwidth provides more bandwidth but how much more?
I'm not sure how to answer this question at the moment, at least without changing my service.  Perhaps data on speedtest.net or similar sites.  For instance, if my connection is 50Mb/sec download and I'm receiving 25Mb/sec will upgrading to 128Mb/sec produce half as much throughput or 64Mb/sec on average?  Likewise, if I cut my bandwidth to 25Mb/sec do I really get half as much throughput or is it only 20% of the advertised capacity for slower speeds?  My assumption is that higher speed connections have more wiggle room for throttling whereas the lower speed connections are closer to the advertised throughput.  I think these questions are more suitable for an ISP review as opposed to a broadband tool measurement discussion but it's still something I'm interested to know.

Is there more Internet bandwidth available during off-peak hours?
It seems common sense during early morning or late evening hours more bandwidth is available since less people in your region are surfing the Internet.  Of course, does it really matter if there is Internet bandwidth when nobody wants to use it?  If a tree falls in the woods... ah, never mind.  Ok, unless your a torrent freak or schedule off peak downloads there's probably nothing practical in this line of thinking but I'm curious to find out.

Complexities of bandwidth measurement
Measuring your throughput is easy.  Understand the reasons for the numbers are more complicated.  There are many factors that influence overall results.  I don't claim to be a network expert but following are some areas that come to mind.

WAN Speed - Yay.  Generally, the broadband speed rating is between you and your ISP.  There is absolutely no guarantee to other servers on the Internet.  So the at best, you will only receive your advertised throughput to your ISP.  Traffic anywhere else may take the slow boat.

LAN Speed - Are the kids playing games or downloading content over the same connection?  If so, this will impact any measurements.  Also any IP enabled devices that share the LAN, television sets, smart phones, almost anything.  Even thermostats are IP enabled (incidentally NEST is really cool - Google it).  You could have significant traffic on your LAN interfering with your results.

WIFI - Are you working over WIFI?  Limited WIFI bandwidth or poor signal quality will negatively impact your results.  Better to be connected directly Ethernet.

Network Hardware - All boxes with blinky lights are not created equal.  I've tried many brands and settled on a highly rated ASUS RT-N56U router recommended on SmallNetBuilder.  One word -- awesome!  The value of good network hardware cannot be understated.  Don't assume a popular brand is the best.  In the age of the Internet, lots of people can make the same mistakes.  Don't follow the sheeple.  Do your own homework.

Traffic Shaping - ISP's shape traffic all the time.  Some have been accused of throttling peer to peer traffic like bittorrent.  It's almost guaranteed if your running speed tests against popular sites like speedtest.net your ISP optimizes that traffic.  Meaning, your throughput to speedtest.net is great but perhaps not consistent across other web sites.  On the surface, traffic shaping sounds a little dubious but the practice has it's place.  Even in the home, most consumer devices support some amount of traffic shaping.  A really comprehensive test would probably test traffic over several different protocols.  A test over different protocols would help work around traffic shaping.  I'm not going to do this just yet but it might produce a better result.  Electronic Frontier Foundation has some interesting resources on shaping[2].

Broadband history
Taking a speedtest.net reading and tossing it over the fence to your ISP is not very convincing if you have bandband throughput concerns.  Too many factors, as mentioned previously, can influence a short speedtest run and produce misleading results.  On the other hand, approaching your ISP with a 2 month history of hour-by-hour Internet bandwidth trend data is compelling evidence.  Even if you don't have a case to prove, more comprehensive information is always desirable.  What is the average WAN usage over time by hour of day?  The mean, median, standard deviation of download times for different size payloads?  Are 100Mb payload download times one order of magnitude greater than 10Mb or different?

My environment
I have several computer systems at the house.  In my home we have at least two computers per person.  We also have a cadre of WIFI enabled devices like Kindle Fire, iPhones, IP printer, VOIP devices, flatscreen TV, etc.  We use several operating systems: OSX for business, Ubuntu Linux for special tasks, and Windows for gaming.  I develop on my OSX system using Eclipse and deploy longer running tests on my Ubuntu server which works tirelessly nights and weekends without interruption.

The experiment
I thought about adding lots of bells and whistles to the program code.  Adding a user interface, fancy logging, etc.  In the end, I chose to keep the code very simple.  The output of the program is a simple CSV file that can be loaded into Microsoft Excel.  Following are a few lines of output from the CSV log file.
milton@sparky:~/bin$ tail -f ./ispspeedtest.csv
DATE,TIME,SERVER,PAYLOADSIZE,ELAPSED
11/14/2012,8:31:13 PM,sjc01-10,11536384,6355
11/14/2012,8:32:15 PM,sjc01-100,104874307,61625
11/14/2012,8:32:23 PM,wdc01-10,11536384,7625
11/14/2012,8:32:31 PM,dal01-10,11536384,7210
11/14/2012,9:32:37 PM,sjc01-10,11536384,6469
11/14/2012,9:33:37 PM,sjc01-100,104874307,60103
11/14/2012,9:33:45 PM,wdc01-10,11536384,7121
11/14/2012,9:33:52 PM,dal01-10,11536384,6979

The first line is the command line.  I use Linux tail command to show the log realtime in a command shell window.  Following the command line are, column headings, date and time message was logged, server code the payload was downloaded from, payload size in bytes, and the download time in milliseconds.  For instance, the first line of data, the payload is downloaded from sjc01-10 (a code for a speedtest server in SF Bay Area).  The size of the payload was 11.5Mb and it took 6.3 seconds to download.  The longer the program runs the more data is produced.  The tests are single threaded.  Once you have collected enough data you can stop the program (e.g., CTRL-C) or the program will stop on it's own after 1000hrs with the default settings.

Figure 1 shows a couple of days of hourly broadband trend data.  The top red line is the hourly data for a 100Mb zip file payload.  The blue line on the bottom of the figure is the hourly data for the 10Mb zip file payload.
Figure 1: Broadband bandwidth hourly trend
I realize the chart is small and not easy to read.  I provided the chart to give you an idea of what a finished chart may look like when it's done.  Unlike the stock market or your 401k, a line trending down is good and means your downloads are taking less time.

Source code, binaries, and limitations (and dragons)
Following is a sample source code package as well as executable jar if you don't want to compile.  Keep in mind the program is not full featured and not suitable for production systems.  I have provided no security controls like input validation.  The program is not multi-process safe or even thread safe in some places if you choose to repurpose some of the classes.  I originally made this for myself.  Thar be dragons!  Consider yourself warned.

The servers where I download payloads are hard coded in the program for North America only.  If you live elsewhere, you will want to experiment with ISPs closer to your region and you will need to make some improvements to the code.
Code - ispspeedcheck.zip

Running the binary depends upon your system but to get it running on my system I use the following.  The command launches the program and tells it to log CSV data to the ispspeedtest.csv file in the current working directory.
java -Dfile=./ispspeedtest.csv -jar ./ispspeedcheck.jar

I have tried it under Java 6 and Java 7 on OSX and Java 6 under Ubuntu Linux.  If anyone needs some assistance let me know and I might be able to make some changes or suggest some.  I'm just not sure who's interested in this stuff at the moment so I see need to invest anymore time than necessary to help answer my personal curiosities.

Alternative ideas
Writing a program to measure your Internet throughput is probably not the most efficient use anyone's time.  My first thought was to use Linux Curl command or WGET command to download the payloads.  Then use a graphing facility like RRDtool.  I'm not too familiar with RRDtool but it looked powerful and promising.  If your a Linux user a little Googling and hammering out a shell script is a good way to go.  I did some Googling to find desktop applications.  I didn't find anything for OSX.  I did find an interesting package for Windows.  To be honest, I just wanted to do this in Java.

Contacting your ISP
Before you decide to share any information with your ISP make sure you have your facts straight.  Double check your information, cross check with other tools, etc.  It's easy to misinterpret results or make a programming mistakes if you decide to make some improvements to the code.  If you've taken all precautions and you still wish to contact your ISP then be courteous.  Often raising awareness will get you on the path to resolution.  Also keep in mind your advertised bandwidth is not a guarantee.  Bandwidth guarantees are available but not generally to consumers since they cost a fortune.  Take the time to become familiar with your ISP's Terms of Service for your Internet connection before you call.

Verdict and next steps
No real results just yet.  I'm still getting started and I only have a couple of days of data.  I will update in the future with my results or describe how the program works if there's interest.

Good luck.

[1] Alves, J. Clipart - Wavy Checkered Flag. Digital image. Clipart - Wavy Checkered Flag. Clipart.org, 25 Mar. 2010. Web. 15 Nov. 2012. <http://openclipart.org/detail/62779/wavy-checkered-flag-by-j_alves>.

[2] "Test Your ISP." Electronic Frontier Foundation. EFF, n.d. Web. 15 Nov. 2012. <https://www.eff.org/testyourisp>.

Share It!