Always check twice

You should always double check your timing testing methodology.

Robbie Ferguson got very excited about:

So I thought, let’s run the world’s simplest test: how fast does wget receive the jQuery library on Linux? It may not be a realistic benchmark in all cases, but it gives us a bit of a look at how quickly each service delivers the js.

uh oh …. “simple” tests aren’t so simple sometimes.

FIRST TIME
Cloudflare:

Connecting to cdnjs.cloudflare.com|190.93.242.8|:80... connected
92,629      --.-K/s   in 0.05s   
2013-03-22 14:04:54 (1.73 MB/s) - ‘jquery.min.js’ saved [92629]

Google:

Connecting to ajax.googleapis.com|74.125.28.95|:80... connected.
92,629      56.3KB/s   in 1.6s   
2013-03-22 14:05:10 (56.3 KB/s) - ‘jquery.min.js.1’ saved [92629]

But then on the second (and third and …) request:

Cloudflare:

Connecting to cdnjs.cloudflare.com|141.101.123.8|:80... connected.
HTTP request sent, awaiting response... 200 OK
92,629      --.-K/s   in 0.1s    
2013-03-22 14:08:07 (893 KB/s) - ‘jquery.min.js.6’ saved [92629]

Google:

Connecting to ajax.googleapis.com|74.125.141.95|:80... connected.
92,629      --.-K/s   in 0.1s    
2013-03-22 14:08:01 (643 KB/s) - ‘jquery.min.js.5’ saved [92629]

Notice that Google and Cloudflare both download at about the same rate (and very fast).

Notice the difference? In both cases, the second time out the DNS resolved to a different and closer server:

Cloudflare traceroute to server:

 3  te-0-1-0-1-ur05.santaclara.ca.sfba.comcast.net (68.87.196.113)  17.153 ms  9.443 ms  12.030 ms
 4  te-1-1-0-0-ar01.sfsutro.ca.sfba.comcast.net (69.139.198.82)  15.320 ms
    te-1-1-0-9-ar01.sfsutro.ca.sfba.comcast.net (69.139.198.178)  20.916 ms
    te-1-1-0-7-ar01.sfsutro.ca.sfba.comcast.net (69.139.198.174)  17.887 ms
 5  he-3-8-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.94.85)  18.458 ms  22.712 ms  23.970 ms
 6  pos-0-5-0-0-pe01.11greatoaks.ca.ibone.comcast.net (68.86.87.162)  20.503 ms  19.316 ms  19.999 ms
 7  173.167.57.122 (173.167.57.122)  39.213 ms  25.379 ms  20.759 ms
 8  as13335.xe-9-0-2.ar2.sjc1.us.nlayer.net (69.22.153.74)  17.737 ms
    as13335.xe-8-0-5.ar2.sjc1.us.nlayer.net (69.22.130.146)  31.771 ms
    as13335.xe-9-0-2.ar2.sjc1.us.nlayer.net (69.22.153.74)  20.209 ms
 9  190.93.242.8 (190.93.242.8)  15.915 ms  17.249 ms  16.325 ms

Cloudflare occasionally sent me to a different server but the speed was consistently 0.1s

Google traceroute to first server:

 3  te-0-1-0-1-ur05.santaclara.ca.sfba.comcast.net (68.87.196.113)  9.586 ms  11.568 ms  11.461 ms
 4  te-1-1-0-5-ar01.sfsutro.ca.sfba.comcast.net (68.86.143.94)  12.663 ms
    te-1-1-0-4-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.66)  19.735 ms
    te-1-1-0-3-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.62)  47.036 ms
 5  he-1-5-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.90.93)  19.118 ms  21.943 ms  35.681 ms
 6  pos-0-2-0-0-pe01.529bryant.ca.ibone.comcast.net (68.86.87.6)  13.524 ms  13.981 ms  34.744 ms
 7  66.208.228.226 (66.208.228.226)  14.712 ms  16.271 ms  14.320 ms
 8  72.14.232.138 (72.14.232.138)  14.418 ms  13.487 ms
    72.14.232.136 (72.14.232.136)  21.560 ms
 9  209.85.250.63 (209.85.250.63)  17.685 ms
    209.85.250.66 (209.85.250.66)  44.087 ms  20.266 ms
10  72.14.232.63 (72.14.232.63)  48.702 ms
    216.239.49.198 (216.239.49.198)  31.927 ms
    72.14.232.63 (72.14.232.63)  31.606 ms
11  72.14.233.200 (72.14.233.200)  65.608 ms
    72.14.233.202 (72.14.233.202)  39.036 ms
    72.14.233.140 (72.14.233.140)  43.722 ms
12  64.233.174.125 (64.233.174.125)  33.895 ms  39.076 ms
    64.233.174.97 (64.233.174.97)  36.646 ms
13  * * *
14  pc-in-f95.1e100.net (74.125.28.95)  66.471 ms  35.639 ms  76.047 ms

Google traceroute to second server:

 3  te-0-1-0-1-ur05.santaclara.ca.sfba.comcast.net (68.87.196.113)  11.852 ms  10.684 ms  11.592 ms
 4  te-1-1-0-5-ar01.sfsutro.ca.sfba.comcast.net (68.86.143.94)  16.404 ms
    te-1-1-0-4-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.66)  23.193 ms
    te-1-1-0-3-ar01.sfsutro.ca.sfba.comcast.net (68.85.155.62)  20.005 ms
 5  he-1-6-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.90.157)  40.921 ms  20.899 ms  23.967 ms
 6  pos-0-9-0-0-pe01.11greatoaks.ca.ibone.comcast.net (68.86.88.110)  20.604 ms  18.826 ms  17.855 ms
 7  173.167.57.122 (173.167.57.122)  20.478 ms  18.200 ms  21.693 ms
 8  as13335.xe-8-0-5.ar2.sjc1.us.nlayer.net (69.22.130.146)  15.495 ms
    as13335.xe-9-0-2.ar2.sjc1.us.nlayer.net (69.22.153.74)  16.727 ms
    as13335.xe-8-0-5.ar2.sjc1.us.nlayer.net (69.22.130.146)  34.721 ms
 9  141.101.123.8 (141.101.123.8)  16.346 ms  16.799 ms  35.777 ms

I then tried with a different file and for both google and cloudflare I got 0.1s

So what does this mean? The initial delta was related to which server replied to the DNS lookup. For whatever reason google responded to the first DNS request for ajax.googleapis.com with a server that was far away. Later requests got better.
Cloudflare was consist about providing a better server on the first try.