[Snap] WOW! Got International Bandwidth!?

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › [Snap] WOW! Got International Bandwidth!?

1 | 2 | 3

yitz

2080 posts

Uber Geek

#731716 13-Dec-2012 01:23

More of an advertisement than anything:
https://i.imgur.com/46Gu5.jpg

SamF

1578 posts

Uber Geek

Trusted

#731717 13-Dec-2012 01:36

Nice! Cheers! :)

Was that scanned?

Lorenceo

904 posts

Ultimate Geek

Trusted

#731728 13-Dec-2012 03:23

Pretty sure that their capacity on the SCC is to Australia only. Snap are peering directly at Equinix Sydney now, which eliminates a lot of Telstra's odd routing.
Bandwidth to the rest of the world however seems to either go through Telstra Global, Alter.net or Cognetco.

mercutio

1392 posts

Uber Geek

#731826 13-Dec-2012 10:12

tested tcp and udp performance with sidefx last night, got good speeds with TCP/IP to NZ but not to Los Angeles, and got good speeds using UDP with udt (udt.sourceforge.net) and iperf after reducing the segment size, from both Los Angeles and NZ.

Was pretty confused why NZ was even worse than Los Angeles, but his connection is on 1492 mtu it seems. Connection was peaking at 36 megabit and dropping to 17 megabit shortly after. With TCP got 4.4 megabytes/sec to NZ which is 35 megabit, so it appears there's an issue with downloading over your max speed, which can impact maximum rates obtained. From NZ can recover quicker than from overseas. The Los Angeles host was about 142 msec away, NZ about 5 msec away.

This is pretty much the same problem TelstraClear cable had...

Would be curious how a UFB connection goes with UDT. It's easier to test on Linux, but you can compile on Windows with visual studio.

mercutio

1392 posts

Uber Geek

#731828 13-Dec-2012 10:13

Lorenceo: Pretty sure that their capacity on the SCC is to Australia only. Snap are peering directly at Equinix Sydney now, which eliminates a lot of Telstra's odd routing.
Bandwidth to the rest of the world however seems to either go through Telstra Global, Alter.net or Cognetco.

That's where Telstraclear in general go through. That said, now they've bought SCC capacity to Australia they may buy to the US. Cogent and Telstraglobal both in general seem a little hit and miss for fast single-stream tcp/ip transfers. Sometimes repeating tests can give different routes and different speeds. Still at the beginning of this year, or late last year sometime after Telstraclear's unlimited weeekend performance went up significantly, and it's not drastically bad now.

quakeguy

111 posts

Master Geek

Trusted

#731939 13-Dec-2012 11:38

With the stock Linux kernel settings you can run into limits in TCP single-threaded throughput, even in modern (3.2-3.5) kernels (For reference, Ubuntu 12.04 uses kernel 3.2).

The main limitation you hit is that window size becomes a throughput constraint at latencies higher than about 150ms. I don't have the exact figure, so don't quote me on this...

Since most speedtest.net servers will be running Linux, and some administrators of these servers will know about this tweak, you will get random results which range from quite good to quite poor, depending on what tweaks (if any) have been done.

I have adjusted a server I run in Germany to compensate for this. Basically, you can tweak the maximum TCP window and buffer sizes in the sysctl.conf on most Linux distributions. I run Ubuntu 12.04LTS with kernel 3.2.0-32-generic.

The NIC on my server is connected at 100mbit/sec.

Here's what I've done:

[appended to /etc/sysctl.conf]

net.core.wmem_max=12582912
net.core.rmem_max=12582912
net.ipv4.tcp_rmem= 10240 87380 12582912
net.ipv4.tcp_wmem= 10240 87380 12582912
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_no_metrics_save = 1
net.core.netdev_max_backlog = 5000

You can then either reboot, or run 'sysctl -p' to reload the settings from sysctl.conf into the running kernel.

Here, try a speedtest from my server: https://de.tw.net.nz/testfile

Straight wget: wget -O /dev/null --no-check-certificate https://de.tw.net.nz/testfile

(output to /dev/null - and don't check certificate because I'm cheap and can't afford a certificate ;-))

**Edit: You might want to note that a number of users immediately did simultaneous tests as soon as I posted this, and I only have 100mbit/sec of upload, so... run your test a few times :-)

“I do not think there is any thrill that can go through the human heart like that felt by the inventor as he sees some creation of the brain unfolding to success... Such emotions make a man forget food, sleep, friends, love, everything.” - Nikola Tesla

Disclaimer: Views expressed in my posts do not necessarily reflect those views of my employer.

mercutio

1392 posts

Uber Geek

#731945 13-Dec-2012 11:43

quakeguy: With the stock Linux kernel settings you can run into limits in TCP single-threaded
Here, try a speedtest from my server: https://de.tw.net.nz/testfile

Straight wget: wget -O /dev/null --no-check-certificate https://de.tw.net.nz/testfile

(output to /dev/null - and don't check certificate because I'm cheap and can't afford a certificate ;-))

quakeguy... https compresses null bytes, you need random data. or to use http.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

quakeguy

111 posts

Master Geek

Trusted

#731947 13-Dec-2012 11:47

mercutio:
quakeguy: With the stock Linux kernel settings you can run into limits in TCP single-threaded
Here, try a speedtest from my server: https://de.tw.net.nz/testfile

Straight wget: wget -O /dev/null --no-check-certificate https://de.tw.net.nz/testfile

(output to /dev/null - and don't check certificate because I'm cheap and can't afford a certificate ;-))

quakeguy... https compresses null bytes, you need random data. or to use http.

Oh, does it? Hmm. Well, I've replaced testfile with random data now!

Disclaimer: Views expressed in my posts do not necessarily reflect those views of my employer.

mercutio

1392 posts

Uber Geek

#731952 13-Dec-2012 11:51

quakeguy: With the stock Linux kernel settings you can run into limits in TCP single-threaded throughput, even in modern (3.2-3.5) kernels (For reference, Ubuntu 12.04 uses kernel 3.2).

The main limitation you hit is that window size becomes a throughput constraint at latencies higher than about 150ms. I don't have the exact figure, so don't quote me on this...

Since most speedtest.net servers will be running Linux, and some administrators of these servers will know about this tweak, you will get random results which range from quite good to quite poor, depending on what tweaks (if any) have been done.

I have adjusted a server I run in Germany to compensate for this. Basically, you can tweak the maximum TCP window and buffer sizes in the sysctl.conf on most Linux distributions. I run Ubuntu 12.04LTS with kernel 3.2.0-32-generic.

The NIC on my server is connected at 100mbit/sec.

Here's what I've done:

[appended to /etc/sysctl.conf]

net.core.wmem_max=12582912
net.core.rmem_max=12582912
net.ipv4.tcp_rmem= 10240 87380 12582912
net.ipv4.tcp_wmem= 10240 87380 12582912
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_no_metrics_save = 1
net.core.netdev_max_backlog = 5000

Are you sure you really need all those?

In my own testing I've fun much more moderate settings to work well. Although I also use more recent Linux version for testing. (3.7.0-rc7 atm)

The default seems to be around 6 megabytes for rmem third field giving 3 megabytes window and 3 megabytes socket buffer. You're about doubling it. But your server should be about 1/3rd of a second away... so it's only really if you expect 200 megabits/sec throughput.

Here, try a speedtest from my server: https://de.tw.net.nz/testfile

Straight wget: wget -O /dev/null --no-check-certificate https://de.tw.net.nz/testfile

(output to /dev/null - and don't check certificate because I'm cheap and can't afford a certificate ;-))

**Edit: You might want to note that a number of users immediately did simultaneous tests as soon as I posted this, and I only have 100mbit/sec of upload, so... run your test a few times :-)

I found I could do http without https... hmm

mercutio

1392 posts

Uber Geek

#731953 13-Dec-2012 11:52

quakeguy:
mercutio:
quakeguy: With the stock Linux kernel settings you can run into limits in TCP single-threaded
Here, try a speedtest from my server: https://de.tw.net.nz/testfile

Straight wget: wget -O /dev/null --no-check-certificate https://de.tw.net.nz/testfile

(output to /dev/null - and don't check certificate because I'm cheap and can't afford a certificate ;-))

quakeguy... https compresses null bytes, you need random data. or to use http.

Oh, does it? Hmm. Well, I've replaced testfile with random data now!

Yeah, I tested a whole lot of vps places in many locations a while back, and ran into people trying to do https with zero data a few times. I was a bit surprised when DSL was showing me over my max speed the first time.

helis

96 posts

Master Geek

#732030 13-Dec-2012 13:45

Not sure this is all that new. I've been seeing 4MB/s from US and EU servers since I joined up in September/October. Even at the slower times I still get 2MB/s+ from US servers.

Cbfd

307 posts

Ultimate Geek

#732161 13-Dec-2012 17:10

Just done some tests on a friends Snap Fibre Connect with a few tweeks - not to bad tbh :)

SamF

1578 posts

Uber Geek

Trusted

#732227 13-Dec-2012 19:32

mercutio: ... so it appears there's an issue with downloading over your max speed, which can impact maximum rates obtained.

Yeah, this is something I noticed a few years ago. I tend to rate limit just under the consistently achievable max download speed using download managers or firewalls so that this doesn't happen. It makes quite a bit of difference!

Kickass

292 posts

Ultimate Geek

#733204 16-Dec-2012 17:59

i am also on snap and about a month ago international got heaps better ....

Great for Streaming HD!

1 | 2 | 3