[Snap] WOW! Got International Bandwidth!?

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › [Snap] WOW! Got International Bandwidth!?

1 | 2 | 3

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731646 12-Dec-2012 22:26

I guess the point that SideFX is making is that he can't be sure that this is pure international bandwidth from his particular test source.

mercutio

1392 posts

Uber Geek
+1 received by user: 134

#731648 12-Dec-2012 22:30

SamF: I guess the point that SideFX is making is that he can't be sure that this is pure international bandwidth from his particular test source.

That he's getting 30 megabit on an international download in peak time with a single thread.

Sure I'd love to say that 30 megabit single threaded performance should be the norm, but it's just not realistic.

Maybe to Los Angeles hosts you may peak around there.

One thing that can happen with high speed downloads is there's a lot of collection collapse type stuff as you start moving into those faster speeds where speed can start to go up then suddenly collapse in speed. If you watch a single thread you can see the effect. Sometimes reducing window size can help reduce the chances of that collapsing.

Basically from what I understand this is because various queues end up being too short and can't take the burst of new packets, and drop, giving sudden tcp/ip speed drops. There's a few things in very recent Linux versions that can help if the server runs recent Linux version, but a lot of servers are infrequently updated. (client side changes aren't needed in this case)

sidefx

3775 posts

Uber Geek
+1 received by user: 1295

Trusted

#731653 12-Dec-2012 22:33

mercutio:
1 megabyte/sec is reasonable on a high speed connection for uncached international content.

Sure I'd love to get consistent line rate everywhere, no packet loss, no delays, etc etc, but it's reasonably comparable to what a lot of other people around the world experience.

If the caching actually caches stuff that you want to download, and you get fast speeds then surely it's a good thing, and it's working as intended.

Yeah absolutely, just to be clear I'm completely happy with both rates. And I have no issue with caching - indeed it's very useful. But as above, essentially my point is just that getting 3.5-4 MB/s to US East coast Servers on speedtest and single threaded file downloads seems too good to be true ;-)

"I was born not knowing and have had only a little time to change that here and there." | Octopus Energy | Sharesies
- Richard Feynman

SamF

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731655 12-Dec-2012 22:36

Right, well, to be honest, I've never seen ANY download from ANYWHERE get even close to 30mbit on a single thread.

I just tested a popular locally Akamai cached file from Microsoft just now; 500kbytes/sec max. Overall speed also depends on how much speed the remote server is allowing per connection too.

TBH, I'm not quite sure what everyone's concern is about single threaded downloads.

mercutio

1392 posts

Uber Geek
+1 received by user: 134

#731657 12-Dec-2012 22:43

SamF: Right, well, to be honest, I've never seen ANY download from ANYWHERE get even close to 30mbit on a single thread.

I just tested a popular locally Akamai cached file from Microsoft just now; 500kbytes/sec max. Overall speed also depends on how much speed the remote server is allowing per connection too.

TBH, I'm not quite sure what everyone's concern is about single threaded downloads.

multithreading is bad :)

Try: http://cachefly.cachefly.net/100mb.test for a good speed test. It's Australia, fast connection. They're much better than Akamai, you don't have the low hit rate problem.

SamF

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731660 12-Dec-2012 22:47

Lol, why do you say multithreading is bad?

I've heard the argument that using multiple threads is 'selfish' because you use more bandwidth than other users, but I don't buy that. If I use 10 threads instead of 1, sure I use 10x more bandwidth than others, BUT I'm finished 10x faster than everyone else, so the net bandwidth usage over time is exactly the same!

Yeah, that 100mb test file came down at 3.5MB/sec so not bad. Obviously that server has less (or no) restriction on per connection speeds.

Mighty Ape

Shop now at Mighty Ape (affiliate link).

sidefx

3775 posts

Uber Geek
+1 received by user: 1295

Trusted

#731662 12-Dec-2012 22:50

Nothing wrong with multithreading - wget (single thread) was just convenient to see if it was achieving higher rates than I would have expected :)

"I was born not knowing and have had only a little time to change that here and there." | Octopus Energy | Sharesies
- Richard Feynman

mercutio

1392 posts

Uber Geek
+1 received by user: 134

#731670 12-Dec-2012 23:07

SamF: Lol, why do you say multithreading is bad?

I've heard the argument that using multiple threads is 'selfish' because you use more bandwidth than other users, but I don't buy that. If I use 10 threads instead of 1, sure I use 10x more bandwidth than others, BUT I'm finished 10x faster than everyone else, so the net bandwidth usage over time is exactly the same!

Yeah, that 100mb test file came down at 3.5MB/sec so not bad. Obviously that server has less (or no) restriction on per connection speeds.

Multithreading means that you can get erratic and unstable transfers, giving strong bursts of packets often getting dropped, wasting network traffic/bandwidth, encouraging people to do complicated weird shaping to maintain adequete performance, meaning casual web browsing ends up doing retransmits on normal web pages.

That said, it used to be that a lot of servers didn't have big enough window sizes for fast connections, so it was necessary, and now some providers like cogent and telstraglobal drop enough packets that you really need to to get good transfer speeds.

If you are going to multithread, it's best to not use more than is necessary, ie if you get 800k/sec then using 5 threads should be adequete, although i reckon 4 is more fair.

The net bandwidth usage over time actually increases the closer you are to your line rate. Leading to either buffering or dropping packets.

It'd be a lot better when can just get decent speeds with single connection.

SamF

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731672 12-Dec-2012 23:14

For sure, obviously less threads is better than more, but I'm not so sure that there's that much more overhead. My multithreaded transfers tend to run very smoothly. I have done quite a bit of work to get them like that of course, but it is possible to do efficiently.

mercutio

1392 posts

Uber Geek
+1 received by user: 134

#731673 12-Dec-2012 23:15

SamF: For sure, obviously less threads is better than more, but I'm not so sure that there's that much more overhead. My multithreaded transfers tend to run very smoothly. I have done quite a bit of work to get them like that of course, but it is possible to do efficiently.

Do you have your own server that you're downloading from? If so I'd be curious what data volumes server sends out compared to what you receive.

SamF

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731674 12-Dec-2012 23:18

Naa, I'm testing from a Usenet server.

Dyson

Shop now for Dyson appliances (affiliate link).

mercutio

1392 posts

Uber Geek
+1 received by user: 134

#731680 12-Dec-2012 23:20

SamF: Naa, I'm testing from a Usenet server.

West coast? Shouldn't need 10 threads for west coast.

SamF

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731681 12-Dec-2012 23:21

Not exactly sure where it is TBH.

ObidiahSlope

260 posts

Ultimate Geek
+1 received by user: 66

#731697 13-Dec-2012 00:26

There was an item about Snap in yesterdays(Wed 12/12) Christchurch Press.

Said that Snap had directly rented a slice of the Southern Cross Cable.

Maybe that is the reason foe WOW speeds to the land of the free and the home of the brave.

Obsequious hypocrite

SamF

1598 posts

Uber Geek
+1 received by user: 252

Trusted

#731699 13-Dec-2012 00:32

Hmm, interesting! Maybe you're right!

Is there an online version of the article?

1 | 2 | 3