Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


3 posts

Wannabe Geek


Topic # 11689 6-Feb-2007 15:58
Send private message

Every large file I download is corrupt, as per the thread here and also at gp.

http://www.gpforums.co.nz/thread/348250/?s=

Is there any download manager that slows the download to check for data corruption on the fly? Hash checks on 1MB sections or something, and then automatically re-downloads any corrupt sections?

I don't know if this is even possible...

But Torrent clients do this, so I can download torrents fine. If I could find a way to do this with normal downloads, I would be very happy.

Windows or Mac.

Cheers
Tec

Create new topic
643 posts

Ultimate Geek


  Reply # 59884 6-Feb-2007 17:38

this is symptom of ATM congestion/over-subscription, when 53 byte cells are lost (=~40 lost bytes to higher protocol layers?) TCP & PPP's CRC16 check won't always fail. Telecom should know about it already they will have alarms in place.

However, this is mere speculation. Hopefully I'll be proven wrong! (sbiddle, fraktul?)

Stick to using Bittorrent, emule or anything else that uses MD5 hash checks.




Sniffing the glue holding the Internet together

1818 posts

Uber Geek
+1 received by user: 52

Trusted

Reply # 59900 6-Feb-2007 20:12
Send private message

Most download managers should do all that CRC checking on the packets it get sent.

You could try to reduce the number of splits that your download manager breaks the download into.
The more streams u have the more chance of them getting dropped/packet filtered/confusing telecoms stuff.







3 posts

Wannabe Geek


  Reply # 59993 7-Feb-2007 13:01
Send private message

barf: this is symptom of ATM congestion/over-subscription, when 53 byte cells are lost (=~40 lost bytes to higher protocol layers?) TCP & PPP's CRC16 check won't always fail. Telecom should know about it already they will have alarms in place.

However, this is mere speculation. Hopefully I'll be proven wrong! (sbiddle, fraktul?)


Thanks for the reply. The following bit of info about the cause of the corruption was posted elsewhere:

In short, Telecom have a problem with their Alcatal ASAM equipment, at least in the Ponsonby exchange and are working with Alcatel to find out what's going on. If you can get you line switched over to the Nokia DSLAM then your problems will go away.

I was able to compare a large test file on a host server with what was downloaded over my faulty connection while running an Ethernet capture on both sides (sending and receiving systems), and then identify the packets in error. At the TCP/IP level, the fault shows as a low-order bit swap within a TCP data packet, followed by the reverse low-order bit swap later in the same packet (some multiple of 128 bytes later). The TCP data packet only has a simple checksum on it, and hence this type of bit-swap error is not detected at the TCP/IP level. However the lower transport levels (PPPoA and ATM AAL5) do have CRC checks on their payload which is designed to catch these types of errors. So why are they being ignored? Why aren't the underlying protocols recovering from these bit errors? And have Telecom thought to set up their own test-case and watch the actual ATM cells??


This is why the normal CRC checks are not working, and why I'm looking for some kind of double-checking downloader, like torrent or emule clients do.

Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.