[TelstraClear] SOLVED - Telstra Proxy Cache Frustration

Forums › One New Zealand (including Vodafone, WxC, Farmside) › [TelstraClear] SOLVED - Telstra Proxy Cache Frustration

1 | 2

jamgol

42 posts

Geek

#274177 18-Nov-2009 11:38

No updates to the exe's yesterday.

The changes to the news site happened on the 27th as indicated. On the 29th the cache was still giving me old versions. The reason I remember this example is because I had dinner at my parents house on Wednesday night (28th), they have Telecom ADSL, I had my laptop with me so checked it and had the new version perfectly.
Got home, same laptop, had another look .... bang ... old version with old images.
Fired up my proxy, same laptop, did not clear history this time .... refresh ... bang .. new version.

I've had examples like this every week for the last two years, being aware of it makes it almost not a problem. The only reason I've started looking into it again is because downloading large files through the proxy can be slow and I hate to waste all that beautiful warp speed !

I generally use Chrome (Dev branch) and IE8 for testing.

I'm told that business customers with Telstra have options here (2nd hand information, not verified). I would happily pay for a business service but I'm also told that it's "impossible" to deliver business services to a residential address.

chiefie

I iz your trusted friend
5877 posts

Uber Geek

Retired Mod

Trusted

Lifetime subscriber

#274195 18-Nov-2009 12:34

I never had issue downloading huge file on my TCL connection, either just plain IE/browser download, or via download manager.

Internet is my backyard...

«Geekzone blog: Tech 'n Chips Takeaway» «Personal blog: And then...»

Please read the Geekzone's FUG

michaeln

238 posts

Master Geek

Trusted

#274196 18-Nov-2009 12:35

Can you let us know when a change has happened so the headers can be looked at?

jamgol

42 posts

Geek

#274200 18-Nov-2009 12:44

Yes, certainly. I'll be interested to see myself.

I would expect an update tonight (although I expected an update last night as well !)

Michaeln, do you work for Telstra ?
I'm told that there is a gateway IP I might be able to use for testing that will bypass the proxy. Again, this is second hand information.

michaeln

238 posts

Master Geek

Trusted

#274204 18-Nov-2009 12:59

jamgol:

I'm told that there is a gateway IP I might be able to use for testing that will bypass the proxy. Again, this is second hand information.

Not that I know of. The caches are essentially the last thing before the links to other carriers, so everything goes through them.

michaeln

238 posts

Master Geek

Trusted

#274425 19-Nov-2009 10:24

The caches are at the boundary with other network providers, so everything passes
through them. There is no way to put a customer in a cache bypass list. However,
it is possible to put a web site into a bypass list.

Caches make a HUGE improvement to customer experience. We are about 150ms away (RTT)
from the West Coast of the USA and further again from the UK.
Most of the world still uses Win/XP, which ships with a default RWIN of 16kB.
The delay product limits the maximum bandwidth to RWIN/RTT, which is about 835kbps to the
West Coast. Now, you can just set RWIN to 64kB and get 3.34Mbps, or better yet 256kB and
get 13.4Mbps---but that requires that the customer changes a registry setting.
With caches the latency reduces to about 20ms and the bandwidth for a standard XP setup
rises to about 7Mbps.

Linux, Mac OS X and Vista/7 don't have this problem, but reducing latency is still a huge win.

Most servers are not cache hostile. There are some, but they are rare. All modern browsers
also keep local caches, and many enterprises have their own caches. The vast majority of sites
work fine.

The caches are set up so that they estimate when an object (page, IMG, CSS,
dowloaded fille, etc.) is likely to expire, based on the headers (Expires:, etc.)
and when it last DID change. They will check several times within that
window. That means that popular pages and pages that change frequently will
be checked more frequently.

If the cache sees a 'no-cache' in the request header, it will check
immediately. That means that if you force your browser to reload the page
(usually by holding down Shift when you hit reload or similar. Safari does
it any time you explicitly reload) the cache will check and bring in the latest versions.

That should be fine for everything except links to files that are just going to be
downloaded.

Under some circumstances however, the caches can have a rather long estimate of when they should
next check for staleness.

Note that the cache doesn't expire objects from the cache just because the indicated
expiry date has passed. It only deletes stuff when it needs to because it needs the
space, so the most likely scenario is that it will check an expired object,
find the ETag has not changed, and serve up the local copy.

If you want to force an update---for yourself---then appending "." to the site
name will make it look like a 'new' site. Of course, this only works once :)

jamgol

42 posts

Geek

#274444 19-Nov-2009 11:40

Brilliant, thanks.

Knowing what and why will make it infinitely less annoying when it happens, and I now have a suite of tricks to try thanks to you.

Thanks again.

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

chiefie

I iz your trusted friend
5877 posts

Uber Geek

Retired Mod

Trusted

Lifetime subscriber

#274469 19-Nov-2009 13:10

So.... you said this is solved... and with interests, I like to know how it was solved, for educational purposes.

Internet is my backyard...

«Geekzone blog: Tech 'n Chips Takeaway» «Personal blog: And then...»

Please read the Geekzone's FUG

jamgol

42 posts

Geek

#274498 19-Nov-2009 14:46

Hi Chiefie,

Biggest thing for me was learning how the Telstra cache works, what it's looking for and what to expect .... knowledge is power !

Telstra made some changes yesterday to tune things and alleviate the problem, only time will tell how successful that is.

One of our servers has been configured with the "no-cache" suggestion which works perfectly. Another has had the expire times changed.
These servers, although pubiclally hosted, are used to host internal Intranets and do not require ISP provided caching (the users go through an internal proxy and cache).

We've also changing the way we upload new executables to ensure no cache problems with any ISP, not just Telstra.

The dot "." idea is also gold ... had never thought to try that. Fools the cache perfectly.

Thanks

chiefie

I iz your trusted friend
5877 posts

Uber Geek

Retired Mod

Trusted

Lifetime subscriber

#274524 19-Nov-2009 16:29

Very cool. the dot ".", is that to put at the end of the URL? For example "http://www.domain.com/folder/file.exe."?

I must say, the dot is new to me too.

Internet is my backyard...

«Geekzone blog: Tech 'n Chips Takeaway» «Personal blog: And then...»

Please read the Geekzone's FUG

jamgol

42 posts

Geek

#274527 19-Nov-2009 16:36

Yes, it appears to work at the end of any url ..... very handy.

1 | 2