Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.




190 posts

Master Geek


#96667 1-Feb-2012 23:04
Send private message

I thought I would recount what I have just wasted 5 hours over the last 3 days on for amusement and perhaps to save someone else the same grief...

I am shifting web hosts. One of my sites I duplicated on the new server but wanted to test before pointing DNS to the new server so I added an entry to my hosts file. Unfortunately this did not work and I spent a lot of time running in circles putting test files on both servers to easily identify them, stuffing this up due to my ftp client following the hosts directive etc etc etc.

The problem I was having was that no matter what I did, visiting the site always resulted in the old (live) server responding. I even was able to see the new server's IP in a firefox add-on that displays the IP of the current site being visited, or with wget. I also could ping the url and get responses from the correct IP. Only my web browsers appeared to be misbehaving. However this was true on multiple computers running Windows 7, Mac Os Lion and Ubuntu 10.10. Eventually I found that if I tunneled my internet through my VPS or tethered my cellphone, the problem went away. A friend finally tipped me off to the possibility of a transparent proxy and a quick tweak to enable SSL, soon proved this to be true.

Some searches (with the benefit of hindsight) show that this proxy has been in place for some time (and there was a lot of talk about it in 2010) but I was not aware of it until now. Pity about the 5 hours but I guess I've learned something new today. Embarassed  Part of the reason that it never occurred to me is that I had figured a proxy would just cache whatever IP I visited, not override my requested IP by performing its own DNS query on the referrer!! Although I am (clearly) not very experienced with proxies, this really seems to me to be a very very bad way of implementing one. Damn you Telstraclear!!!   

Create new topic
28831 posts

Uber Geek

Moderator
Trusted
Biddle Corp
Lifetime subscriber

  #576149 2-Feb-2012 06:17
Send private message

TCL have had a transparent proxy for ~10 years now and what you experienced is the standard way a transparent proxy works, you'll get the exact same issues on many other ISP's who also use transparent proxys.






2584 posts

Uber Geek

Mod Emeritus
Trusted
Lifetime subscriber

  #576152 2-Feb-2012 06:47
Send private message

sbiddle: TCL have had a transparent proxy for ~10 years now and what you experienced is the standard way a transparent proxy works, you'll get the exact same issues on many other ISP's who also use transparent proxys.


So just how many ISP's use transparent proxy's? I was looking at Unblock-us the other week which is a DNS based unblocking service and they only have a handful of ISP's listed using them globally.

http://support.unblock-us.com/customer/portal/articles/291561







Media centre PC - Case Silverstone LC16M with 2 X 80mm AcoustiFan DustPROOF, MOBO Gigabyte MA785GT-UD3H, CPU AMD X2 240 under volted, RAM 4 Gig DDR3 1033, HDD 120Gig System/512Gig data, Tuners 2 X Hauppauge HVR-3000, 1 X HVR-2200, Video Palit GT 220, Sound Realtek 886A HD (onboard), Optical LiteOn DH-401S Blue-ray using TotalMedia Theatre Power Corsair VX Series, 450W ATX PSU OS Windows 7 x64

 
 
 
 


677 posts

Ultimate Geek

Trusted

  #576182 2-Feb-2012 08:56
Send private message

Nety:
sbiddle: TCL have had a transparent proxy for ~10 years now and what you experienced is the standard way a transparent proxy works, you'll get the exact same issues on many other ISP's who also use transparent proxys.


So just how many ISP's use transparent proxy's? I was looking at Unblock-us the other week which is a DNS based unblocking service and they only have a handful of ISP's listed using them globally.

http://support.unblock-us.com/customer/portal/articles/291561


Transparent proxy cache is very common in ISP's. Not all are doing HTTP, some are specifically for P2P though, just depends on what the ISP sees as providing the most value.

Issues like the one seen here are very common also when you dont use the DNS servers of your ISP, you can have random broken internet issues with things like YouTube, Akamai and any other CDN as well as hosts file having no effect...

You can in many instances ask the ISP to not put your connection through the proxy, not sure if that is true for TCL though.






meat popsicle

129 posts

Master Geek

Trusted

  #576305 2-Feb-2012 13:47
Send private message

Another question, did the site/pages have cache control statements embedded to ensure the transparent proxy knows when to refresh the content or discard the cached copy?  HTTP meta tags etc.

I might be guilty of assuming those statements were absent, if so, I retract and withdraw... :)

a quick search turned up this useful (at initial glance) site.
[url=http://www.mnot.net/cache_docs/#CONTROL]





"Customers don’t expect you to be perfect. They do expect you to fix things when they go wrong." Donald Porter – British Airways

The views expressed here are my own and are not reflective of other organisms or organisations.

677 posts

Ultimate Geek

Trusted

  #576309 2-Feb-2012 13:53
Send private message

In the OP's case that will only stop stale content from the current prod server, he would have still been getting responses from the old server until DNS is changed to point to the new IP.





meat popsicle

8035 posts

Uber Geek

Trusted

  #576381 2-Feb-2012 17:23
Send private message

If the web host has a ssh server supports a SSH connection you can bypass the transparent proxy via ssh tunnel to the server and your local dns entry should work.

Just need a client like Putty for windows to send your web browser request over the ssh tunnel.

1631 posts

Uber Geek


  #576384 2-Feb-2012 17:39
Send private message

Nety:
sbiddle: TCL have had a transparent proxy for ~10 years now and what you experienced is the standard way a transparent proxy works, you'll get the exact same issues on many other ISP's who also use transparent proxys.


So just how many ISP's use transparent proxy's? I was looking at Unblock-us the other week which is a DNS based unblocking service and they only have a handful of ISP's listed using them globally.

http://support.unblock-us.com/customer/portal/articles/291561


I use unblock-us.com on Telecom, and it works 100% for Hulu and Netflix.

 
 
 
 


677 posts

Ultimate Geek

Trusted

  #576513 2-Feb-2012 21:15
Send private message

Ragnor: If the web host has a ssh server supports a SSH connection you can bypass the transparent proxy via ssh tunnel to the server and your local dns entry should work.

Just need a client like Putty for windows to send your web browser request over the ssh tunnel.


I always forget to mention this.
If you use the ssh option -D1040 or any port you like you can set the socks proxy configuration in your browser to use localhost and the port number after the -D then you dont need to browse to localhost to test the remote site.
With Putty you just specify a dynamic proxy port, it can be a simple way to test things.




meat popsicle



190 posts

Master Geek


  #576529 2-Feb-2012 21:29
Send private message

ptinson: In the OP's case that will only stop stale content from the current prod server, he would have still been getting responses from the old server until DNS is changed to point to the new IP. 



Correct. I wanted to do some testing before updating my DNS for the rest of the world. As for what the cache controls were, I haven't the foggiest as I've never used them.

 

RmACK: Eventually I found that if I tunneled my internet through my VPS or tethered my cellphone, the problem went away.

ragnor & ptinson, by tunneling through my VPS, I did indeed mean that I used ssh as a socks proxy

677 posts

Ultimate Geek

Trusted

  #576537 2-Feb-2012 21:37
Send private message

Ah i missed that bit, so i told you how to suck an egg really. Good for me.




meat popsicle



190 posts

Master Geek


  #576538 2-Feb-2012 21:39
Send private message

No worries :)

15804 posts

Uber Geek

Trusted
Subscriber

  #576679 3-Feb-2012 09:48
Send private message

A VPN would get around this. I use Astrill.

Create new topic



Twitter and LinkedIn »



Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





News »

Intel introduces 10th Gen Intel Core H-series for mobile devices
Posted 2-Apr-2020 21:09


COVID-19: new charitable initiative to fund remote monitoring for at-risk patients
Posted 2-Apr-2020 11:07


Huawei introduces the P40 Series of Android-based smartphones
Posted 31-Mar-2020 17:03


Samsung Galaxy Z Flip now available for pre-order in New Zealand
Posted 31-Mar-2020 16:39


New online learning platform for kids stuck at home during COVID-19 lockdown
Posted 26-Mar-2020 21:35


New 5G Nokia smartphone unveiled as portfolio expands
Posted 26-Mar-2020 17:11


D-Link ANZ launches wireless AC1200 4G LTE router
Posted 26-Mar-2020 16:32


Ring introduces two new video doorbells and new pre-roll technology
Posted 17-Mar-2020 16:59


OPPO uncovers flagship Find X2 Pro smartphone
Posted 17-Mar-2020 16:54


D-Link COVR-2202 mesh Wi-Fi system now protected by McAfee
Posted 17-Mar-2020 16:00


Spark Sport opens its platform up to all New Zealanders at no charge
Posted 17-Mar-2020 10:04


Spark launches 5G Starter Fund
Posted 8-Mar-2020 19:19


TRENDnet launches high-performance WiFi Mesh Router System
Posted 5-Mar-2020 08:48


Sony boosts full-frame lens line-up with introduction of FE 20mm F1.8 G large-aperture ultra-wide-angle prime Lens
Posted 5-Mar-2020 08:44


Vector and Spark teamed up on smart metering initiative
Posted 5-Mar-2020 08:42



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.


Support Geekzone »

Our community of supporters help make Geekzone possible. Click the button below to join them.

Support Geezone on PressPatron



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.