Big issue with Telecom DNS caches for a web developer

Forums › Spark New Zealand (including Skinny and BigPipe) › Big issue with Telecom DNS caches for a web developer

asianbro

47 posts

Geek

#74750 6-Jan-2011 18:23

I'm having a problem awfully like what's described here: http://www.geekzone.co.nz/forums.asp?forumid=39&topicid=66599

Same error message, local hosts entries, and I can ping but not access via browser

I developed a website, then upload it to my US based testing server, created DNS entry for it to point to that server. Because I need to check it right away, I don't want to wait for the 48 hours or so until the new DNS fully populated. So I add it to my etc/hosts file so I can access the website on the test server in US straight away.

The error is:

Network Error (dns_unresolved_hostname)

Your requested host "xyz" could not be resolved by DNS.
For assistance, contact your network support team.

I spent all day trying to figure out what's happening, but I really have to calm myself down after making a call to Telecom 'technical' support.

Telecom 'technical' support kept thinking I'm a web developer who is trying to get web development support from Telecom. And because I can access all other websites, the attitude I'm getting is that this isn't Telecom's issue.

Can anyone please tell me how I can work around this issue?

1 | 2 | 3

1623 posts

Uber Geek

Trusted

#424739 6-Jan-2011 18:30

asianbro: I'm having a problem awfully like what's described here: http://www.geekzone.co.nz/forums.asp?forumid=39&topicid=66599

Same error message, local hosts entries, and I can ping but not access via browser

I developed a website, then upload it to my US based testing server, created DNS entry for it to point to that server. Because I need to check it right away, I don't want to wait for the 48 hours or so until the new DNS fully populated. So I add it to my etc/hosts file so I can access the website on the test server in US straight away.

The error is:

Network Error (dns_unresolved_hostname)

Your requested host "xyz" could not be resolved by DNS.
For assistance, contact your network support team.

I spent all day trying to figure out what's happening, but I really have to calm myself down after making a call to Telecom 'technical' support.

Telecom 'technical' support kept thinking I'm a web developer who is trying to get web development support from Telecom. And because I can access all other websites, the attitude I'm getting is that this isn't Telecom's issue.

Can anyone please tell me how I can work around this issue?

What is the website? I'll see if I can access it from where I am on TelstraClear.

"When the people are being beaten with a stick, they are not much happier if it is called 'the People's Stick'"

freitasm

BDFL - Memuneh
79250 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#424752 6-Jan-2011 19:00

Accessing from other ISPs is not a problem. The problem is that having the entry in the hosts file make your PC resolve the address, but when the request goes out the proxy can't see it so you get this error.

You could make the domain a default in your server and browse by IP instead. Still would have problems with any resources where the full URL is used though.

Or try http://www.cloudshare.com and get free account to create a virtual machine and test from there instead.

asianbro

47 posts

Geek

#424754 6-Jan-2011 19:06

I have removed my local entries but still the same error. I guess without local entries I just have to wait until the DNS is populated?

BlakJak

1275 posts

Uber Geek

Trusted

#424756 6-Jan-2011 19:15

Have you considered using a dns entry with a deliberately low ttl - say 60 seconds - at least while testing ?

No signature to see here, move along...

asianbro

47 posts

Geek

#424758 6-Jan-2011 19:22

Hi BlakJak,

How would that make a difference? Wouldn't it still take around 24-48 hours for a new DNS to populate?

asianbro

47 posts

Geek

#424761 6-Jan-2011 19:28

I should say, it could take 24 hours.

Anyway, Telecom has picked up the DNS now. So all good for now.

I think Telecom really need to brand the error and a link to an article explaining what's going on. It would've saved me hours of time and frustration.

BlakJak

1275 posts

Uber Geek

Trusted

#424771 6-Jan-2011 20:40

asianbro: Hi BlakJak,

How would that make a difference? Wouldn't it still take around 24-48 hours for a new DNS to populate?

As a long time ISP geek I find it frustrating when webby types make blanket statements about things they don't actually know about, so please bear with me.

Dns records have a ttl - a time to live. That is, the maximum age of a record that a dns server should cache into memory before it ditches it and goes and looks up the record again. TTL can be set by zone (eg every dns record in the zone has the same TTL) or an individual record can have it's own specified TTL.

If your TTL is for example 8hrs, and I look up your record via my own dns server, my server will cache the answer in memory and for further lookups against my dns server (from me or anyone else) the caches answer will be served - so even if you've made a change since, I won't see it until my caching resolver expires what it has in cache and goes back to your dns server for the record again.

If I know in advance that I will be making changes - say a planned transition of a website from one server to another - I will drop the TTL on the record to something nice and low - say 1 minute - well in advance. This means that the max time where folks out there will be pointed to the wrong place, is only 60s long.

Operating on a low TTL increases the load on your authoritative dns servers, but it does also give you the means to make rapid changes. Sites like trademe dothis in order for them to be able to move load between servers quickly.

If your ISP (the ones to whom you connect) allow the caching resolvers that they operate (ie the dns servers against which you issue requests when surfing the web) to ignore the TTL and hold records for longer than the TTL you have legit cause for complaint. Otherwise dnswill operate as designed, and cache records so that lookup load is shared amongsb clients and the authoritative servers (where the domain is hosted) won't grind to a halt under the strain.

"24-48 hours" is someones attempt to come up wig a generic timeframe within which all changes should reasonnably be seen, this is because 48hrs is typically the longest TTL you will see. The zones I host on my personal DNS have default TTLs between 1 hour and 12 hours.

So create a valid dns entry for the site you're working on but pull the TTL down while in test mode. When satisfied, up the TTL. Job done . (sorry for crusty typing - on my iPhone - what a mission...)

No signature to see here, move along...

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

asianbro

47 posts

Geek

#424808 6-Jan-2011 22:33

I did not make any blanket statement. It was genuine question.

I kind of understand what you're saying, but not sure if it is applicable to my scenario. I'll read more about DNS and TTL, thanks for the pointer.

It still stinks that the error isn't branded to cause confusion and wasting time. Also that we can't use local host entries.

freitasm

BDFL - Memuneh
79250 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#424823 6-Jan-2011 23:15

Put it simply: a well configured DNS will follow the instructions the domain owner puts in the DNS records. The TTL is determined by the domain administrator when configuring the records.

AFAIK Telecom's DNS respect the TTL they find in the records.

asianbro

47 posts

Geek

#424833 6-Jan-2011 23:34

So if I set the TTL of mywebsite.com to 1 minute ... then if I add a new subdomain say subdomain1.mywebsite.com, will it get picked up in 1 minute?

freitasm

BDFL - Memuneh
79250 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#424834 6-Jan-2011 23:37

asianbro: So if I set the TTL of mywebsite.com to 1 minute ... then if I add a new subdomain say subdomain1.mywebsite.com, will it get picked up in 1 minute?

No. If you set TTL of mywebsite.com to 60 seconds it means your DNS will cache the current IP address for 60 seconds, and then the next time you request the address it will contact the authoritative DNS to get the IP address again.

If you add a new subdomain, providing the authoritative server is updated fast then any request will automatically get the new address, since the subdomain didn't exist in any cache before.

Zeon

3916 posts

Uber Geek

Trusted

#424840 6-Jan-2011 23:55

asianbro: So if I set the TTL of mywebsite.com to 1 minute ... then if I add a new subdomain say subdomain1.mywebsite.com, will it get picked up in 1 minute?

If its a new record and never been queries (and thus cached) before it will work straight away.

As said before, set the TTL really low. While it may have an impact on a production site with more DNS traffic it will be fine for development.

You really should never change your hosts file.

Speedtest 2019-10-14

Regs

4066 posts

Uber Geek

Trusted

Snowflake

#424842 6-Jan-2011 23:56

asianbro: I did not make any blanket statement. It was genuine question.

I kind of understand what you're saying, but not sure if it is applicable to my scenario. I'll read more about DNS and TTL, thanks for the pointer.

It still stinks that the error isn't branded to cause confusion and wasting time. Also that we can't use local host entries.

first, this is not really a "DNS Cache issue". The problem appears to be that you are being served http via a proxy, and it doesnt see the dns changes yet because you didnt wait long enough to test it (or you didnt manipulate ttls a couple of days ago).

As for the 'cant use localhost entries' bit - if you buy your own commercial internet connection with dedicated bandwidth then it is probably an option to bypass the proxy. if you want to stick to a contested residential service, then you have to put up with it and its limitations.

The error is:

Network Error (dns_unresolved_hostname)

Your requested host "xyz" could not be resolved by DNS.
For assistance, contact your network support team.

This is pretty much exactly the error. the only thing they could potentially add to make it any clearer is a statement that the error occurred at the proxy server.

as a web developer, you should really learn about DNS TTLs and how to manipulate them. If you ever move a website from one server to another its really something you want to understand as it can really speed up the transition. In addition, you could always use an HTTP trace tool which will likely have shown you that the error was coming from their proxy server. something like this: http://www.httpwatch.com/download/

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

Regs

4066 posts

Uber Geek

Trusted

Snowflake

#424844 7-Jan-2011 00:00

asianbro: So if I set the TTL of mywebsite.com to 1 minute ... then if I add a new subdomain say subdomain1.mywebsite.com, will it get picked up in 1 minute?

if the TTL was 24 hours before you changed it to 1 minute, then it may be up to 24 hours before all the DNS servers get the new TTL of 60 seconds. so any changes you make will only start turning over in 60 seconds after the existing cached versions expire.

You might also find that some DNS servers will drop TTL to a minimum of 60 minutes, 4 hours etc. in those cases theres not much you can do

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

asianbro

47 posts

Geek

#424849 7-Jan-2011 00:25

I understand what TTL is ... I thought there's something I didn't know about it when someone here said TTL is my solution, that's why I said I'll read more

Yes new DNS is picked up quick, but I find it's not quick enough. I use local host entry because sometime I need to use my sub domain as soon as I create it (to set it up and test it right away).

With local host entry, it's instant and this had been working fine for a long time until it hit me today.

I wasn't aware of this new cache thing Telecom implemented, so it took me a while to figure out. It would've been less time wasted if there're more clue on the error message.

Local hosts is also useful for quick temporary fix. If a computer somehow can not access a website, sometime it will work with local host entry. Not a good fix, but if you have to use the website, then it will at least allow things to work while proper fix is underway.

1 | 2 | 3