Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.
Buying anything on Amazon? Please use the Geekzone Amazon aff link.




72 posts

Master Geek


Topic # 62661 11-Jun-2010 16:46 Send private message

Hi Guys

A new feature in our software requires that we screen scrape the NZ post website to give the functionality that is provided by their Post Code finder. I know that there have been some people trying to do such things in past years with getting post codes etc.

So far I have had no luck with sending some data to the form and getting a response.
I have tried the screen scraping example here:
http://www.worldofasp.net/tut/WebRequest/Working_with_HttpWebRequest_and_HttpWebResponse_in_ASPNET_114.aspx

But it doesn't seem to be able to send the form inputs to the URL: http://www.nzpost.co.nz/Cultures/en-NZ/OnlineTools/PostCodeFinder/

I think I am missing something big here with regards to how screen scraping works with the technology/techniques used for the NZ post postcode finder.

Basically what I am asking is, if anyone can see why I my attempt isn't working using that example and could give me some pointers?

Or even better if anyone has successfully screen scraped this page/knows someone who has or can give me a better way of doing it I would be very thankful.

Thanks
Liam

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2
Just A Geek
1946 posts

Uber Geek
+1 received by user: 313

Trusted
Subscriber

  Reply # 340700 11-Jun-2010 16:57 Send private message

Have you permssion to do this? Why not just buy the database from NZ Post and use it (if your allowed to)

See:
http://www.nzpost.co.nz/Cultures/en-NZ/ProductsAndServices/PostcodeNetworkFileOverview/PostcodeNetworkFileOverview.htm for information on getting the databae.

Their T&C of using their website says:

Intellectual property and our other rights

We hold all copyright and other intellectual property rights in this
website. You may use the information on this website and reproduce it
(electronically or in hard copy) for your personal use only.

In all other circumstances, you must get written permission from us
before you use, send or reproduce any of the information on this
website.


BDFL
50374 posts

Uber Geek
+1 received by user: 4866

Administrator
Trusted
Geekzone
Subscriber

  Reply # 340702 11-Jun-2010 17:01 Send private message

I very much doubt you are allowed to do this. From their Terms and Conditions:

"We hold all copyright and other intellectual property rights in this website. You may use the information on this website and reproduce it (electronically or in hard copy) for your personal use only.

In all other circumstances, you must get written permission from us before you use, send or reproduce any of the information on this website."

http://www.nzpost.co.nz/Cultures/en-NZ/Footer/TermsAndConditions






20209 posts

Uber Geek
+1 received by user: 1774

Moderator
Trusted
Biddle Corp
Subscriber

  Reply # 340724 11-Jun-2010 17:35 Send private message

I know NZ Post went to a lot of effort to stop people doing what you want to do. If you want Postcodes for commercial use the data is available to purchase.



72 posts

Master Geek


  Reply # 340748 11-Jun-2010 18:30 Send private message

I still don't understand why they have to charge $1000 a year to get post code information.

We tried contacting them about an API for their post code finder. They said that they would get back to us, but we haven't heard anything.

20209 posts

Uber Geek
+1 received by user: 1774

Moderator
Trusted
Biddle Corp
Subscriber

  Reply # 340764 11-Jun-2010 19:22 Send private message

lanks: I still don't understand why they have to charge $1000 a year to get post code information.

We tried contacting them about an API for their post code finder. They said that they would get back to us, but we haven't heard anything.


I suspect the answer is simply "because they can".

Large companies who bulk mail need correctly addressed mail including postcodes to obtain postage discounts (or surcharges depending on how you want to look at it). $1000 sort is nothing compared to what they will save by having correct data.

Infrastructure Geek
3769 posts

Uber Geek
+1 received by user: 112

Trusted
Microsoft NZ
Subscriber

  Reply # 340823 11-Jun-2010 22:52 Send private message

there are other companies who will sell you that API into NZ post data, based on the commercial licenses they have paid to NZ Post. check the nz post site for 'partners' and you can probably find one there.

We (my company) licenses the commercial version of the PAF file and provide a web service based lookup tool to our own clients which integrates into their CRM systems.




Technical Evangelist
Microsoft NZ
about.me/nzregs
Twitter: @nzregs


I'm @nate
5906 posts

Uber Geek
+1 received by user: 205

Moderator
Trusted
Subscriber

  Reply # 340842 12-Jun-2010 00:00 Send private message

Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.




Infrastructure Geek
3769 posts

Uber Geek
+1 received by user: 112

Trusted
Microsoft NZ
Subscriber

  Reply # 340850 12-Jun-2010 00:32 Send private message

nate: Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.


define 'free'.... last time i heard someone talk about it, it was free for personal use only.  commercial users had to purchase a license.




Technical Evangelist
Microsoft NZ
about.me/nzregs
Twitter: @nzregs


I'm @nate
5906 posts

Uber Geek
+1 received by user: 205

Moderator
Trusted
Subscriber

Reply # 340852 12-Jun-2010 01:03 Send private message

Regs: define 'free'.... last time i heard someone talk about it, it was free for personal use only.  commercial users had to purchase a license.


Damn you and your technicalities! 




1768 posts

Uber Geek
+1 received by user: 45

Trusted

  Reply # 340918 12-Jun-2010 12:04 Send private message

nate: Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.

That's a great way to get the post code data. Thanks for the tip Nate.

1768 posts

Uber Geek
+1 received by user: 45

Trusted

  Reply # 340931 12-Jun-2010 13:13 Send private message

nate:
Regs: define 'free'.... last time i heard someone talk about it, it was free for personal use only.? commercial users had to purchase a license.


Damn you and your technicalities!?

They don't ban the commercial use of the data as longing as there are no fees associated.



72 posts

Master Geek


  Reply # 340932 12-Jun-2010 13:13 Send private message

nate: Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.


Thanks nate, this looks like it could be useful for what I am wanting.

One last concern, if or when the NZ Post make changes to their post code system, would Google be reliable enough to update in a practical timespan?

I'm @nate
5906 posts

Uber Geek
+1 received by user: 205

Moderator
Trusted
Subscriber

Reply # 340941 12-Jun-2010 13:53 Send private message

lanks: One last concern, if or when the NZ Post make changes to their post code system, would Google be reliable enough to update in a practical timespan?


I would say so.  Google knows everything and everyone.




Infrastructure Geek
3769 posts

Uber Geek
+1 received by user: 112

Trusted
Microsoft NZ
Subscriber

Reply # 341313 13-Jun-2010 21:07 Send private message

nate:
lanks: One last concern, if or when the NZ Post make changes to their post code system, would Google be reliable enough to update in a practical timespan?


I would say so.  Google knows everything and everyone.


Google probably knows it *before* NZ Post.....




Technical Evangelist
Microsoft NZ
about.me/nzregs
Twitter: @nzregs


152 posts

Master Geek
+1 received by user: 1

Trusted
Subscriber

  Reply # 341322 13-Jun-2010 21:50 Send private message






_Allan (my blogmy tweetscompany tweetscompany web site)
           Geekzone Speedtest  -  Download: 16.28 mbps  Upload: 5.86 Mbps

 1 | 2
View this topic in a long page with up to 500 replies per page Create new topic




Twitter »
Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





Trending now »

Hot discussions in our forums right now:

Police Camera Van Disguise
Created by Reanalyse, last reply by lNomNoml on 21-Dec-2014 23:33 (75 replies)
Pages... 3 4 5


Do I have the right to return this?
Created by corksta, last reply by kiwibro111 on 21-Dec-2014 23:54 (45 replies)
Pages... 2 3


Slaughter of Innocents
Created by networkn, last reply by networkn on 19-Dec-2014 17:46 (64 replies)
Pages... 3 4 5


youtube downloader
Created by Ford, last reply by jarledb on 22-Dec-2014 16:57 (18 replies)
Pages... 2


What has Geekzone led you to buy?
Created by russelo, last reply by jarledb on 22-Dec-2014 16:41 (18 replies)
Pages... 2


Spray Foam Insulation
Created by AACTech, last reply by timbosan on 19-Dec-2014 16:58 (36 replies)
Pages... 2 3


Crew Drinking on Flights - Why!?
Created by networkn, last reply by Geektastic on 22-Dec-2014 09:35 (34 replies)
Pages... 2 3


Some lowlife is using my easy to remember number to commit idiocy
Created by joker97, last reply by joker97 on 22-Dec-2014 15:48 (15 replies)


Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.