From 6pm (NZ time) join us in our chat page to listen to the NZTechPodcast live recording.

Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.



69 posts

Master Geek


Topic # 62661 11-Jun-2010 16:46 Send private message

Hi Guys

A new feature in our software requires that we screen scrape the NZ post website to give the functionality that is provided by their Post Code finder. I know that there have been some people trying to do such things in past years with getting post codes etc.

So far I have had no luck with sending some data to the form and getting a response.
I have tried the screen scraping example here:
http://www.worldofasp.net/tut/WebRequest/Working_with_HttpWebRequest_and_HttpWebResponse_in_ASPNET_114.aspx

But it doesn't seem to be able to send the form inputs to the URL: http://www.nzpost.co.nz/Cultures/en-NZ/OnlineTools/PostCodeFinder/

I think I am missing something big here with regards to how screen scraping works with the technology/techniques used for the NZ post postcode finder.

Basically what I am asking is, if anyone can see why I my attempt isn't working using that example and could give me some pointers?

Or even better if anyone has successfully screen scraped this page/knows someone who has or can give me a better way of doing it I would be very thankful.

Thanks
Liam

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2Next
Just A Geek
1615 posts

Uber Geek

Trusted
Subscriber

  Reply # 340700 11-Jun-2010 16:57 Send private message

Have you permssion to do this? Why not just buy the database from NZ Post and use it (if your allowed to)

See:
http://www.nzpost.co.nz/Cultures/en-NZ/ProductsAndServices/PostcodeNetworkFileOverview/PostcodeNetworkFileOverview.htm for information on getting the databae.

Their T&C of using their website says:

Intellectual property and our other rights

We hold all copyright and other intellectual property rights in this
website. You may use the information on this website and reproduce it
(electronically or in hard copy) for your personal use only.

In all other circumstances, you must get written permission from us
before you use, send or reproduce any of the information on this
website.


BDFL
44197 posts

Uber Geek

Administrator
Trusted
Geekzone
Subscriber

  Reply # 340702 11-Jun-2010 17:01 Send private message

I very much doubt you are allowed to do this. From their Terms and Conditions:

"We hold all copyright and other intellectual property rights in this website. You may use the information on this website and reproduce it (electronically or in hard copy) for your personal use only.

In all other circumstances, you must get written permission from us before you use, send or reproduce any of the information on this website."

http://www.nzpost.co.nz/Cultures/en-NZ/Footer/TermsAndConditions






16843 posts

Uber Geek

Moderator
Trusted
Biddle Corp
Subscriber

  Reply # 340724 11-Jun-2010 17:35 Send private message

I know NZ Post went to a lot of effort to stop people doing what you want to do. If you want Postcodes for commercial use the data is available to purchase.




*Need help configuring your Linksys ATA or IP Phones for New Zealand? Check my blog post



69 posts

Master Geek


  Reply # 340748 11-Jun-2010 18:30 Send private message

I still don't understand why they have to charge $1000 a year to get post code information.

We tried contacting them about an API for their post code finder. They said that they would get back to us, but we haven't heard anything.

16843 posts

Uber Geek

Moderator
Trusted
Biddle Corp
Subscriber

  Reply # 340764 11-Jun-2010 19:22 Send private message

lanks: I still don't understand why they have to charge $1000 a year to get post code information.

We tried contacting them about an API for their post code finder. They said that they would get back to us, but we haven't heard anything.


I suspect the answer is simply "because they can".

Large companies who bulk mail need correctly addressed mail including postcodes to obtain postage discounts (or surcharges depending on how you want to look at it). $1000 sort is nothing compared to what they will save by having correct data.




*Need help configuring your Linksys ATA or IP Phones for New Zealand? Check my blog post

10100111001
3203 posts

Uber Geek

Trusted
Subscriber

  Reply # 340823 11-Jun-2010 22:52 Send private message

there are other companies who will sell you that API into NZ post data, based on the commercial licenses they have paid to NZ Post. check the nz post site for 'partners' and you can probably find one there.

We (my company) licenses the commercial version of the PAF file and provide a web service based lookup tool to our own clients which integrates into their CRM systems.




Need help implementing Microsoft CRM? Give OA Systems a shout. 
Winners of CRM Solution of the Year at the 2010 Microsoft NZ Partner Awards

3bit.com
5426 posts

Uber Geek

Moderator
Trusted
Subscriber

  Reply # 340842 12-Jun-2010 00:00 Send private message

Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.




10100111001
3203 posts

Uber Geek

Trusted
Subscriber

  Reply # 340850 12-Jun-2010 00:32 Send private message

nate: Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.


define 'free'.... last time i heard someone talk about it, it was free for personal use only.  commercial users had to purchase a license.




Need help implementing Microsoft CRM? Give OA Systems a shout. 
Winners of CRM Solution of the Year at the 2010 Microsoft NZ Partner Awards

3bit.com
5426 posts

Uber Geek

Moderator
Trusted
Subscriber

Reply # 340852 12-Jun-2010 01:03 Send private message

Regs: define 'free'.... last time i heard someone talk about it, it was free for personal use only.  commercial users had to purchase a license.


Damn you and your technicalities! 




1682 posts

Uber Geek

Trusted

  Reply # 340918 12-Jun-2010 12:04 Send private message

nate: Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.

That's a great way to get the post code data. Thanks for the tip Nate.

1682 posts

Uber Geek

Trusted

  Reply # 340931 12-Jun-2010 13:13 Send private message

nate:
Regs: define 'free'.... last time i heard someone talk about it, it was free for personal use only.? commercial users had to purchase a license.


Damn you and your technicalities!?

They don't ban the commercial use of the data as longing as there are no fees associated.



69 posts

Master Geek


  Reply # 340932 12-Jun-2010 13:13 Send private message

nate: Is the Google Maps address encoder provide the same postcodes as the NZ Post website? If so, I'd use that cause it's free.


Thanks nate, this looks like it could be useful for what I am wanting.

One last concern, if or when the NZ Post make changes to their post code system, would Google be reliable enough to update in a practical timespan?

3bit.com
5426 posts

Uber Geek

Moderator
Trusted
Subscriber

Reply # 340941 12-Jun-2010 13:53 Send private message

lanks: One last concern, if or when the NZ Post make changes to their post code system, would Google be reliable enough to update in a practical timespan?


I would say so.  Google knows everything and everyone.




10100111001
3203 posts

Uber Geek

Trusted
Subscriber

Reply # 341313 13-Jun-2010 21:07 Send private message

nate:
lanks: One last concern, if or when the NZ Post make changes to their post code system, would Google be reliable enough to update in a practical timespan?


I would say so.  Google knows everything and everyone.


Google probably knows it *before* NZ Post.....




Need help implementing Microsoft CRM? Give OA Systems a shout. 
Winners of CRM Solution of the Year at the 2010 Microsoft NZ Partner Awards

142 posts

Master Geek

Trusted
Subscriber

  Reply # 341322 13-Jun-2010 21:50 Send private message






_Allan (my blogmy tweetscompany tweetscompany web site)
           Geekzone Speedtest  -  Download: 16.28 mbps  Upload: 5.86 Mbps

 1 | 2Next
View this topic in a long page with up to 500 replies per page Create new topic





Twitter »
Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when new jobs are posted to our jobs board:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:




News »

Trending now »
Hot discussions in our forums right now:

Condenser Dryer: anyone has one?
Created by joker97, last reply by Dunnersfella on 18-Jun-2013 17:16 (30 replies)
Pages... 2


Calling all mazda 3/Axela owners in NZ
Created by coolcat21, last reply by Talkiet on 18-Jun-2013 13:20 (28 replies)
Pages... 2


Finding strongest carrier signal in a given location
Created by timmmay, last reply by timmmay on 16-Jun-2013 14:51 (16 replies)
Pages... 2


HTC One (2013) owners' discussion
Created by Dingbatt, last reply by jeffnz on 18-Jun-2013 16:00 (1812 replies)
Pages... 119 120 121


Stationary Vehicle Infringement Notice Has Wrong Details
Created by Wheelbarrow01, last reply by Dratsab on 18-Jun-2013 15:59 (30 replies)
Pages... 2


New Fibre connection is good but laptop won't connect with WPA2 security
Created by lissie, last reply by cyril7 on 17-Jun-2013 20:06 (14 replies)

Google Loon starts in New Zealand
Created by freitasm, last reply by dwl on 17-Jun-2013 13:45 (43 replies)
Pages... 2 3


Slow YouTube Response
Created by SneakerPimps, last reply by l43a2 on 18-Jun-2013 17:05 (13 replies)


Geekzone Jobs »
Most recent NZ jobs in technology:

SOE Deployment Engineer
Posted 18-Jun-2013 16:37

Service Desk Consultant
Posted 18-Jun-2013 15:37

Technical Assistants - Masterton
Posted 18-Jun-2013 15:37

Senior Business Analyst/Consultant | Client Facing
Posted 18-Jun-2013 15:37

IT Systems Technician
Posted 18-Jun-2013 15:37

Service Desk Analyst | Contract
Posted 18-Jun-2013 14:37

Web Developer - Christchurch
Posted 18-Jun-2013 14:37


Geekzone Live »
Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.