Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.




761 posts

Ultimate Geek

Trusted

#89298 30-Aug-2011 14:32
Send private message

Afternoon all,

I need to scrape information off the 2degrees website, namely the your 2degrees tables that store balances (Voice, Data, Texts). I tried googling it and almost all guides are old and of no use to me.

Some context. This is to run on a WP7 app built in Visual Studios Express for WP7 using VB code. (also http://htmlagilitypack.codeplex.com/)

Thanks for your time

Chris

Table appears to be called "tableBillAmount"




Professional Forum Lurker


Create new topic
1819 posts

Uber Geek

Trusted

  #513988 30-Aug-2011 16:09
Send private message

What is the url to the page you want to scrap from?

Is it on an account settings page, that you have to login to see?

The link below would help with working out how to login the a website via vb.

Login to Website Using VB.NET - with Example Code for Facebook | Hassan Ahmad




761 posts

Ultimate Geek

Trusted

  #513989 30-Aug-2011 16:21
Send private message

I need to scrape from https://secure.2degreesmobile.co.nz/web/ip/login

Which needs your phone number and password, obviously im going to have to retrieve and save those from a textbox the user can access. 

From here i need the program to access the website and return the information i require, currently we are trying to figure out how to scrape that information, then we need to program the app.

EDIT

Turns out that WP7 does not support some of the methods in HttpWebResponse specifically the GetResponse and GetRequestStream methods that code needs to function. Apparently the phone only supports Asynchronous methods but i have no idea what the hell this means

Source: http://social.msdn.microsoft.com/Forums/en/windowsphone7series/thread/f3d24bee-e6a3-41e9-a3b1-c940ee0ddb21

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx
 




Professional Forum Lurker


 
 
 
 


1819 posts

Uber Geek

Trusted

  #514052 30-Aug-2011 19:49
Send private message

Here are some links that may help. I'm into PHP so not much help with VB WP7 really.
I would do the scrapping in php using a web server.
I would send requests from the WP7 to the server and the server would do all the scrapping and just send back the data you need. Would help save ya mobile data use and data crunching on ya phone.

anyway...

Stackover flow might be a resource worth reading.
http://stackoverflow.com/search?q=wp7+vb+https&submit=search

http://stackoverflow.com/questions/906543/using-httpwebrequest-in-asp-net-vb
http://stackoverflow.com/questions/4324835/secure-authenticated-interaction-from-a-wp7-app
http://stackoverflow.com/questions/3457894/retrieve-xml-from-https-using-webclient-httpwebrequest-wp...


http://social.msdn.microsoft.com/Forums/en-US/windowsphone7series/thread/594e1422-3b69-4cd2-a09b-fb5...

http://www.silverlightshow.net/items/Using-WebClient-and-HttpWebRequest.aspx

Hope that helps ya in the right direction.



761 posts

Ultimate Geek

Trusted

  #514058 30-Aug-2011 19:56
Send private message

Thanks we'll check it out




Professional Forum Lurker


6389 posts

Uber Geek

Moderator
Trusted
Lifetime subscriber

  #514075 30-Aug-2011 20:32
Send private message

Surely you could ask 2degrees to expose this data? It would help other like minded people write apps that would help promote their services?

1463 posts

Uber Geek


  #514078 30-Aug-2011 20:48
Send private message

I might suggest asking 2Degrees before you pull information off of their website. If you look at the bottom left corner of their web site you might notice a small disclaimer around their rights.

Also I think if you read their site terms there is a small section that explains what you can or cant use.




Software Engineer
   (the practice of real science, engineering and management)




761 posts

Ultimate Geek

Trusted

  #514079 30-Aug-2011 20:51
Send private message

I have asked but with all their technical issues currently i would doubt it would go ahead. I'm considering asking to make my app officially endorsed and get special access for it. 

I've already released v1.0 to the NZ marketplace, it's going through the check process. It allows a graphical interface for adding and removing value packs.

Version 2.0 uses a fancy Panorama GUI (like the Facebook app), allows value pack management, has an integrated webbrowser for account management, has an FAQ, has mobile settings, and now i am trying to add balance information which will (hopefully) end up on a live tile display.

VFNZ has an unofficial app on there for $2 and my V2 app drags it off and laughs at it :D Best of all it's currently free (student developer account = no developer fee) 




Professional Forum Lurker


 
 
 
 


1819 posts

Uber Geek

Trusted

  #514134 30-Aug-2011 22:29
Send private message

TwoSeven: I might suggest asking 2Degrees before you pull information off of their website. If you look at the bottom left corner of their web site you might notice a small disclaimer around their rights.

Also I think if you read their site terms there is a small section that explains what you can or cant use.

I think there would not be a problem when scraping your user account information. It is yours after all.



761 posts

Ultimate Geek

Trusted

  #514142 30-Aug-2011 22:44
Send private message

There are existing apps that do this already




Professional Forum Lurker


Create new topic




News »

Freeview On Demand app launches on Sony Android TVs
Posted 6-Aug-2020 13:35


UFB hits more than one million connections
Posted 6-Aug-2020 09:42


D-Link A/NZ extends COVR Wi-Fi EasyMesh System series with new three-pack
Posted 4-Aug-2020 15:01


New Zealand software Rfider tracks coffee from Colombia all the way to New Zealand businesses
Posted 3-Aug-2020 10:35


Logitech G launches Pro X Wireless gaming headset
Posted 3-Aug-2020 10:21


Sony Alpha 7S III provides supreme imaging performance
Posted 3-Aug-2020 10:11


Sony introduces first CFexpress Type A memory card
Posted 3-Aug-2020 10:05


Marsello acquires Goody consolidating online and in-store marketing position
Posted 30-Jul-2020 16:26


Fonterra first major customer for Microsoft's New Zealand datacentre
Posted 30-Jul-2020 08:07


Everything we learnt at the IBM Cloud Forum 2020
Posted 29-Jul-2020 14:45


Dropbox launches native HelloSign workflow and data residency in Australia
Posted 29-Jul-2020 12:48


Spark launches 5G in Palmerston North
Posted 29-Jul-2020 09:50


Lenovo brings speed and smarter features to new 5G mobile gaming phone
Posted 28-Jul-2020 22:00


Withings raises $60 million to enable bridge between patients and healthcare
Posted 28-Jul-2020 21:51


QNAP integrates Catalyst Cloud Object Storage into Hybrid Backup solution
Posted 28-Jul-2020 21:40



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.


Support Geekzone »

Our community of supporters help make Geekzone possible. Click the button below to join them.

Support Geezone on PressPatron



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.