Can a website legally stop you from scraping it?

Forums › IT Pro and developers › Can a website legally stop you from scraping it?

quickt

8 posts

Wannabe Geek

#64776 22-Jul-2010 15:38

I'm not sure where to post this but this forum seems good enough.

Simple question really, if a particular website lists in its terms & condition that you can not scrape/crawl it etc then legally what kind they do if you decide to scrape it anyway?

I could manually collect data and complete this task anyhow but obviously manually doing it is very inefficient.? Obviously the site can 'ban' you etc.? But are you breaking any laws by breaking such terms and conditions?

I'm looking for the? legal implications, whether it's moral and/or ethical and so on is up for debate.

To clarify New Zealand website, server and laws.

Be interested to see if anyone has any information to offer,

Thanks.

EDIT:

Just ahd a quick read of the NZ postcode thread, no that's not the website :) but didn't see anything about the legal ramifications.

1 | 2

BDFL - Memuneh
80652 posts

Uber Geek
+1 received by user: 41045

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#354962 22-Jul-2010 15:41

If the website claims copyright on its contents, then yes you are breaking the law by copying it, by any form or means. Copyright is protected under New Zealand laws.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

quickt

8 posts

Wannabe Geek

#354963 22-Jul-2010 15:43

I see so your breaking basic copyright laws. Have there been any cases in New Zealand about people/business scraping someone elses data for commercial use?

quickt

8 posts

Wannabe Geek

#354974 22-Jul-2010 15:56

I see a case overseas with Ryanair where a price comparison site was ordered to stop scraping Ryanair's price data.

I also see trademe has gone after a few developers to:

Various software developers have received legal threats after developing third-party software which interfaces with the Trade Me website.

On August 19, 2006 the New Zealand Listener published an article, "Bidding War" [25] on one such developer. The developer, Ciaran Riddell, created a piece of software, AuctionBar, which used a technique known as screen scraping. The software allowed for more detailed searches for goods on sale as well as bids and updates via text-messaging and a tool known as a "sniper", which acted as an automated bidding-tool. (wikipedia)

I guess my idea is not viable, I'd still like to read up on any cases in New Zealand just for interests sake if anybody knows of any.

freitasm

BDFL - Memuneh
80652 posts

Uber Geek
+1 received by user: 41045

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#354982 22-Jul-2010 16:02

Try scrapping data from SkyTV guide and see how fast you get a legal letter.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

Handsomedan

7769 posts

Uber Geek
+1 received by user: 7402

ID Verified

Trusted

Subscriber

#354985 22-Jul-2010 16:06

Ah the old EPG debacle... ;)

Handsome Dan Has Spoken.
Handsome Dan needs to stop adding three dots to every sentence...

Handsome Dan does not currently have a side hustle as the mascot for Yale

*Gladly accepting donations...

quickt

8 posts

Wannabe Geek

#354990 22-Jul-2010 16:12

Haha sounds like somebody's been there before eh :)

Anyhow this is an article on the ryanair case, it's a tad interesting:

http://www.lexology.com/library/detail.aspx?g=83babc8a-0c69-473f-a3c6-12b352dfa257

Onto the next idea I guess..

Mighty Ape

Shop now at Mighty Ape (affiliate link).

hellonearthisman

1819 posts

Uber Geek
+1 received by user: 52

Trusted

#355001 22-Jul-2010 16:29

If your idea's good then ask the people running the site that you want to scrap them.
They may give you permission, they may say on or they could say yes but for a price.

freitasm

BDFL - Memuneh
80652 posts

Uber Geek
+1 received by user: 41045

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#355004 22-Jul-2010 16:33

Another options APIs. Some services may not allow scrap, but might an API that allow developers to access content for their software?

Obviously having an API doesn't mean you can use it - you have to receive authorisation, most of the times in form of an API key.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

graemeh

2080 posts

Uber Geek
+1 received by user: 226

#355006 22-Jul-2010 16:34

quickt: Haha sounds like somebody's been there before eh :)

Anyhow this is an article on the ryanair case, it's a tad interesting:

http://www.lexology.com/library/detail.aspx?g=83babc8a-0c69-473f-a3c6-12b352dfa257

Onto the next idea I guess..

Can't read it, looks like the site needs a login.

Can you scape it for us and paste it here?

Just kidding

hellonearthisman

1819 posts

Uber Geek
+1 received by user: 52

Trusted

#355035 22-Jul-2010 17:30

Google cached version http://webcache.googleusercontent.com/search?q=cache:Ks_3qWhw6CsJ:www.lexology.com/library/detail.as...

nate

6473 posts

Uber Geek
+1 received by user: 458

Retired Mod

Trusted

Lifetime subscriber

#355097 22-Jul-2010 20:36

freitasm: Try scraping data from SkyTV guide and see how fast you get a legal letter.

If you were using it for your own personal use, I doubt they'd be able to catch you.

If you (stupidly) republish/sell it, then you are fair game.

/just sayin.

HP

Shop now for HP laptops and other devices (affiliate link).

graemeh

2080 posts

Uber Geek
+1 received by user: 226

#355198 23-Jul-2010 07:58

hellonearthisman: Google cached version http://webcache.googleusercontent.com/search?q=cache:Ks_3qWhw6CsJ:www.lexology.com/library/detail.as...

Thanks. Now why didn't I think of that?

mattk

255 posts

Ultimate Geek
+1 received by user: 1

#355206 23-Jul-2010 08:26

http://dizzythinks.net/2009/06/i-stole-your-images-put-them-back-or-i.html

Phaedrus

74 posts

Master Geek

#355238 23-Jul-2010 09:42

I could be completely wrong but in reading between the lines I wonder if the OP was interested in doing something with a certain NZ site...

... if so I'd merely make the comment that if that site made things easier for a buyer then there would be a need for a scraper, but they don't. So, depending really on what you're wabnting to do, I'd take note of what Nate said. Assuming that is that you're just wanting to do something like see only what you want to see, and how you want to see it, as opposed to trying to mine data and use it for other purposes - that I would have an issue with I think.

P.

quickt

8 posts

Wannabe Geek

#355311 23-Jul-2010 12:30

I've read up on the EPG debacle now, I see what you mean.

Lets just say 'hypothetically" of course. The website was a certain NZ auction site that does not offer an API so it makes it incredibly difficult to 'legally' use information that they 'own' to further your own business and hence increase theirs. But because reasons unknown (like the real reasons) they feel the need to 'protect' this side of their business is beyond me.

I'm not a programmer as such. I know enough about what's possible, I was just going to freelance the job. It would have been relatively simple and just a small tool to help me increase my business, the program would not have been resold as it's probably worthless to anyone else anyway.

But big boys will be big boys.

Just as I was going to post this I think I may have just come up with the solution which is so simple. Guess you got to take the blinkers off to see the full picture eh, I'll give you a hint starts with E and it has 4 letters.

1 | 2