How does facebook screen scrape?

Forums › IT Pro and developers › How does facebook screen scrape?

jbard

1377 posts

Uber Geek
+1 received by user: 17

#65240 31-Jul-2010 17:22

Hi

I would like to place a utility on my site which lets people share links, similar to facebook.
Basically users would type in a URL and the scrapper would grap the title of the page, and maybe the first few lines of the main article. I assume this would not break copyright rules as facebook does this?

anyway does anyone have any idea or any experience in this area. Any pointers would be great?
Are their any techniques i could search for on google to help me out, or maybe a tutorial?

thanks

hellonearthisman

1819 posts

Uber Geek
+1 received by user: 52

Trusted

#360728 31-Jul-2010 17:47

One think to keep an eye on is the use of short urls that use redirection to go to the site referenced.
It sounds like a pretty easy thing to write in php.

Can't think of any tutorial sites that could help ya though.

edwinprakash

77 posts

Master Geek
+1 received by user: 3

ID Verified

#360730 31-Jul-2010 17:50

Have a look at this page... http://wiki.developers.facebook.com/index.php/Facebook_Share/Specifying_Meta_Tags

This'll give you an idea of what facebook looks for when scraping a page.

The techniques for doing the actual scraping involves making a web request from code and parsing the response for things like the facebook meta tags or the more generic HTML tags and elements.

How you do this depends on the programming language you're using.

Just my 2c, hope this helps...

------------------------------------------
Ed

jbard

1377 posts

Uber Geek
+1 received by user: 17

#360734 31-Jul-2010 17:54

cheers guys, i will be using coldfusion and i have managed to find a few links on the web for tutorials, will post them here when i have checked them out for future use.