Using R to perform a web query and return URL's

Question

I am fairly new to R, and have never used it for writing a web query before. I was wondering if there were any preexisting packages out there that would fit my needs. I am trying to search for a company and return the URL of their website. I have the company name, address, and phone number. Is there a way to run a program that will check the website against the information I have to confirm it's the correct website?

Have a look at RCurl, that could be a good option for you. — Paul Hiemstra
– Paul Hiemstra, Commented May 6, 2013 at 20:01
And httr: github.com/hadley/httr. It provides a more user friendly wrapper on top of RCurl. — hadley
– hadley, Commented May 7, 2013 at 12:42

Thomas · Accepted Answer · 2013-05-06 20:07:40Z

5

Can't guarantee that this will work every time but definitely use the RCurl package

library(RCurl)
geturlname <- function(name){
    h = getCurlHandle()
    z <- getURL(paste0("http://google.com/search?btnI=1&q=",name), # google i'm feeling lucky
        followlocation=TRUE, curl=h)
    getCurlInfo(h)$effective.url # catch the url redirect
}
geturlname("Apple")
geturlname("Google")
geturlname("Blockbuster")

answered May 6, 2013 at 20:07

Thomas

44.7k12 gold badges115 silver badges144 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

hadley · Accepted Answer · 2013-05-07 12:44:08Z

4

Thomas' function is a little easier to write with httr because it:

automatically manages handles for you
automatically follows redirects
returns an object that represents the results of the request

And here's the function:

library(httr)
geturlname <- function(name){
  url <- paste0("http://google.com/search?btnI=1&q=", name)
  GET(url)$url
}
geturlname("Apple")
geturlname("Google")
geturlname("Blockbuster")

answered May 7, 2013 at 12:44

hadley

104k35 gold badges186 silver badges248 bronze badges

Collectives™ on Stack Overflow

Using R to perform a web query and return URL's

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related