Ruby on Rails - Create Net::HTTP request from curl

Question

The curl that I have works and is very simple:

curl -d

"semester=20141Summer+2013+++++++++++++++++++++++++++++&courseid=&subject=IT++INFORMATION+TECHNOLOGY&college=&campus=1%2C2%2C3%2C4%2C5%2C6%2C7%2C9%2CA%2CB%2CC%2CI%2CL%2CM%2CN%2CP%2CQ%2CR%2CS%2CT%2CW%2CU%2CV%2CX%2CZ&courselevel=&coursenum=&startTime=0600&endTime=2359&days=ALL&All=All+Sections" http://www3.mnsu.edu/courses/selectform.asp

I'm trying to do this using Net::HTTP in ruby on rails, I have this:

uri = URI.parse("http://www3.mnsu.edu/courses/selectform.asp")
params = {"semester" => "20141Summer+2013+++++++++++++++++++++++++++++", 
          "subject" => "IT++INFORMATION+TECHNOLOGY",
          "campus" => "1%2C2%2C3%2C4%2C5%2C6%2C7%2C9%2CA%2CB%2CC%2CI%2CL%2CM%2CN%2CP%2CQ%2CR%2CS%2CT%2CW%2CU%2CV%2CX%2CZ",
          "startTime" => "0600",
          "endTime" => "2359",
          "days" => "ALL",
          "ALL" => "All+Sections"}

#=====  FORM POST  =====#
@response = Net::HTTP.post_form(uri, params)

However, everytime it's run, it returns a "Microsoft VBScript runtime error 800a0009" The curl never does that.

Any hints on what doesn't match up?

andersonvom · Accepted Answer · 2017-01-05 00:12:41Z

1

Looks like the remote server doesn't like Ruby's user agent. Two ways you can go about it:

#1 Modify Net::HTTP's user agent:

require 'net/http'
params = {"semester" => "20141Summer+2013+++++++++++++++++++++++++++++", 
    "subject" => "IT++INFORMATION+TECHNOLOGY",
    "campus" => "1%2C2%2C3%2C4%2C5%2C6%2C7%2C9%2CA%2CB%2CC%2CI%2CL%2CM%2CN%2CP%2CQ%2CR%2CS%2CT%2CW%2CU%2CV%2CX%2CZ",
    "startTime" => "0600",
    "endTime" => "2359",
    "days" => "ALL",
    "ALL" => "All+Sections"}

uri = URI.parse("http://www3.mnsu.edu/courses/selectform.asp")
url = uri.to_s + "?" + URI.encode_www_form(params)

req = Net::HTTP::Get.new(url)
req['User-Agent'] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/534.51.22 (KHTML, like Gecko) Version/5.1.1 Safari/534.51.22"

res = Net::HTTP.start(uri.hostname, uri.port) {|http| http.request(req) }

This works, but gives you a "302 Object moved" message, which you would have to follow manually.

#2 Use the Mechanize gem

require 'rubygems'
require 'mechanize'
@agent = Mechanize.new
@agent.user_agent_alias = 'Mac Safari'
page = @agent.get 'http://www3.mnsu.edu/courses/selectform.asp?semester=20141Summer+2013+++++++++++++++++++++++++++++&courseid=&subject=IT++INFORMATION+TECHNOLOGY&college=&campus=1%2C2%2C3%2C4%2C5%2C6%2C7%2C9%2CA%2CB%2CC%2CI%2CL%2CM%2CN%2CP%2CQ%2CR%2CS%2CT%2CW%2CU%2CV%2CX%2CZ&courselevel=&coursenum=&startTime=0600&endTime=2359&days=ALL&All=All+Sections'

which gets you to "MSU Class Schedule -- search for courses"

edited Jan 5, 2017 at 0:12

andersonvom

11.9k4 gold badges39 silver badges41 bronze badges

answered Apr 3, 2013 at 0:39

Arman H

5,64610 gold badges54 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

xelihope Over a year ago

Thanks for the help. But it doesn't look like your first provided code considers my post parameters at any point. That's why it returns the "Object moved" message. When I include a request.set_form_data with my parameters, I get the same runtime error back :-/

Arman H Over a year ago

I updated the code to include params with the Net::HTTP example, but the 302 redirect is still there. I can confirm it using URL-encoded GET params and wget, which tells me "302 Object moved. Location: Default.asp?blnNoYrtr=True". That redirection occurs even when you visit http://www3.mnsu.edu/courses/selectform.asp through the browser, with no params at all. This tell me the redirect is the INTENDED behavior of the site, and you have to handle the 302 yourself if you want to use Net::HTTP. Which is why I'd use Mechanize.

Collectives™ on Stack Overflow

Ruby on Rails - Create Net::HTTP request from curl

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related