I would like to check where a single URL redirects. An example of that could be a link from Google's search result page (where a click always goes through Google server).
Can I do that with curl?
Add a comment
|
5 Answers
There is an even easier way
curl -w "%{url_effective}\n" -I -L -s -S $URL -o /dev/null
it would print
http://raspberrypi.stackexchange.com/questions/1508/how-do-i-access-the-distributions-name-on-the-command-line/1521
for URL
http://raspberrypi.stackexchange.com/a/1521/86
-
4That uses more time and bandwidth though, since you're downloading the second page as well.unhammer– unhammer2015-05-06 12:30:09 +00:00Commented May 6, 2015 at 12:30
-
2@unhammer You are right, updated my answer to do head requests only.ismail– ismail2015-08-07 08:11:26 +00:00Commented Aug 7, 2015 at 8:11
-
1This has an issue where on websites with a lot of redirecting this won't work, it'll exit with "too many redirects". How do i make it STOP on the first one?TheTechRobo– TheTechRobo2021-05-27 19:28:03 +00:00Commented May 27, 2021 at 19:28
-
1Update: figured it out thanks to unix.stackexchange.com/a/515645/401349. i need to use redirect_url rather than url_effective and programmatically remove the error message (it's being used in a script). then i add
--max-redirs 1(I am archiving shortURLs).TheTechRobo– TheTechRobo2021-05-27 20:02:19 +00:00Commented May 27, 2021 at 20:02
Try this:
$ LOCATION=`curl -I http://raspberrypi.stackexchange.com/a/1521/86 | perl -n -e '/^Location: (.*)$/ && print "$1\n"'`
$ echo "$LOCATION"
/questions/1508/how-do-i-access-the-distributions-name-on-the-command-line/1521#1521
Google Redirects
Google redirect URLs are slightly different. They return a Javascript redirect, which could easily be processed, but why not process the original URL and for go curl all together?
$ URL="http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CFAQFjAA&url=http%3A%2F%2Fwww.raspberrypi.org%2F&ei=rv8oUODIIMvKswa4xoHQAg&usg=AFQjCNEBMoebclm0Gk0LCZIStJbF04U1cQ"
$ LOCATION=`echo "$URL" | perl -n -e '/url=([a-zA-Z0-9%\.]*)/ && print "$1\n"'`
$ echo "$LOCATION"
http%3A%2F%2Fwww.raspberrypi.org%2F
$ echo "$LOCATION" | perl -pe 's/%([0-9a-f]{2})/sprintf("%s", pack("H2",$1))/eig'
http://www.raspberrypi.org/
Reference
-
How about special URLs with special characters (like Google's redirects), e.g.: google.com/…syntagma– syntagma2012-08-13 13:24:05 +00:00Commented Aug 13, 2012 at 13:24
-
@REACHUS The redirect is encoded into the query string of the URL given.Alex Chamberlain– Alex Chamberlain2012-08-13 13:28:53 +00:00Commented Aug 13, 2012 at 13:28
-
@REACHUS No worries - you can probably combine the 2 perl statements.Alex Chamberlain– Alex Chamberlain2012-08-13 14:12:03 +00:00Commented Aug 13, 2012 at 14:12
-
If there are multiple redirects, this will get you the first one only! This will only show you the location, not check if that is a redirect too, not check if URL is available. You may be better off writing a script with PHP or any other language with a good HTTP request library or using other methods which check the target URL and which follow multiple redirects.Sybille Peters– Sybille Peters2021-08-04 11:14:29 +00:00Commented Aug 4, 2021 at 11:14
-
Add --no-progress-meter to curl to turn off the progress meter.Sybille Peters– Sybille Peters2021-08-04 11:14:49 +00:00Commented Aug 4, 2021 at 11:14
curl can be configured to follow redirects and to print variables after completion. So what you ask can be achieved with the following command:
curl -Ls -w %{url_effective} -o /dev/null https://google.com
The man page explains the necessary parameters like that:
-L, --location Follow redirects (H)
-s, --silent Silent mode (don't output anything)
-w, --write-out FORMAT Use output FORMAT after completion
-o, --output FILE Write to FILE instead of stdout
or try this
curl -s -o /dev/null -I -w "HTTP_CODE: %{http_code}\nREDIRECT_URL: %{redirect_url}\n" http://raspberrypi.stackexchange.com/a/1521/86
-
How about special URLs with special characters (like Google's redirects), e.g.: google.com/…syntagma– syntagma2012-08-13 13:25:00 +00:00Commented Aug 13, 2012 at 13:25
-
just put the url in single quotes, so that your shell ignores the special characters in the link. But the link you have stated does not redirect to another uri, the status code of the response is 200 and not 3xx. The uri you look for is hidden in the uri itself respectively in the content of the response. For further examination you can study the header of the response with
curl -s -I 'http://yoururl'and the content of the response withcurl -s 'http://yoururl'(you will see that google use a simple javascript for the redirection).user1146332– user11463322012-08-13 14:30:18 +00:00Commented Aug 13, 2012 at 14:30
The parameters -L (--location) and -I (--head) still doing unnecessary HEAD-request to the location-url.
If you are sure that you will have no more than one redirect, it is better to disable follow location and use a curl-variable %{redirect_url}.
This code do only one HEAD-request to the specified URL and takes redirect_url from location-header:
curl --head --silent --write-out "%{redirect_url}\n" --output /dev/null "https://www.youtube.com/watch?v=tVfWBlBd7LE"
