I use try except to avoid error,but My terminal still show error but not the log message :
raise ValueError('Missing scheme in request url: %s' % self._url)
exceptions.ValueError: Missing scheme in request url:
How can I avoid this error when scrapy didn't get image_urls?
Please guide me ,thank you very much.
try:
item['image_urls'] = ["".join(image.extract()) ]
except:
log.msg("no image foung!. url={}".format(response.url),level=log.INFO)
the image_urls field should be a list, not a str.
item['image_urls'] = image.extract()
If you do so, and still raise the exception, it seems that the urls you scraped is the relative path.
the ImagePipeline doesn't know your host, so it can not generate the absolute path to crawl.
import urlparse
item['image_urls'] = [ urlparse.urljoin(response.url, u) for u in image.extract() ]
from urllib.parse import urljoin and ...[urljoin(..."Missing scheme in request url" means that you are missing the "http://" part of the url, most probably because your URL's are relative and should be absolute.
