How to get title from class attribute in XPath(Python/scrapy)

Question

Im working on getting the data from tripadvisor but most of the first ones are relative date and the rest are normal MM/DD/YYYY, but with closer inspection I see that relative date has this

<span class="ratingDate relativeDate" title="20 June 2015">Reviewed 4 weeks ago
</span>

I am using this Xpath to get the data

response.xpath('//div[@class="col2of2"]//span[@class="ratingDate relativeDat
e" or @class="ratingDate"]/text()').extract()

My question is How do I add the @title so that I can get the title which has the normal date format.

I tried

response.xpath('//div[@class="col2of2"]//span[@class="ratingDate relativeDat
e"/@title or @class="ratingDate"]/text()').extract()

response.xpath('//div[@class="col2of2"]//span[@class="ratingDate relativeDat
e" or @class="ratingDate"]/@title/text()').extract()

Also Forgot to mention I cannot have 2 seperate Xpaths because it is hard to format it in the pipline which prints to CSV — Smashed
– Smashed, Commented Jul 20, 2015 at 7:14
Why not? It is really easy to set the item's field to one of those XPath results. In this case the solution is transparent for your pipeline. — GHajba
– GHajba, Commented Jul 20, 2015 at 7:15
I just rialised that I can set it to the same field until the relative runs out and then the second one takes over. Thus letting me two Xpaths. But I still cant figure out how to call the title attribute — Smashed
– Smashed, Commented Jul 20, 2015 at 7:19
Figured it out, I was calling text while I shouldent have. response.xpath('//div[@class="col2of2"]//span[@class="ratingDate relativeDat e"]/@title').extract() — Smashed
– Smashed, Commented Jul 20, 2015 at 7:20

Smashed · Accepted Answer · 2015-07-20 16:17:33Z

7

Figured it out in the spider you have to do a conditional statement that will dynamically check whether that xpath contains values or not.

Here's my rendition.

item['date'] = sel.xpath('//*[@class="ratingDate relativeDate"]/@title').extract()
item['date'] += sel.xpath('//div[@class="col2of2"]//span[@class="ratingDate"]/text()').extract()

edited Jul 20, 2015 at 16:17

answered Jul 20, 2015 at 8:31

Smashed

3412 gold badges5 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to get title from class attribute in XPath(Python/scrapy)

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related