How to use Python variable in an XPath expression?

Question

I am trying to get all the followers name from this website https://www.quora.com/profile/Karan-Bansal-3/followers

Since the whole page is not loaded at once, I am using this everytime in a loop :

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Now since I can't select all the element at once, I am trying to use indexing to find the element in the loop.

people = driver.find_element_by_xpath("//div[@class='pagedlist_item'][i]/*/div[@class='ObjectCard-header']/a[@class='user']")

Here as you can see, I am trying to give the indexing using [i] which clearly doesn't work and in place of it, if I give [1] or any number it works well. So how can I select the element one by one.

Code snippet :

i=1
target = open(filename,'w')
driver.get('https://www.quora.com/profile/Karan-Bansal-3/followers')
while True:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
people = driver.find_element_by_xpath("//div[@class='pagedlist_item'][i]/*/div[@class='ObjectCard-header']/a[@class='user']")
target.write(people.text)
target.write("\n")
i = i+1

@derloopkat I have shown that i is 1 initially and I know XPATH is not taking the variable [i], so how can it be selected then because I need all the elements and it can only be done in a loop which will increment the value and so will the index. — Arkham
– Arkham, Commented May 19, 2016 at 12:39

Community · Accepted Answer · 2017-05-23 11:44:05Z

2

Use string concatenation in the hosting language so that i is evaluated before constructing the XPath. Otherwise, [i] is a predicate testing for the presence of an i element. You didn't state what your hosting language is, but assuming string concatenation is "string" + "string":

 "//div[@class='pagedlist_item'][" + i + "]/*/div[@class='ObjectCard-header']/a[@class='user']"

See also: How to pass variable parameter into XPath expression?

Update: Ok, so you're hosting XPath in Python.

You can use + to concatenate above if you first cast i to a string via str(i),

 "//div[@class='pagedlist_item'][" + str(i) + "]/*/div[@class='ObjectCard-header']/a[@class='user']"

or you can use format() as is used in the link I provided:

 "//div[@class='pagedlist_item'][{}]/*/div[@class='ObjectCard-header']/a[@class='user']".format(i)

either way, place the above constructed XPath expressions into your call to find_element_by_xpath() and your problem should be solved.

Caution: Do not use this approach with untrusted values for i or you could open your code to XPath injection attacks.

edited May 23, 2017 at 11:44

CommunityBot

11 silver badge

answered May 19, 2016 at 12:41

kjhughes

113k31 gold badges198 silver badges276 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Daniel Manta Over a year ago

this is quite basic, if you're using Phyton then check how to do a for in range.

kjhughes Over a year ago

Updated answer for Python for you.

Sajan Kaundal · Accepted Answer · 2018-07-17 09:49:11Z

1

Firstly cast index/i variable to string using str(index) and then try using below:

content = rows.xpath('//div[@class="LookupHelpDesc"]['+index+']//text()').extract_first()

always use single quote.

answered Jul 17, 2018 at 9:49

Sajan Kaundal

414 bronze badges

Comments

Arkham · Accepted Answer · 2016-05-19 15:54:45Z

0

The solution is to first convert the index to a string.

index = str(i)
people = driver.find_element_by_xpath("//div[@class='pagedlist_item'][" + index + "]/*/div[@class='ObjectCard-header']/a[@class='user']")
i++

answered May 19, 2016 at 15:54

Arkham

691 silver badge7 bronze badges

Collectives™ on Stack Overflow

How to use Python variable in an XPath expression?

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related