Getting specific part of <div> from webpage

Question

I am currently using aiohttp and lxml to scrape webpages and return values. So far, I have

def get_sr(page, tree): 
    sr = tree.xpath(".//div[@class='competitive-rank']/div/text()")[0]
    return sr 
def get_icon_url(page, tree):
    url = tree.xpath('.//img[@class="player-portrait"]/@src')[0]
    return url
def get_sr_icon_url(page, tree):
    url = tree.xpath('.//div[@class="competitive-rank"]/img/@src')[0]
    return url
def get_level(page, tree): 
    level = tree.xpath('.//div[@class="header-avatar"]/text()')[0]
    return level

The first 3 functions work perfectly, and yet the final function will not correctly get the text I am looking for. This:

<div class="header-avatar">
                        <img src="https://blzgdapipro-a.akamaihd.net/game/unlocks/0x0250000000001150.png" width="80" height="80">
                        <span>369</span>
                    </div>

Is the code block I am trying to get the number from. Currently, the number is 369 but it constantly changes. I have confirmed that the page and tree are correct through print statements, so instead it's an issue w/ the actual get_level method itself.

Help? Other pieces of code needed to determine issue?

Thank you for the help.

Tiny.D · Accepted Answer · 2017-05-26 00:45:49Z

1

Try this:

level = tree.xpath('.//div[@class="header-avatar"]/span/text()')[0]

answered May 26, 2017 at 0:45

Tiny.D

6,5562 gold badges18 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Getting specific part of <div> from webpage

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related