AttributeError - webscraping - Python - Selenium

Question

I need scrape the following table from the web and I cannot solve the issue with "find_all" function. The PyCharm always says:

AttributeError: 'NoneType' object has no attribute 'find_all'

I dont know what is wrong. Trying it with table.find_all("tr") or table.find_all('tr') characters and with next attributes like table.find_all("tr", attrs={"class": "table table-export"}) and next options and nothing works. Please could you tell me what I doing wrong?

Table:

<div class="table-options">
    <table class="table table-export">
                <thead>
                <tr>
                    <!-- ngIf: ActuallyPoints && ActuallyPoints.name == 'AXB' --><th ng-if="currentRole &amp;&amp; currentRole.name == 'AXB'" class="id check">
                        <label ng-click="selectAll()"><input disabled="" id="select-all" type="checkbox" ng-model="all" class="valid value-ng">All</label>
                    </th><!-- end ngIf: currentRole && currentRole.name == 'AXB' -->
                    <th>AAA</th>
                    <th>BBB</th>
                    <th>CCC</th>
        </tr>
                </thead>
                <tbody>
<!-- ngRepeat: x in ErrorStatus --><tr ng-repeat="x in ErrorStatus" class="random-id">
                    <!-- ngIf: currentRole && currentRole.name == 'AXB' --><td ng-if="currentRole &amp;&amp; currentRole.name == 'AXB'" class="random-id">
                        <input type="checkbox" ng-model="x.checked" ng-change="selectOne(x)" class="valid value-ng">
                    </td><!-- end ngIf: currentRole && currentRole.name == 'AXB' -->
                    <td class="pax">111</td>
                    <td class="pax">222</td>
                    <td class="pax">333</td>
                    </td>
                </tr><!-- end ngRepeat: x in ErrorStatus -->
                </tbody>
            </table>
        </div>

Code:

import lxml
from urllib.request import urlopen
from bs4 import BeautifulSoup

url = 'xxx'
website = request.urlopen(url).read()

soup = BeautifulSoup(website, "lxml")

table = soup.find("table", attrs={"class": "table table-export"})
rows = table.find_all('tr')

Many thanks.

what is the url that your trying to scrape.. pls add it in your code — mnm
– mnm, Commented May 18, 2020 at 13:17
I am sorry but this is a private url with confidential database and unfortunately I cannot share it with everyone and the login is neccessary.. — Sinda
– Sinda, Commented May 18, 2020 at 14:05
then I'm afraid you'll get little to no help in here.. if others can't replicate the problem in real time, then its not worth solving. — mnm
– mnm, Commented May 18, 2020 at 23:25

maliarov · Accepted Answer · 2020-05-20 15:49:55Z

I won't be able to provide a solution, since there is no link, but explanation to the error is quite simple:

AttributeError: 'NoneType' object has no attribute 'find_all'

Let's see where you use .find_all in you code:

rows = table.find_all('tr')

Considering what interpreter says, this piece of code actually looks like:

rows = None.find_all('tr')

In other words, your variable table equals None. Therefore, your problem is here:

table = soup.find("table", attrs={"class": "table table-export"}) # returns None

In human language, you were trying to find some table within your html and then store it to variable table, but soup.find() wasn't able to find the element, using the instructions you gave, and therefore returned None. You didn't notice it and tried to call None.find_all(), but None doesn't have this method.

That's why you receive this error. If you can't share the link, please re-check this piece by yourself, since it's not working:

table = soup.find("table", attrs={"class": "table table-export"}) # returns None

UPD: First of all, try printing variable soup and check if the table is present, because html, which you see at browser, and html, which you receive by request, might be quite different:

soup = BeautifulSoup(website, "lxml")
print(soup)

Collectives™ on Stack Overflow

AttributeError - webscraping - Python - Selenium

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related