3

How do I parse a table within a table for a simple web scraping application?

My process is:

  • Create soup object of url
  • find all tags with
  • find tags within the already found tables
  • find the rows
  • find the headers
  • find the data in the table

HTML code:

<table border="0" cellpadding="2" cellspacing="1" width="660">
    <tbody><tr bgcolor="#CCCCCC">
        <th class="IntWxHeader">Time</th>
        <th class="IntWxHeader">Weather</th>
        <th class="IntWxHeader">Temperature</th>
        <th class="IntWxHeader">Dewpoint</th>
        <th class="IntWxHeader">Humidity</th>
        <th class="IntWxHeader">Pressure</th>
        <th class="IntWxHeader">Winds</th>
        <th class="IntWxHeader">Visibility</th>
    </tr>
<tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:11AM</span></td><td align="center"><span style="font-size: 16px;">Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">12:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Cloudy</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE&nbsp;5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">1:55AM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">S&nbsp;5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">2:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">3:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;6 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">4:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;8 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">5:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">80%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WSW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:07AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:29AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:55AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">7:55AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SE&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">8:09AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">ESE&nbsp;5 MPH</span></td><td align="center"><span style="font-size: 16px;">5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;9 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">36°F/2°C</span></td><td align="center"><span style="font-size: 16px;">27°F/-3°C</span></td><td align="center"><span style="font-size: 16px;">70%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;6 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">46%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;10 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">48°F/9°C</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">34%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;9 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">52°F/11°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">17%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SW&nbsp;8 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">1:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">14%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">2:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">7°F/-14°C</span></td><td align="center"><span style="font-size: 16px;">13%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;6 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">3:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">11%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NNW&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">4:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">59°F/15°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;12 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">5:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">-6°F/-21°C</span></td><td align="center"><span style="font-size: 16px;">7%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W&nbsp;20 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">-2°F/-19°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;15 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">7:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">46°F/8°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">18%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">45°F/7°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">23%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">41°F/5°C</span></td><td align="center"><span style="font-size: 16px;">12°F/-11°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;9 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">14°F/-10°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;10 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">34°F/1°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">65%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N&nbsp;21 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr></tbody></table>

I want to grab all of the data in the last line (

Here is what I have so far:

  try:
    resp = urlopen(url)
  except URLError as e:
    print "An error occured fetching %s \n %s" %  (url, e.reason)
    return 1
  soup = BeautifulSoup(resp.read(), 'lxml')

  # get outer table
  try:
    outerTables = soup.findAll('table')
  except AttributeError as e:
    print "No tables found, exciting"
    return 1

  # get inner table
  try:
    for table in outerTables:
      innerTables = soup.findAll('table')
  except AttributeError as e:
    print "No inner tables found"
    return 1

  # get rows
  try:
    for table in innerTables:
      rows = soup.findAll('tr')
  except AttributeError as e:
    print "No rows found"
    return 1

  # get headers
  headers = []
  try:
    for row in rows:
      markup = soup.findAll('th')
  except AttributeError as e:
    print "No headers found"
    return 1
  for i in range(0,len(markup)):
    headers.append(markup[i].string)

Am I on the right path here?

Improve this question

1 Answer 1

Reset to default
1

Your html has one table with headers which are the th tags, some rows which are the tr tags and inside the rows is the data you want inside the td tags:

h = """<table border="0" cellpadding="2" cellspacing="1" width="660">
      <tr bgcolor="#CCCCCC">
        <th class="IntWxHeader">Time</th>
        <th class="IntWxHeader">Weather</th>
        <th class="IntWxHeader">Temperature</th>
        <th class="IntWxHeader">Dewpoint</th>
        <th class="IntWxHeader">Humidity</th>
        <th class="IntWxHeader">Pressure</th>
        <th class="IntWxHeader">Winds</th>
        <th class="IntWxHeader">Visibility</th>
    </tr>
<tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:11AM</span></td><td align="center"><span style="font-size: 16px;">Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">12:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Cloudy</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE&nbsp;5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">1:55AM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">S&nbsp;5 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">2:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSE&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">3:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;6 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">4:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">19°F/-7°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;8 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">5:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">80%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WSW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:07AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">86%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:29AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">6:55AM</span></td><td align="center"><span style="font-size: 16px;">Freezing Fog</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">100%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">0.25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">7:55AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SE&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">8:09AM</span></td><td align="center"><span style="font-size: 16px;">Mist and Fog</span></td><td align="center"><span style="font-size: 16px;">18°F/-8°C</span></td><td align="center"><span style="font-size: 16px;">16°F/-9°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">ESE&nbsp;5 MPH</span></td><td align="center"><span style="font-size: 16px;">5mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">25°F/-4°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">93%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;9 MPH</span></td><td align="center"><span style="font-size: 16px;">7mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">36°F/2°C</span></td><td align="center"><span style="font-size: 16px;">27°F/-3°C</span></td><td align="center"><span style="font-size: 16px;">70%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;6 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">46%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;10 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55AM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">48°F/9°C</span></td><td align="center"><span style="font-size: 16px;">21°F/-6°C</span></td><td align="center"><span style="font-size: 16px;">34%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SSW&nbsp;9 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">12:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">52°F/11°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">17%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">SW&nbsp;8 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">1:55PM</span></td><td align="center"><span style="font-size: 16px;">Partly Cloudy</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">14%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N&nbsp;3 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">2:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">7°F/-14°C</span></td><td align="center"><span style="font-size: 16px;">13%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;6 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">3:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">11%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NNW&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">4:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">59°F/15°C</span></td><td align="center"><span style="font-size: 16px;">3°F/-16°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;12 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">5:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">57°F/14°C</span></td><td align="center"><span style="font-size: 16px;">-6°F/-21°C</span></td><td align="center"><span style="font-size: 16px;">7%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W&nbsp;20 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">6:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">54°F/12°C</span></td><td align="center"><span style="font-size: 16px;">-2°F/-19°C</span></td><td align="center"><span style="font-size: 16px;">10%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;15 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">7:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">46°F/8°C</span></td><td align="center"><span style="font-size: 16px;">5°F/-15°C</span></td><td align="center"><span style="font-size: 16px;">18%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">70mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">8:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">45°F/7°C</span></td><td align="center"><span style="font-size: 16px;">9°F/-13°C</span></td><td align="center"><span style="font-size: 16px;">23%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">W&nbsp;7 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">9:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">41°F/5°C</span></td><td align="center"><span style="font-size: 16px;">12°F/-11°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">NW&nbsp;9 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#E6EFFF;"><td align="center"><span style="font-size: 16px;">10:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">43°F/6°C</span></td><td align="center"><span style="font-size: 16px;">14°F/-10°C</span></td><td align="center"><span style="font-size: 16px;">31%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">WNW&nbsp;10 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr><tr style="line-height:20pt; background-color:#F8FBFF;"><td align="center"><span style="font-size: 16px;">11:55PM</span></td><td align="center"><span style="font-size: 16px;">Mostly Clear</span></td><td align="center"><span style="font-size: 16px;">34°F/1°C</span></td><td align="center"><span style="font-size: 16px;">23°F/-5°C</span></td><td align="center"><span style="font-size: 16px;">65%</span></td><td align="center"><span style="font-size: 16px;">0.00</span></td><td align="center"><span style="font-size: 16px;">N&nbsp;21 MPH</span></td><td align="center"><span style="font-size: 16px;">25mi.</span></td></tr>
/table>"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(h)
table = soup.find("table")
# get all the headers/columns
headers = [th.text for th in table.find_all("th")]
print(", ".join(headers))
# select all rows in the table, after the first as the first includes the th tags which we already have.
rows = table.select("tr + tr")
# iterate over all the trs/rows
for row in rows:
   # pull the text from each td
    print(", ".join([td.text for td in row.find_all("td"))

Which would give you:

Time, Weather, Temperature, Dewpoint, Humidity, Pressure, Winds, Visibility
12:11AM, Cloudy, 25°F/-4°C, 25°F/-4°C, 100%, 0.00, SSE 7 MPH, 7mi.
12:55AM, Mostly Cloudy, 23°F/-5°C, 23°F/-5°C, 100%, 0.00, SSE 5 MPH, 7mi.
1:55AM, Partly Cloudy, 25°F/-4°C, 23°F/-5°C, 93%, 0.00, S 5 MPH, 7mi.
2:55AM, Mostly Clear, 19°F/-7°C, 19°F/-7°C, 100%, 0.00, SSE 7 MPH, 7mi.
3:55AM, Mostly Clear, 23°F/-5°C, 19°F/-7°C, 86%, 0.00, SSW 6 MPH, 7mi.
4:55AM, Mostly Clear, 23°F/-5°C, 19°F/-7°C, 86%, 0.00, SSW 8 MPH, 7mi.
5:55AM, Mostly Clear, 23°F/-5°C, 18°F/-8°C, 80%, 0.00, WSW 3 MPH, 7mi.
6:07AM, Freezing Fog, 21°F/-6°C, 18°F/-8°C, 86%, 0.00, NW 3 MPH, 0.5mi.
6:29AM, Freezing Fog, 16°F/-9°C, 16°F/-9°C, 100%, 0.00, NW 3 MPH, 0.25mi.
6:55AM, Freezing Fog, 16°F/-9°C, 16°F/-9°C, 100%, 0.00, WNW 3 MPH, 0.25mi.
7:55AM, Mist and Fog, 18°F/-8°C, 16°F/-9°C, 93%, 0.00, SE 3 MPH, mi.
8:09AM, Mist and Fog, 18°F/-8°C, 16°F/-9°C, 93%, 0.00, ESE 5 MPH, 5mi.
8:55AM, Mostly Clear, 25°F/-4°C, 23°F/-5°C, 93%, 0.00, SSW 9 MPH, 7mi.
9:55AM, Mostly Clear, 36°F/2°C, 27°F/-3°C, 70%, 0.00, SSW 6 MPH, 25mi.
10:55AM, Mostly Clear, 43°F/6°C, 23°F/-5°C, 46%, 0.00, SSW 10 MPH, 70mi.
11:55AM, Mostly Clear, 48°F/9°C, 21°F/-6°C, 34%, 0.00, SSW 9 MPH, 70mi.
12:55PM, Partly Cloudy, 52°F/11°C, 9°F/-13°C, 17%, 0.00, SW 8 MPH, 70mi.
1:55PM, Partly Cloudy, 54°F/12°C, 5°F/-15°C, 14%, 0.00, N 3 MPH, 70mi.
2:55PM, Mostly Clear, 57°F/14°C, 7°F/-14°C, 13%, 0.00, WNW 6 MPH, 70mi.
3:55PM, Mostly Clear, 57°F/14°C, 3°F/-16°C, 11%, 0.00, NNW 7 MPH, 70mi.
4:55PM, Mostly Clear, 59°F/15°C, 3°F/-16°C, 10%, 0.00, NW 12 MPH, 70mi.
5:55PM, Mostly Clear, 57°F/14°C, -6°F/-21°C, 7%, 0.00, W 20 MPH, 70mi.
6:55PM, Mostly Clear, 54°F/12°C, -2°F/-19°C, 10%, 0.00, WNW 15 MPH, 70mi.
7:55PM, Mostly Clear, 46°F/8°C, 5°F/-15°C, 18%, 0.00, NW 7 MPH, 70mi.
8:55PM, Mostly Clear, 45°F/7°C, 9°F/-13°C, 23%, 0.00, W 7 MPH, 25mi.
9:55PM, Mostly Clear, 41°F/5°C, 12°F/-11°C, 31%, 0.00, NW 9 MPH, 25mi.
10:55PM, Mostly Clear, 43°F/6°C, 14°F/-10°C, 31%, 0.00, WNW 10 MPH, 2

If you did have a table inside a table you would simply find the first then use first_table.find("table"), or if the table had a class or id you could maybe just use that to select the table directly.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.