python scrape input value from html

Question

Hey I need to scrape an html for a specific shipping rate here's the HTML:

<ul class="methods" id="shipping-quotes" data-url="/en/webshipper/setshippingquote" data-shipping-fee="£ 12">
                                                                <li class="method-option method-option--shipping">
            <div class="method-option__header">
                <input
                    id="shippingrate-8"
                    name="shippingrate"
                    type="radio"
                    value="8"
                    class="method-option__input shipping-rate"
                    
                />
                <label for="shippingrate-8" class="method-option__label">
                    <span class="method-option__name">UPS Standard UK - Home Delivery (2-4 business days)</span>
                    <span class="method-option__fee">9 GBP</span>
                </label>
            </div>
                        </li>
                                                                <li class="method-option method-option--shipping">
            <div class="method-option__header">
                <input
                    id="shippingrate-9"
                    name="shippingrate"
                    type="radio"
                    value="9"
                    class="method-option__input shipping-rate"
                    
                />
                <label for="shippingrate-9" class="method-option__label">
                    <span class="method-option__name">UPS Standard UK - Pick-up Point (2-4 business days)</span>
                    <span class="method-option__fee">6 GBP</span>
                </label>
            </div>
                        </li>
                                                                <li class="method-option method-option--shipping">
            <div class="method-option__header">
                <input
                    id="shippingrate-15"
                    name="shippingrate"
                    type="radio"
                    value="15"
                    class="method-option__input shipping-rate"
                    
                />
                <label for="shippingrate-15" class="method-option__label">
                    <span class="method-option__name">UPS Express UK - Home Delivery (2-4 business days)</span>
                    <span class="method-option__fee">15 GBP</span>
                </label>
            </div>
                        </li>
                                                                <li class="method-option method-option--shipping">
            <div class="method-option__header">
                <input
                    id="shippingrate-29"
                    name="shippingrate"
                    type="radio"
                    value="29"
                    class="method-option__input shipping-rate"
                    checked
                />
                <label for="shippingrate-29" class="method-option__label">
                    <span class="method-option__name">UPS Express UK - Pick-up Point (1-3 business days)</span>
                    <span class="method-option__fee">12 GBP</span>
                </label>
            </div>
                                
<div class="drop-points" id="drop-points">
    <p class="drop-point drop-point--no-result">
        No pickup points found near your chosen location
    </p>
</div>
                        </li>
        </ul>

Specifically the home delivery option so in the HTML below I would need to get the value= 9 value so I would like to get the value of 9. How could I get the specific value for the home delivery option using bs4?

If I am unclear i mean the value option within this

<input
                    id="shippingrate-9"
                    name="shippingrate"
                    type="radio"
                    value="9"
                    class="method-option__input shipping-rate"

                />

For the home delivery 2-4 days option.

As discussed here, you should be able to use something like soup.find('input', {'name': 'shippingrate'}).attrs['value']. — user5386938
– user5386938, Commented Mar 8, 2021 at 17:13
that gets the first value, how do I get the shipping rate value for the "home delivery" option? — Movie Clips
– Movie Clips, Commented Mar 8, 2021 at 18:37
Use soup.find_all and a loop. You need to read the docs as you seem to lack the basics and are trying to run before you can walk. — user5386938
– user5386938, Commented Mar 8, 2021 at 18:41
i understand find all but how can I search for the value in between a class within a label? — Movie Clips
– Movie Clips, Commented Mar 8, 2021 at 18:45

Rustam Garayev · Accepted Answer · 2021-03-08 18:48:12Z

1

If you want to get the input value of shipping method (let's say UPS Standard UK - Pick-up Point (2-4 business days) in your case), you can first find all divs then pick the one with this method. So something like this will do the job:

data = soup.find('ul', attrs={'id': 'shipping-quotes'}).find_all('div', class_='method-option__header')

values = [each.input['value'] for each in data if each.find('span', class_='method-option__name').text == 'UPS Standard UK - Pick-up Point (2-4 business days)']

values will be:

['9']

edited Mar 8, 2021 at 18:48

answered Mar 8, 2021 at 17:21

Rustam Garayev

2,6921 gold badge11 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Movie Clips Over a year ago

hey how could i get the value of 9 not the shipping method for the value of 9. SO what I mean is I want to get the value number for the home delivery option

Rustam Garayev Over a year ago

@MovieClips edited, check if this is what you want

user5386938 · Accepted Answer · 2021-03-08 20:24:24Z

You've not shown any effort at all but since I needed a break...

from bs4 import BeautifulSoup

data = '''\
<ul class="methods" id="shipping-quotes" data-url="/en/webshipper/setshippingquote" data-shipping-fee="£ 12">
  <li class="method-option method-option--shipping">
    <div class="method-option__header">
    <input id="shippingrate-8" name="shippingrate" type="radio" value="8" class="method-option__input shipping-rate" /> 
    <label for="shippingrate-8" class="method-option__label">
    <span class="method-option__name">UPS Standard UK - Home Delivery (2-4 business days)</span> 
    <span class="method-option__fee">9 GBP</span></label></div>
  </li>
  <li class="method-option method-option--shipping">
    <div class="method-option__header">
    <input id="shippingrate-9" name="shippingrate" type="radio" value="9" class="method-option__input shipping-rate" /> 
    <label for="shippingrate-9" class="method-option__label">
    <span class="method-option__name">UPS Standard UK - Pick-up Point (2-4 business days)</span> 
    <span class="method-option__fee">6 GBP</span></label></div>
  </li>
  <li class="method-option method-option--shipping">
    <div class="method-option__header">
    <input id="shippingrate-15" name="shippingrate" type="radio" value="15" class="method-option__input shipping-rate" /> 
    <label for="shippingrate-15" class="method-option__label">
    <span class="method-option__name">UPS Express UK - Home Delivery (2-4 business days)</span> 
    <span class="method-option__fee">15 GBP</span></label></div>
  </li>
  <li class="method-option method-option--shipping">
    <div class="method-option__header">
    <input id="shippingrate-29" name="shippingrate" type="radio" value="29" class="method-option__input shipping-rate"
    checked="checked" /> 
    <label for="shippingrate-29" class="method-option__label">
    <span class="method-option__name">UPS Express UK - Pick-up Point (1-3 business days)</span> 
    <span class="method-option__fee">12 GBP</span></label></div>
    <div class="drop-points" id="drop-points">
      <p class="drop-point drop-point--no-result">No pickup points found near your chosen location</p>
    </div>
  </li>
</ul>
'''

soup = BeautifulSoup(data, 'html.parser')
for d in soup.select('div.method-option__header'):
    rate = d.find('input', {'name': 'shippingrate'}).attrs['value']
    name = d.find('span', {'class': 'method-option__name'}).text
    fee = d.find('span', {'class': 'method-option__fee'}).text
    print(f"{rate}\t{name}\t{fee}")

Collectives™ on Stack Overflow

python scrape input value from html

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related