I am trying to web scrape text from a href inside a scope, which can't be accessed through xpath as I want to iterate through a table and find the text inside the box.
8
-
looks accessible to me; what have you tried?Sam Mason– Sam Mason2018-10-09 17:43:36 +00:00Commented Oct 9, 2018 at 17:43
-
if its in the DOM, can be reached :)Infern0– Infern02018-10-09 17:46:21 +00:00Commented Oct 9, 2018 at 17:46
-
@SamMason I have tried, but it wouldn't show up, I tried multiple solutionsTudor Popescu– Tudor Popescu2018-10-09 19:34:25 +00:00Commented Oct 9, 2018 at 19:34
-
@TudorPopescu what have you tried? anything like: stackoverflow.com/a/19035495/1358308Sam Mason– Sam Mason2018-10-09 19:55:18 +00:00Commented Oct 9, 2018 at 19:55
-
@SamMason as the /a in my case doesn't have any other class, what should I do?Tudor Popescu– Tudor Popescu2018-10-09 20:00:18 +00:00Commented Oct 9, 2018 at 20:00
|
Show 3 more comments
2 Answers
I think you can do
import urllib.request
page_url = #Insert your url here
with urllib.request.urlopen(page_url) as f:
html = f.read().decode('utf-8')
html.find(#whatever)
which I think should return the entirety of the html for the page, which you can then scrape for whatever you need.
Comments
According to information provided by you, you wanted to print the text, for example in this case: "3i Group PLC".
I just call the xpath link for <a href="stock-info.php?ticker=III:LN">3i Group PLC</a> is link_abc because the xpath link is not provided by you.
Here is the code by using selenium library:
name = driver.find_element_by_xpath("link_abc")
print(name.text)
The output should be 3i Group PLC.
