0

I am using selenium library for python and chrome browser.

I need to click a link from the url:

http://www.youtube-mp3.org/

The idea is to download a list of videos converted to mp3 files in a previous code. My script is:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

import time
url2='http://www.youtube-mp3.org/'

chromedriver = 'C:\\exp\\chromedriver.exe' #where you have the file
browser = webdriver.Chrome(chromedriver)
browser.get(url2)

direc = browser.find_element_by_id("youtube-url")
direc.clear()

direc.send_keys("https://www.youtube.com/watch?v=nYh-n7EOtMA") #an example url
browser.find_element_by_id("btns").click()

Until here everything is ok.

Now I need to press the download link that appears after the video is converted.

The html for that part is:

   <div id="dl_link" style="display: block;">
    <a href="/get?video_id=nYh-n7EOtMA&amp;h=-1&amp;r=-1.1" style="display:none"><b>Download</b></a>
    <a href="/get?video_id=nYh-n7EOtMA&amp;ts_create=1463533555&amp;r=MTg2LjYwLjE2MS4yMTE%3D&amp;h2=5ad90182ae65fea567f844c3b6a933aa&amp;s=145334"><b>Download</b></a><a href="/get?video_id=nYh-n7EOtMA&amp;h=-1&amp;r=-1.1" style="display:none">
  <b>Download</b>
 </a></div>

But I can only see it when I select "inspect object", because if I select "view source code" (after I pasted the url and the video and link appear), there's nothing in it.

I tried with:

browser.find_element_by_partial_link_text("create").click()

But I got an error:

NoSuchElementException: Message: no such element

and also with:

browser.find_element_by_name("dl_link").click()

error:

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"name","selector":"dl_link"}

and also with:

browser.find_element_by_id("dl_link").click()

and got:

ElementNotVisibleException: Message: element not visible

So, my questions are:

  1. why I cannot see the html script for the download link when I press view source code?
  2. is it possible to finish the process (downloading the mp3 file) with python?
  3. does the fact that the web page has java scripts in it is related to the issue?
  4. is there any other library to do this?

thanks.

2
  • Changing to this browser.find_element_by_name("dl_link").click() shall work ideally. Commented May 18, 2016 at 3:23
  • @nullpointer, I've already tried that before and got no result. The error message is : Message: no such element: Unable to locate element: {"method":"name","selector":"dl_link"} Commented May 18, 2016 at 12:34

1 Answer 1

1

Why do the elements appear in the inspect function but not in the source code?

The HTML elements may either be automatically generated by some sort of script (e.g. Javascript, VBScript, ...), or received from AJAX. The Source code contains the response in a text format, what means no script had a chance to run. The inspect function of your browser, on the other hand, displays the current state

What You could do to narrow down the issue:

  1. Visit the page in a browser and turn off Javascript. If the elements aren't there now, You can safely assume that Javascript inserts the elements in question.

  2. If the above text happens to be the case, turn javascript back on and bring up the network tab in the developer tools. Reload the page and watch the ressource that are requested. Check if you can find any AJAX/API requests in there. If so, check how those are made and try to replicate them.

  3. If You can't find any AJAX/API calls, the elements are generated differently. Check the source code for script files ( html tag) and check the code inside for more info.

I am not aware of any libraries that could help You out here (although there might be one or two, I dont code Python that much), but it's definitely possible to get the content You need (might be a little tricky, depends on how much the site owner tries

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.