1

I have the below code

from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options

opts = Options()
opts.set_headless()
assert opts.headless  # Operating in headless mode
browser = Firefox(options=opts)
txt=browser.get('https://duckduckgo.com')
print(txt.text)

However, when i execute it, i get the error "AttributeError: 'NoneType' object has no attribute 'text'" I think I should get the text of the HTML.What am I doing wrong?

1 Answer 1

1

This error message...

AttributeError: 'NoneType' object has no attribute 'text'

...implies that the you have tried to extract the text attribute from a NoneType object.


get(url)

get(url) method loads a web page in the current browser session and doesn't returns anything.

Though you have tried to assign the variable text with the return value of get() as in:

txt=browser.get('https://duckduckgo.com')

txt still remains 'NoneType'


Solution

A bit more information about your usecase would have helped us to construct an answer in a better way. However if your are trying to retrieve the HTML you can use the page_source property as follows:

  • Code Block:

    from selenium.webdriver.firefox.options import Options 
    from selenium.webdriver import Firefox
    
    opts = Options()
    # opts.set_headless()
    opts.headless = True
    assert opts.headless  # Operating in headless mode
    browser = Firefox(options=opts)
    browser.get("https://duckduckgo.com")
    print(browser.page_source.encode("utf-8"))
    
  • Console Output:

    b'<html class="js no-touch opacity csstransforms3d csstransitions svg cssfilters is-not-mobile-device full-urls has-search-focus" lang="en_US"><!--<![endif]--><head>\n\t<meta http-equiv="X-UA-Compatible" content="IE=Edge">\n<meta http-equiv="content-type" content="text/html; charset=UTF-8;charset=utf-8">\n<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=1">\n<meta name="HandheldFriendly" content="true">\n\n<link rel="canonical" href="https://duckduckgo.com/">\n\n<link rel="stylesheet" href="/s1771.css" type="text/css">\n\n<link rel="stylesheet" href="/o1771.css" type="text/css">\n\n\n\n<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" sizes="16x16 32x32">\n<link rel="apple-touch-icon" href="/assets/icons/meta/DDG-iOS-icon_60x60.png">\n<link rel="apple-touch-icon" sizes="76x76" href="/assets/icons/meta/DDG-iOS-icon_76x76.png">\n<link rel="apple-touch-icon" sizes="120x120" href="/assets/icons/meta/DDG-iOS-icon_120x120.png">\n<link rel="apple-touch-icon" sizes="152x152" href="/assets/icons/meta/DDG-iOS-icon_152x152.png">\n<link rel="image_src" href="/assets/icons/meta/DDG-icon_256x256.png">\n<link rel="manifest" href="/manifest.json">\n\n<meta name="twitter:card" content="summary">\n<meta name="twitter:site" value="@duckduckgo">\n\n<meta property="og:url" content="https://duckduckgo.com/">\n<meta property="og:site_name" content="DuckDuckGo">\n<meta property="og:image" content="https://duckduckgo.com/assets/logo_social-media.png">\n\n\n\t<title>DuckDuckGo \xe2\x80\x94 Privacy, simplified.</title>\n<meta property="og:title" content="DuckDuckGo \xe2\x80\x94 Privacy, simplified.">\n\n\n<meta property="og:description" content="The Internet privacy company that empowers you to seamlessly take control of your personal information online, without any tradeoffs.">\n<meta name="description" content="The Internet privacy company that empowers you to seamlessly take control of your personal information online, without any tradeoffs.">\n\n\n<link title="DuckDuckGo" type="application/opensearchdescription+xml" rel="search" href="https://duckduckgo.com/opensearch.xml?atb=v167-3__"></head>\n<body id="pg-index" class="page-index body--home body--onboarding">\n\t<script type="text/javascript">\nvar settings_js_version = "/s2472.js",\n    locale = "en_US";\n</script>\n<script type="text/javascript" src="/lib/l113.js"></script>\n<script type="text/javascript" src="/locale/en_US/duckduckgo9.js"></script>\n<script type="text/javascript" src="/util/u344.js"></script>\n<script type="text/javascript" src="/d2613.js"></script>\n\n\n\n<script type="text/javascript">\n    DDG.page = new DDG.Pages.Home();\n</script>\n\n\n\n\t<div class="site-wrapper  site-wrapper--home  js-site-wrapper">\n\t\n\t\t\n\t\t\t<div class="header-wrap--home  js-header-wrap">\n\t<div class="header--aside js-header-aside is-hidden"><a class="header__button--menu  js-side-menu-open" href="#">\xe2\x87\xb6</a><div class="header--aside__item showcase header__label"><span class="header__clickable js-hl-button" data-type="showcase"><span class="js-popout-trig" aria-haspopup="true" aria-label="Check out the list of things that we\'ve also made." role="button" aria-pressed="false"><span id="wedonttrack">Privacy, simplified.</span></span><span class="popout-trig  js-popout"><span class="js-popout-link js-showcase-popout ddgsi ddgsi-down" aria-hidden="true" data-type="showcase"></span><div class="modal modal--popout modal--popout--bottom-left modal--popout--sm  js-popout-main" data-type="showcase"><div class="modal__box"><div class="modal__body"><nav aria-labelledby="wedonttrack"><section class="showcase__dropdown-top"><ul aria-label="Here are some things that we made that you might like.">                                        <li class="fix showcase__dropdown__list"><a href="/app" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="app"><div class="woman-icon"></div></a><a href="/app" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="app"><h1 class="showcase__heading">Get Our App &amp; Extension</h1><p class="showcase__subheading">Protect your data on every device.</p></a></li><li class="fix showcase__dropdown__list"><a href="/newsletter" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="newsletter"><div class="newsletter-icon"></div></a><a href="/newsletter" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="newsletter"><h1 class="showcase__heading">Subscribe to our Privacy Crash Course</h1><p class="showcase__subheading">Learn about online privacy right in your inbox.</p></a></li><li class="fix showcase__dropdown__list"><a href="https://spreadprivacy.com/tag/device-privacy-tips/" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="blog"><div class="privacy-simplified-icon"></div></a><a href="https://spreadprivacy.com/tag/device-privacy-tips/" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="blog"><h1 class="showcase__heading">Protect Your Devices</h1><p class="showcase__subheading">Check out our privacy device guides.</p></a></li><li class="fix showcase__dropdown__list"><a href="https://duckduckgo.com/spread" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="spread"><div class="spread-icon"></div></a><a href="https://duckduckgo.com/spread" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="spread"><h1 class="showcase__heading">Spread DuckDuckGo</h1><p class="showcase__subheading">Help your friends and family join the Duck Side!</p></a></li></ul></section><section class="showcase__dropdown-bottom"><ul class="text-left" aria-label="We\'ve got even more things for you."><li class="fix showcase__dropdown__list"><a href="https://duckduckgo.com/donations" class="eighteen showcase__icon js-hl-item" aria-hidden="true" data-type="showcase" data-id="donations"><div class="donations-icon"></div></a><a href="https://duckduckgo.com/donations" class="text-left showcase__link eighty showcase__text js-hl-item" data-type="showcase" data-id="donations">$1,300,000 in privacy donations!</a></li><li class="fix showcase__dropdown__list"><a href="https://duckduckgo.com/traffic" class="eighteen showcase__icon js-hl-item" aria-hidden="true" data-type="showcase" data-id="traffic"><div class="traffic-icon"></div></a><a href="https://duckduckgo.com/traffic" class="text-left showcase__link eighty showcase__text js-hl-item" data-type="showcase" data-id="traffic">Over 29 Billion anonymous searches.</a></li><li class="fix showcase__dropdown__list"><a href="https://donttrack.us/" class="eighteen showcase__icon js-hl-item" aria-hidden="true" data-type="showcase" data-id="dnt"><div class="privacy-tips-icon"></div></a><a href="https://donttrack.us/" class="text-left showcase__link eighty showcase__text js-hl-item" data-type="showcase" data-id="dnt">Learn why reducing tracking is important.</a></li></ul></section></nav></div></div></div></span></span></div><div class="header--aside__item social"><a aria-label="Twitter" class="header--aside__item header--aside__twitter js-hl-twitter header__clickable header__link" href="https://twitter.com/duckduckgo"></a><span class="popout-trig  js-popout header__clickable header__label js-hl-button" data-type="social"><span class="js-popout-link ddgsi ddgsi-down" aria-hidden="true" data-type="social"></span><div class="modal modal--popout modal--popout--bottom-left modal--popout--sm  js-popout-main" data-type="social"><div class="modal__box"><div class="modal__body"><div class="social__link"><a href="https://twitter.com/duckduckgo" class="js-hl-item" aria-hidden="true" data-type="social" data-id="twitter"><img class="social__icon js-lazysvg" data-src="/assets/icons/header/twitter-color.svg"></a><a href="https://twitter.com/duckduckgo" class="js-hl-item social__link__text" data-type="social" data-id="twitter">Twitter</a></div><div class="social__link"><a href="https://reddit.com/r/duckduckgo" class="js-hl-item" aria-hidden="true" data-type="social" data-id="reddit"><img class="social__icon--light js-lazysvg" data-src="/assets/icons/header/reddit-color.svg"><img class="social__icon--dark js-lazysvg" data-src="/assets/icons/header/reddit-dark.svg"></a><a href="https://reddit.com/r/duckduckgo" class="js-hl-item social__link__text" data-type="social" data-id="reddit">Reddit</a></div><div class="social__link"><a href="https://spreadprivacy.com" class="js-hl-item" aria-hidden="true" data-type="social" data-id="blog"><img class="social__icon--light js-lazysvg" data-src="/assets/icons/header/globe-alt.svg"><img class="social__icon--dark js-lazysvg" data-src="/assets/icons/header/globe-dark.svg"></a><a href="https://spreadprivacy.com" class="js-hl-item social__link__text" data-type="social" data-id="blog">Blog</a></div></div></div></div></span></div></div>\n</div>\n\t\t\t<div id="" class="content-wrap--home">\n\t\t\t\t<div id="content_homepage" class="content--home">\n\t\t\t\t\t<div class="cw--c">\n\t\t\t\t\t\t\t\t<div class="logo-wrap--home">\n\t\t\t<a id="logo_homepage_link" class="logo_homepage" href="/about">\n\t\t\t\tAbout DuckDuckGo\n\t\t\t\t<span class="logo_homepage__tt">Learn More</span>\n\t\t\t</a>\n\t\t</div>\n\n\t\t\t\t\t\t<div class="search-wrap--home">\n\t\t\t\t\t\t\t\t\t<form id="search_form_homepage" class="search--home  js-search-form search--adv search--focus" name="x" method="GET" action="/">\n\t\t\t<input id="search_form_input_homepage" class="js-search-input search__input--adv" type="text" autocomplete="off" name="q" tabindex="1" value="" autocapitalize="off" autocorrect="off">\n\t\t\t<input id="search_button_homepage" class="search__button  js-search-button" type="submit" tabindex="2" value="S">\n\t\t\t<input id="search_form_input_clear" class="search__clear  empty  js-search-clear" type="button" tabindex="3" value="X">\n\t\t\t<div id="search_elements_hidden" class="search__hidden  js-search-hidden"><input type="hidden" class="js-search-hidden-field" name="t" value="h_"></div>\n\t\t<div class="search__autocomplete" style="position: absolute; display: none;"><div class="acp-wrap js-acp-wrap"></div><div class="acp-footer is-hidden js-acp-footer"><span class="acp-footer__instructions">Shortcuts to other sites to search off DuckDuckGo</span><span class="acp-footer__link"><a class="no-visited js-acp-footer-link" href="/bang">Learn More</a></span></div></div></form>\n\n\t\t\t\t\t\t</div>\n\t\t\n\t\n\n\t\t\t\t\t\t<!-- en_US All Settings -->\n<noscript>\n    <div class="tag-home">\n        <div class="tag-home__wrapper">\n            <div class="tag-home__item">\n                The search engine that doesn\'t track you.\n                <span class="hide--screen-xs"><a href="/about" class="tag-home__link">Learn More</a>.</span>\n            </div>\n        </div>\n    </div>\n</noscript>\n<div class="tag-home  tag-home--slide  no-js__hide  js-tag-home"><div class="tag-home__wrapper"><div class="tag-home__item">The search engine that doesn\'t track you.<span class="hide--screen-xs"> <a class="tag-home__link js-tag-item-link" href="/spread">Help Spread DuckDuckGo!</a></span></div></div></div>\n        <div id="error_homepage"></div>\n\n\n\t\n\t\t\n\t\t\t\t\t</div> <!-- cw -->\n\t\t\t\t</div> <!-- content_homepage //-->\n\t\t\t</div> <!-- content_wrapper_homepage //-->\n\t\t\t<div id="footer_homepage" class="foot-home  js-foot-home"></div>\n\n<script type="text/javascript">\n\t{function seterr(str) {\n\t\tvar error=document.getElementById(\'error_homepage\');\n\t\terror.innerHTML=str;\n\t\t$(error).css(\'display\',\'block\');\n\t}\n\tvar err=new RegExp(\'[\\?\\&]e=([^\\&]+)\');var errm=new Array();errm[\'2\']=\'no search\';errm[\'3\']=\'search too long\';errm[\'4\']=\'not UTF\\u002d8 encoding\';errm[\'6\']=\'too many search terms\';if (err.test(window.location.href)) seterr(\'Oops, \'+(errm[RegExp.$1]?errm[RegExp.$1]:\'there was an error.\')+\' &nbsp;Please try again\');};\n\t\n\tif (kurl) {\n\t  document.getElementById("logo_homepage_link").href += (document.getElementById("logo_homepage_link").href.indexOf(\'?\')==-1 ? \'?t=i\' : \'\') + kurl;\n\t}\n</script>\n\n\t\t\n\t\n\t<div class="nav-menu--slideout"><ul class="nav-menu__list"><i class="nav-menu__close  js-side-menu-close">X</i><ul class="nav-menu--theme"><li class="nav-menu__heading"><span>Settings</span></li><li class="nav-menu__item"><ul class="nav-menu__themes"><li class="nav-menu__theme  js-side-menu-theme theme-is-selected"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#084999;"></span><span class="nav-menu__theme-color-bot" style="background-color:#888;"></span></span></li><li class="nav-menu__theme  js-side-menu-theme"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#333;"></span><span class="nav-menu__theme-color-bot" style="background-color:#888;"></span></span></li><li class="nav-menu__theme  js-side-menu-theme"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#3434d4;"></span><span class="nav-menu__theme-color-bot" style="background-color:#047d00;"></span></span></li><li class="nav-menu__theme  js-side-menu-theme"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#444;"></span><span class="nav-menu__theme-color-bot" style="background-color:#222;"></span></span></li></ul></li><li class="nav-menu__item clear"><a href="/settings#theme" data-settings="1">More Themes</a></li><li class="nav-menu__item"><a href="/settings" tabindex="-1" data-settings="1">Other Settings</a></li><li class="nav-menu__item"><a href="/bangs">!Bang Search Shortcuts</a></li></ul><ul><li class="nav-menu__heading"><span>Privacy Essentials</span></li><li class="nav-menu__item"><a href="https://duckduckgo.com/">Private Search</a></li><li class="nav-menu__item"><a href="/app">App and Extension</a></li></ul><ul><li class="nav-menu__heading"><span>Why Privacy</span></li><li class="nav-menu__item"><a href="https://spreadprivacy.com/">Privacy Blog</a></li><li class="nav-menu__item"><a href="https://duckduckgo.com/newsletter">Privacy Crash Course</a></li><li class="nav-menu__item"><a href="/spread">Help Spread Privacy</a></li></ul><ul><li class="nav-menu__heading"><span>Who We Are</span></li><li class="nav-menu__item"><a href="/about">About Us</a></li><li class="nav-menu__item"><a href="/privacy">Privacy Policy</a></li><li class="nav-menu__item"><a href="/hiring">Careers</a></li><li class="nav-menu__item"><a href="/press">Press Kit</a></li></ul><ul><li class="nav-menu__heading"><span>Keep in Touch</span></li><li class="nav-menu__item"><a href="https://twitter.com/duckduckgo">Twitter</a></li><li class="nav-menu__item"><a href="https://reddit.com/r/duckduckgo">Reddit</a></li><li class="nav-menu__item"><a href="https://help.duckduckgo.com/">Help</a></li></ul></ul></div><div class="badge-link badge-link--top badge-link--fadein  ddg-extension-hide js-badge-link"><a class="badge-link__wrap js-badge-main-msg" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi"><div class="badge-link__thumb"><img class="badge-link__thumb__img" src="assets/onboarding/robot-icon-frameless.svg"></div><p class="badge-link__title js-badge-link__title">Your data shouldn\xe2\x80\x99t be for sale.<br>At DuckDuckGo, we agree.</p><ol class="badge-link__bullets"><li class="badge-link__bullet"><span class="badge-link__bullet-num">1</span>Block advertising trackers.</li><li class="badge-link__bullet"><span class="badge-link__bullet-num">2</span>Keep your search history private.</li><li class="badge-link__bullet"><span class="badge-link__bullet-num">3</span>Take control of your personal data.</li></ol><div class="badge-link__btn-group"><span class="badge-link__btn btn btn--primary js-badge-link-button">Add DuckDuckGo to Firefox</span></div><span class="ddgsi badge-link__close js-badge-link-dismiss">\xc3\x97</span></a><div class="badge-link__wrap badge-link__wrap--hidden js-badge-cookie-msg"><div class="badge-link__cookie-msg"><p>Clear your cookies often?</p><p>Try our homepage that never shows these messages:</p><p><b><a href="https://start.duckduckgo.com">start.duckduckgo.com</a></b></p></div><span class="ddgsi badge-link__close js-badge-link-close">\xc3\x97</span></div></div><div class="onboarding-ed js-onboarding-ed"> <a class="onboarding-ed__arrow-teaser js-onboarding-ed-teaser" style="display: inline;"><img class="onboarding-ed__arrow-teaser__alpinist" src="/assets/onboarding/bathroomguy/[email protected]"><div class="onboarding-ed__arrow-teaser__arrow-wrapper"><img class="onboarding-ed__arrow-teaser__arrow" src="/assets/onboarding/arrow.svg"></div></a><a class="onboarding-ed__arrow js-onboarding-ed-arrow" style="display: none;"><img src="/assets/onboarding/arrow.svg"></a><div class="onboarding-ed__slide onboarding-ed__slide-1 js-onboarding-ed-slide js-onboarding-ed-slide-1"><div class="onboarding-ed__content"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t store your<br data-owner="balance-text">personal information. Ever.</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">Our privacy policy is simple: we don\xe2\x80\x99t collect<br data-owner="balance-text">or share any of your personal information.</p><div class="js-onboarding-ed-button-small-1"><a class="btn onboarding-ed__button-small ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-1"><img src="/assets/onboarding/bathroomguy/1-monster-v2--pre-animation.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"></div></div><div class="onboarding-ed__slide onboarding-ed__slide-2 js-onboarding-ed-slide js-onboarding-ed-slide-2"><div class="onboarding-ed__content onboarding-ed__content--more-shadow"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t follow you<br data-owner="balance-text">around with ads.</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t store your search history. We therefore have nothing<br data-owner="balance-text">to sell to advertisers that track you across the Internet.</p><div class="js-onboarding-ed-button-small-2"><a class="btn onboarding-ed__button-small ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-2"><img src="/assets/onboarding/bathroomguy/2-ghost-v2.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"></div></div><div class="onboarding-ed__slide onboarding-ed__slide-3 js-onboarding-ed-slide js-onboarding-ed-slide-3"><div class="onboarding-ed__content"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t track you in or out<br data-owner="balance-text">of private browsing mode.</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">Other search engines track your searches even when you\xe2\x80\x99re<br data-owner="balance-text">in private browsing mode. We don\xe2\x80\x99t track you \xe2\x80\x94 period.</p><div class="js-onboarding-ed-button-small-3"><a class="btn onboarding-ed__button-small ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-3"><img src="/assets/onboarding/bathroomguy/3-bathtub-v2--pre-animation.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"> </div></div><div class="onboarding-ed__slide onboarding-ed__slide-4 js-onboarding-ed-slide js-onboarding-ed-slide-4"><div class="onboarding-ed__close"><span class="js-onboarding-ed-dismiss onboarding-ed__close-dismiss">Dismiss forever</span><span class="ddgsi onboarding-ed__close-divider">|</span> <span class="onboarding-ed__close-back js-onboarding-ed-back-to-search">Back to search</span> </div><div class="onboarding-ed__content js-onboarding-ed-install"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">Switch to DuckDuckGo and<br data-owner="balance-text">take back your privacy!</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">No tracking, no ad targeting, just searching.</p><div class="js-onboarding-ed-button"><a class="btn onboarding-ed__button ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__content js-onboarding-ed-spread" style="display:none"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">Already a fan?<br>Invite friends to the Duck Side!</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">Share DuckDuckGo and help friends take their privacy back!</p><div><a class="btn onboarding-ed__button onboarding-ed__tweet-button js-onboarding-ed-tweet-button" target="_blank" href="https://twitter.com/?status=DuckDuckGo%20is%20my%20search%20engine%20of%20choice.%20%23ComeToTheDuckSide%20-%20we%20have%20privacy.%20https%3A%2F%2Fduckduckgo.com%2F%231">Tweet #ComeToTheDuckSide</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-4"><img src="/assets/onboarding/bathroomguy/4-alpinist-v2.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"></div></div></div></div> <!-- site-wrapper -->\n\n\n<iframe id="iframe_hidden" src="/post2.html"></iframe></body></html>'
    

Here you can find a detailed discussion on What is a None value?

Sign up to request clarification or add additional context in comments.

3 Comments

Hi the use case for me is to find the Java scripts within the page & then scrape it by passing the data that i want. The "duckduckgo.com" mentioned here is not the website i am working on (it is a different one). In this case, i am trying to find out the input field website. Then, enter a text string in ther & scrape the result that we get. I will try out your solution thanks.
actually, the above script gets me the HTML code but not the java script. ANd that does not serve my purpose. Then i found another webpage to get the java code underneath the page from requests_html import HTMLSession # create an HTML Session object session = HTMLSession() BOP_url = 'https://duckduckgo.com' # Use the object above to connect to needed webpage resp = session.get(BOP_url,verify=False) # Run JavaScript code on webpage resp.html.render() but this trows me "RuntimeError: Cannot use HTMLSession within an existing event loop. Use AsyncHTMLSession instead."
@mohanys There was no mentioning of Java scripts within the page & then scrape within the initial version of your question. This seems to be a new requirement and you need to raise a new question for your new requirement. StackOverflow contributors will be happy to help you out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.