How to call a method within a Python class using Selenium and Python

Question

I am very new to coding, I need to solve one practical thing just get info from site and write it to excel(i hope i can manage with by guides), but the main issue i just cant get into the site(the website is free) Can you look over my code? When i run it, i get

[] Process finished with exit code 0

import requests
from bs4 import BeautifulSoup
import pytest
import time
import json
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

URL ='http://way2drug.com/passonline/'
HEADERS= {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36', 'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'}


class Test1():
  def setup_method(self, method):
    self.driver = webdriver.Chrome()
    self.vars = {}
  
  def teardown_method(self, method):
    self.driver.quit()
  
  def test_1(self):
    # Test name: 1
    # Step # | name | target | value | comment
    # 1 | open | /passonline/ |  | 
    self.driver.get("http://way2drug.com/passonline/")
    # 2 | setWindowSize | 1920x1030 |  | 
    self.driver.set_window_size(1920, 1030)
    # 3 | click | css=#registration img |  | 
    self.driver.find_element(By.CSS_SELECTOR, "#registration img").click()
    # 4 | click | name=user_login |  | 
    self.driver.find_element(By.NAME, "user_login").click()
    # 5 | type | name=user_login |  | 
    self.driver.find_element(By.NAME, "user_login").send_keys("MY USER")
    # 6 | click | id=page1 |  | 
    self.driver.find_element(By.ID, "page1").click()
    # 7 | type | name=user_password |  | 
    self.driver.find_element(By.NAME, "user_password").send_keys("MY PASS")
    # 8 | click | id=register |  | 
    self.driver.find_element(By.ID, "register").click()
    # 9 | click | id=myHeader1 |  | 
    self.driver.find_element(By.ID, "myHeader1").click()
    # 10 | click | id=smiles |  | 
    self.driver.find_element(By.ID, "smiles").click()
    self.driver.find_element(By.ID, "smi").click()
    self.driver.find_element(By.ID, "smi").send_keys("CC1(C)C(O)CC[C@@]2(C)C1CC[C@]3(C)C2CCC4[C@@]3(C)CC[C@]5(C(O)=O)C4[C@H](C)C(C)=CC5")
    self.driver.find_element(By.CSS_SELECTOR, "#myContent4 input:nth-child(4)").click()



def get_html(url, params=None):
  r = requests.get(url, headers=HEADERS, params=params)
  return r

def get_content(html):
  soup = BeautifulSoup(html, 'html.parser')
  items = soup.find_all('a', class_='Antineoplastic')

  print(items)

def parse():
  html = get_html(URL)
  if html.status_code == 200:
    get_content(html.text)
  else:
    print('ALL YOUR BASE ARE BELONG TO US')


parse()

Process finished with exit code 0 - It means that there is no error with your code. You have run it right through and there is nothing wrong with it. So pleasae focus and improve your question. Thanks — HedgeHog
– HedgeHog, Commented Nov 28, 2021 at 9:50
But if It was ok, I should get some data in terminal on this step def get_content(html): soup = BeautifulSoup(html, 'html.parser') items = soup.find_all('a', class_='Antineoplastic') print(items) — Gogi
– Gogi, Commented Nov 28, 2021 at 10:15

HedgeHog · Accepted Answer · 2021-11-28 12:28:41Z

0

Please note - Never ever provide credentials

What happens?

You mentioned you have to perform a login and selenium is a good choice, but what you are doing is to call parse() that only performce a call via requests. So if you take a look into your soup you wont find what you are looking for.

How to fix?

Perform your selenium actions and walk to the website you want to scrape. In next step push your driver.page_source into BeautifulSoup and find your elements:

soup = BeautifulSoup(driver.page_source,'html.parser')
items = soup.find_all('a', class_='Antineoplastic')

print(items)

If your selection is right, you will get your result.

EDIT

Concerning your comments a clue where you can end up, for debugging steps between, you should ask as new question with focused examples:

import requests
from bs4 import BeautifulSoup
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

driver = webdriver.Chrome()
 
driver.get("http://way2drug.com/passonline/")
driver.set_window_size(1920, 1030)
driver.find_element(By.CSS_SELECTOR, "#registration img").click()
driver.find_element(By.NAME, "user_login").click()
driver.find_element(By.NAME, "user_login").send_keys("MY USER")
driver.find_element(By.ID, "page1").click()
driver.find_element(By.NAME, "user_password").send_keys("MY PASS")
driver.find_element(By.ID, "register").click()
driver.find_element(By.ID, "myHeader1").click()
driver.find_element(By.ID, "smiles").click()
driver.find_element(By.ID, "smi").click()
driver.find_element(By.ID, "smi").send_keys("CC1(C)C(O)CC[C@@]2(C)C1CC[C@]3(C)C2CCC4[C@@]3(C)CC[C@]5(C(O)=O)C4[C@H](C)C(C)=CC5")
driver.find_element(By.CSS_SELECTOR, "#myContent4 input:nth-child(4)").click()

soup = BeautifulSoup(driver.page_source,'html.parser')
items = soup.find_all('a', class_='Antineoplastic')

print(items)

driver.quit()

edited Nov 28, 2021 at 12:28

answered Nov 28, 2021 at 11:11

HedgeHog

25.4k5 gold badges18 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Gogi Over a year ago

Thank you for the feedback ! I am sorry, but I am very dumb and never messed with programming before and can't really understand what you meant here: @Perform your selenium actions and walk to the website you want to scrape. In next step push your driver.page_source into BeautifulSoup and find your elements:@ @... , but what you are doing is to call parse() that only performce a call via requests@ But i have imported library requests or is it a problem using several libraries? At least I understood that code generated by Selenium should be good... Still trying to check it

Gogi Over a year ago

The selenium part confirmed via chromedriver! Atleast i can login

Gogi Over a year ago

Thank you for your code, though it din't worked I believe because of script delay With greatest help of my friend it worked with driver.implicitly_wait(10) ##

undetected Selenium · Accepted Answer · 2021-11-28 15:29:02Z

0

When using python-class you have to take care of a few things:

Once you define the class methods, you have to create an instance of the class within main to invoke the methods as follows:
```
test1 = Test1()
test1.setup_method()
```
While defining the method, the arguments needs to match with the number/type of arguments you pass as parameters from the main. As an example:
```
def test_1(self):
```
Should be called as:
```
test1.test_1()
```

This usecase

From your entire code block I have taken a few blocks and modified a bit to demonstrate the implementation/usage of Classes:

Code Block:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service

class Test1():
  def setup_method(self):
    print("Within setup_method")
    s = Service('C:\\BrowserDrivers\\chromedriver.exe')
    self.driver = webdriver.Chrome(service=s)


  def teardown_method(self):
    print("Within teardown_method")
    self.driver.quit()

  def test_1(self):
    print("Within test_1") 
    self.driver.get("http://way2drug.com/passonline/")


test1 = Test1()
test1.setup_method()
test1.test_1()
test1.teardown_method()

Console Output:

Within setup_method
Within test_1
Within teardown_method

References

You can find a couple of relevant detailed discussion in:

How to execute testcases through Python Class using Selenium

answered Nov 28, 2021 at 15:29

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

2 Comments

Gogi Over a year ago

Thank you for the answer and the classes guide I admit that I messed up with classes, but for it was like using modern rifle skipping the gunpowder As I understood the classes are very scalable option in coding

undetected Selenium Over a year ago

@Gogi Reread your question once again: the main issue i just cant get into the site(the website is free) Can you look over my code? When i run it, i get ... Process finished with exit code 0 I thought of addressing your Process finished with exit code 0 first to give you a head start. Good luck.

Collectives™ on Stack Overflow

How to call a method within a Python class using Selenium and Python

2 Answers 2

What happens?

How to fix?

EDIT

3 Comments

This usecase

References

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

What happens?

How to fix?

EDIT

3 Comments

This usecase

References

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related