560 questions
0
votes
0
answers
30
views
Python asyncio script freezes indefinitely using Playwright and Semaphore
I am building a scraper/processor pipeline that iterates over a large list of URLs (~10k). For each URL, I perform the following steps asynchronously:
Scrape HTML using playwright.async_api.
Process ...
- 9
0
votes
1
answer
197
views
Fetch data from https://www.sofascore.com/?
This is my python code using on ubuntu to try fetch and extract data from
https://www.sofascore.com/
I create this test code before using on E2 device in my plugin
# python3 -m venv venv
# source venv/...
- 55
1
vote
2
answers
131
views
How to generate bookmarks in a PDF?
I've been using Playwright to generate a document from HTML code with a table of content corresponding to the H1-6 tags I'm using. I was hoping that bookmarks in the PDF would be generated from those ...
- 11
0
votes
1
answer
112
views
How to abort an entire web request if a particular dialog appeared
I am currently trying to write a Playwright script to test a JavaScript page which is intended to produce several alert boxes. Because the testing is being done for beginners who would not understand ...
- 1,362
2
votes
2
answers
121
views
Python script for Playwright - final page in list causes "domcontentloaded" error
I'm trying to loop through a list of URLs checking if the pages contain the words "no longer available".
I tested this out with a list of Wikipedia pages and it worked perfectly.
When I use ...
- 53
1
vote
1
answer
65
views
How do I properly track authentication across pages from pff.com Domain
I am using Playwright and logging into the main authentication page. When I go to a new page (same domain), it returns results as if I'm not authenticated. If you go to the last URL w/out ...
- 31
0
votes
0
answers
82
views
How can Playwright send the Proxy-Authorization while using Chromium?
I have a proxy server that requires a username and password to access (e.g., 127.0.0.1:8000, username: abcdef, password: 123456).
When I use Playwright to access a destination website through this ...
3
votes
1
answer
224
views
My Playwright python script doesn't find input form fields
So I want to automate adding many team players, I wrote a script to do this with Playwright. As the players are much, the issues is that the script don't seem to find the input fields like the first ...
- 73
1
vote
1
answer
176
views
Using PyTest and Playwright, run one method before loading all the test files
This question seems to be asked a lot, but I'm not finding any answers that work, or at least that work for what I'm looking for.
The premise is simple: I have a website I need to automate testing on (...
- 1,817
0
votes
1
answer
53
views
TypeError: Controller.action() missing 1 required positional argument: 'self' on Controller of AI Agent [closed]
I have an AI Agent for which i have written a controller using Python 3.13 and Playwright.
@Controller.action(description='Get attribute and url of the page')
async def get_attr_url(self, browser: ...
- 41
0
votes
0
answers
82
views
Issues with testing SAP cloud application through Playwright python
I am trying to automate testing for an use case based on SAP Cloud application(Agent Performance Management) using Playwright python package. I am using chrome browser. When I launch the application ...
- 119
0
votes
1
answer
101
views
How to set all <select> elements to a specific value using Playwright sync_api in Python?
I’m using Playwright’s Python sync API (from playwright.sync_api import sync_playwright).
On my page I have multiple <select> elements like this:
<select class="sc-fzXfMv sc-fzXfMw ...
- 37
1
vote
2
answers
127
views
How to wait for element to disappear if you cannot rely on locators
Playwright discourages ElementHandle, but how else would I do the following?
# get a handle to the old message
old_message_element = page.locator("#message").element_handle()
# ...
- 65
1
vote
1
answer
208
views
Python asyncio: execute an asyncronous function while the main thread is blocked [closed]
I'm coding a python project split into two parts:
a PySide6 GUI interface that needs to run on the main thread synchronously, and consequently block it,
a playwright virtual web browser (asynchronous)...
- 304
0
votes
3
answers
319
views
python -playwright mouse pointer UI enable
I want a playwright python mouse UI enable script.
so can i can see my mouse to be working in visual for my humanize automation.
if any one can provide me any method or js script that will work plz ...
0
votes
0
answers
58
views
How to make playwright wait for two elements at the same time and continue after it finds one of them? [duplicate]
I'm using playwright with python. I'm looking to wait for 2 elements. e.g. locator1 and locator2. After playwright sees one of these elements it will return the one it found first then finish the ...
- 29
1
vote
1
answer
143
views
Can't find element by XPATH playwright
I am trying to get all search results (URLs) from https://docs.vgd.ru/search/?v=1. I am using the xpath //a[@class='resultsMain'] to find them. The xpath is valid.
My code:
import asyncio
import time
...
- 1,718
0
votes
0
answers
79
views
Unable to Load Extensions in nodriver Proxy Context
I need to load user profile, extensions and proxy in a single page context.
Looks like it is not possible to do so right now using nodriver, any help/suggestion would be appreciated. Thanks
Right now ...
- 309
0
votes
0
answers
60
views
How to extract debundled JavaScript files via CDP or Playwright
I've been trying to programmatically extract the original, debundled JavaScript source files behind a web app that uses Webpack bundles and source maps. While Chrome DevTools clearly shows the ...
- 19
-2
votes
1
answer
123
views
Detect Http Basic Authentication in Playwright
I am writing a playwright based crawler that encounters various Http Basic Authentication sites.
I would like to detect these sites, abort the crawling, and catalog them as sites that I know to be ...
- 3,313
1
vote
1
answer
117
views
How to receive return value using page.on("response") [closed]
I'm trying to catch contents of response using Playwright Python, but I'm not sure how to achieve this. Here's my sample code.
from playwright.sync_api import Playwright, sync_playwright
def ...
- 11
1
vote
2
answers
560
views
playwright python how to detect browser window is closed
I'm writing a simple Python script using requests.get to scrape data from a website that requires login. The script first checks whether the user is already logged in. If not, it launches a Chromium ...
- 61
3
votes
1
answer
112
views
Wanting to optimize a python playwright proxy applicatoin
I'm writing a Python Flask-based proxy using the playwright library. I've got code that works, but I'd like to make it faster. Here's what I have:
@app.route('/fetch/')
def fetch_url():
url = ...
- 31
-1
votes
1
answer
103
views
How to extract intraday stock data from onvista with python and playwright/selenium?
Using 3.9.6 and playwright on MacOS 14.7.5 I am trying to extract the public available intraday stock data from a webpage like
https://www.onvista.de/aktien/Airbus-Group-EADS-Aktie-NL0000235190
To ...
- 45k
1
vote
1
answer
485
views
How to use browser-use to make screenshots for each step and video recording for the whole session?
I am trying to build a python project that uses browser_use to perform tasks, and create video recording for the whole session and screenshots for each step.
But I can't find a way to do it. The ...
- 1,969
0
votes
2
answers
218
views
How to locate elements simultaneously?
By nature, Playwright locator is blocking, so whenever it's trying to locate for an element X, it stops and waits until that element is located or it times out.
However, I want to see if it is ...
- 286
2
votes
1
answer
155
views
How to programmatically zoom in on a webpage and improve <canvas> resolution (like Ctrl + does)? [closed]
In some webpages where there is a <canvas> element, when i tried every single method of making the browser bigger, the page bigger... I did found some methods that will make everything big, but ...
- 31
1
vote
0
answers
131
views
How to disable certificate invalid error in UI while automating using Python Playwright
Using Pytest with Playwright, I have below lines to ignore ssl certificate invalid error but its not working. Can someone please help resolve this issue?
with sync_playwright() as p:
browser = ...
1
vote
0
answers
327
views
How to resolve Playwright missing dependencies error on Replit
I'm trying to use Playwright in a Python project on Replit. I installed it using pip install playwright, and then ran playwright install. The download of Chromium, Firefox, and WebKit seemed to ...
- 45
0
votes
0
answers
112
views
RSS Memory and Virtual Memory do not decrease even after killing a playwright instance and creating a completely new one
I'm doing continuous iteration tests with playwright on a website and I noticed that RSS Memory and Virtual Memory are gradually increasing and no direct method is managing to dissipate the memory ...
- 2,222
0
votes
1
answer
149
views
Use anyc_playwright within multithreaded Prefect flow (`_channel` seems to be `None`)
I am having trouble using Playwright in my Prefect flow. The flow has to scrape ~800 urls and therefore splits them into chunks of 200 and runs those chunks in parallel. The urls within a chunk are ...
- 33
0
votes
1
answer
272
views
Slow Python Async Playwright tests when running with with async_playwright() as p:
New to writing tests in playwright with Python, I am currently attempting to uplift our current TypeScript playwright tests into Python Playwright.
To begin with I have been experimenting with the ...
1
vote
1
answer
269
views
How to configure Playwright so that the Chrome inspector opens at the bottom instead of the right?
Since I often open the inspector while debugging tests in Playwright, I'd like to know if there's a way to make it open at the bottom when pressing F12 instead of to the right.
I know I can change the ...
- 12k
0
votes
0
answers
41
views
When running all tests via pytest command line with xdist -n 4 option with tracing retain-on-failure, when are the trace files deleted?
I run all my test on Windows via this command in the root of my tests folder:
pytest --tracing retain-on-failure -v -n 4 --dist loadfile -m "my_marks"
Everything has always been fine. I ...
- 25
1
vote
1
answer
107
views
How to automatically start the debugging session in Playwright?
I want to automatically start the debugging session instead of having it starting as paused.
I found out that adding this code makes it work but it feels hackish to me:
context.add_init_script("...
- 12k
0
votes
0
answers
346
views
Playwright Inspector not opening with tests
The following command does run the tests in the browser window, but it does not open the Inspector as per the instructions: https://playwright.dev/python/docs/running-tests#debugging-tests
PWDEBUG=1 ...
- 1,363
1
vote
2
answers
333
views
Playwright's set_input_files() function doesn't work properly in python [closed]
I'm writing tests for my Django project using playwright and StaticLiveServerTestCase. In one test I need to choose two files using input[type=file] field and send them to the server.
file_input = ...
- 11
-1
votes
1
answer
67
views
Python + Playwright browser automation
This part of the code responsible for the behavior launches the profile, prints a query in the search engine, goes to the query page, but freezes on it and does not do any more actions. Then he closes ...
- 1
0
votes
1
answer
56
views
How to get the path to webelement from shadow DOM for further use in Selenium/Playwright Python auto-tests?
I feel difficulties to extract the locator for "Accept all" button from usercentrics dialog, the DOM is the following:
DOM
and
UI
I am using the following:
def user_centrics_accept(driver):
...
0
votes
1
answer
146
views
pytest get playwright executable_path and version
How can I get the playwright browser objects executable_path and version in pytest where I only have a Page object?
I can get it from the garbage collector referrers list but it is ugly.
(I need it ...
- 3,747
0
votes
1
answer
142
views
How to bypass "Too many open files" error when using Playwright?
I'm using Playwright to crawl websites. I have a scraping function that uses Playwright, and implemented a Python object that uses this function to crawl websites in a bread-first search manner.
Below ...
- 3,460
0
votes
1
answer
74
views
how to handle web scraping error beacuase of its data type
I'm starting web scraping with python and playwright. but when I run the code I get an error .how to find type of response data(binary,text,...) and handle the error about convert and save data as ...
- 21
2
votes
0
answers
114
views
PlayWright (python)- Failed to install browsers designated as channels (i.e. chrome-beta) in Powershell / Windows 11
I am working on cross-platform tools that use PlayWright. The tools need both the managed browsers and production browsers (seem to be denoted as channels). Installing channels work fine in MacOS ...
- 21
0
votes
0
answers
75
views
How to add browser type as a CLI argument to custom Browser in Playwright Pytest
I have my own Browser class and created a new CLI argument to be able to execute tests with different browsers.
main conftest.py
def pytest_addoption(parser):
parser.addoption('--browser_type', ...
- 339
0
votes
0
answers
30
views
Playwright exception: no description
I have a strange error with no description when I try to use async playwright
This is a FastApi app and I am on Windows
In the docs there is a mention that incompatible with SelectorEventLoop of ...
0
votes
2
answers
312
views
Playwright container cannot reach web service container in GitLab CI/CD
I am trying to run Playwright tests within a GitLab CI/CD pipeline against a web service running in a Docker container. My gitlab-ci.yml looks like this:
stages:
- test
services:
- docker:27-dind
...
- 161
-1
votes
1
answer
102
views
scrapy booking with playwright-python return an error
I'm using scrapy and playwright to scrape booking.com
in this way I need to click on a button and get ajax response.
but when I run my code it returns error :
TypeError: Page.locator() missing 1 ...
- 21
0
votes
2
answers
131
views
Playwright Python can't find HTML tag which shows up in debugger and in a print statement
I am trying to scrape a page product detail page
but I am not able to find the tag when the code runs. I print the parent tag out, and I see the h2 tag I want, and also when I enter the debugger I can ...
1
vote
1
answer
94
views
can't find correct 'select' HTML tag value, and trying to wait for a select option to load, playwright Python
I have an issue where I use a url that ends such as T-shirts page
I am trying to scrape the product links off the pages. I have been trying for some time now, nothing is working yet. This is my ...
1
vote
1
answer
131
views
TypeError: Failed to fetch when uploading CSV file using Playwright and Python
Environment:
Python with Playwright
Running on localhost:8000
Chrome browser in headed mode
Problem:
I'm trying to upload a CSV file using Playwright with Python, but I'm getting the following errors ...
- 11