unknown error: DevToolsActivePort file doesn't exist when running in Docker

I am trying to run chrome using Python and selenium in a Docker container running alpine. It was running fine until one day when it started throwing the following error when I instantiated chrome.

Message: unknown error: Chrome failed to start: crashed.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/lib/chromium/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

Code

Here is the code that creates this error:

from selenium import webdriver


def generate_plugin():
    pluginfile = 'proxy_auth_plugin.zip'

    # manifest_json, background_js same as https://stackoverflow.com/a/61764363/9809865

    with zipfile.ZipFile(pluginfile, 'w') as zp:
        zp.writestr("manifest.json", manifest_json)
        zp.writestr("background.js", background_js)
    return pluginfile


chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications": 2}
chrome_options.add_experimental_option("prefs", prefs)
plugin = generate_plugin()
chrome_options.add_extension(plugin)
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument("--window-size=1920,1080")
chrome_options.add_argument("--remote-debugging-port=9222")
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("https://google.com")

The full traceback of the exception it generates

Full traceback

driver = webdriver.Chrome(chrome_options=chrome_options)
|   File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
|     desired_capabilities=desired_capabilities)
|   File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
|     self.start_session(capabilities, browser_profile)
|   File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
|     response = self.execute(Command.NEW_SESSION, parameters)
|   File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
|     self.error_handler.check_response(response)
|   File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
|     raise exception_class(message, screen, stacktrace)

Message: unknown error: Chrome failed to start: crashed.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/lib/chromium/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

I am using pyvirtualdisplay for the display for webdriver but even if I add the --headless option in my code, it throws the same error.

Dockerfile

This is my Dockerfile:

FROM python:3.6-alpine

RUN apk add xvfb

# update apk repo
RUN echo "http://dl-4.alpinelinux.org/alpine/v3.7/main" >> /etc/apk/repositories && \
    echo "http://dl-4.alpinelinux.org/alpine/v3.7/community" >> /etc/apk/repositories

# install chromedriver
RUN apk update
RUN apk add chromium=93.0.4577.82-r2 --repository=http://dl-cdn.alpinelinux.org/alpine/v3.15/community chromium-chromedriver=93.0.4577.82-r2

/usr/lib/chromium # ./chrome --version
Chromium 93.0.4577.82
/usr/lib/chromium # ./chromedriver --version
ChromeDriver 93.0.4577.82 (e3a25d9b9e2d0b728e045ec87c0aa4942aa46e4e-refs/branch-heads/4577@{#1237})
/usr/lib/chromium #

Original issue

I was originally using RUN apk add chromium=86.0.4240.111-r0 --repository=http://dl-cdn.alpinelinux.org/alpine/v3.13/community chromium-chromedriver=86.0.4240.111-r0

It had been running fine until it started throwing that exception. So I went to my docker container and tried to run Chrome manually. It threw this error

Error relocating ./chrome: hb_subset_input_set_drop_hints: symbol not found
Error relocating ./chrome: hb_subset_input_set_retain_gids: symbol not found
Error relocating ./chrome: hb_subset: symbol not found

Once I upgraded the Chrome version to 93.0.4577.82, I can manually start both Chrome and Chromedriver, but running the code produces the same traceback.

OS and Docker version

I am using Macbook Pro M1 (2020) with MacOS 12.1

Docker version is the latest: 4.3.2 (72729)

Things I have tried

I have looked at almost all the other questions asked on StackOverflow and other forums but none of the advices have worked for me till now.

• I tried all the Chrome and Chromedriver versions available on alpine from 72.0.3626.121-r0 to the edge version

• I have even purged all the images, containers, and volumes from Docker and reinstalled it. It did not help either.

Update:

Passing the --headless argument makes it run. This makes me assume that the issue is somewhere related to using pyvirtualdisplay

Update 2

I missed to add that I was creating an extension zip file to add it to Chrome on the fly.

Here is what's happening now:

1. Pass the argument "--headless" and disable adding extension, it works completely fine.
2. Pass the argument "--headless" and enable adding extension, it throws, understandably selenium.common.exceptions.WebDriverException: Message: Message: unknown error: failed to wait for extension background page to load: chrome-extension://ilkejjjhcpjkcecgdkkdfaocminnclpo/_generated_background_page.html
3. Don't pass the --headless argument and disable extension, it works completely fine
4. Don't pass the --headless argument and enable extension, it throws the same exception as in point 2.