-1

To explain what I'm trying to accomplish.

Sometimes I visit website1.com, here I can download Linux ISOs but it's a bit annoying because it has a lot of ads, some of which are launched via javascript when I click the download button.

Sometimes I visit spotify.com but I hate green, so I'd like to modify all the images and css to make them red.

Sometimes I visit my local digital newspaper paper.com but they always ask something something pay to keep reading, something something disable adblock, usually I just inspect element and remove the overlay and keep reading.

Well, I thought I could make a mitmproxy script that runs 24/7 that would do all these things for me.

My idea was to create a main script that loads all modules found in proxy_modules/ like website1.py, reddit.py and paper.py.

Some time ago I made a web scrapper to download articles, I had multiple websites coded but in proxy_modules/ (called ArticleDownloader) I had a __init__.py


__all__ = [
    "Website1Downloader",
    "Website2Downloader",
    "Website3Downloader",
    "Website4Downloader",
    "Website5Downloader",
    "Website6Downloader",
    "Website7Downloader",
    "Website8Downloader",
    "Website9Downloader"
    ]

from .Website1Downloader import Website1Downloader
from .Website2Downloader import Website2Downloader
from .Website3Downloader import Website3Downloader
from .Website4Downloader import Website4Downloader
from .Website5Downloader import Website5Downloader
from .Website6Downloader import Website6Downloader
from .Website7Downloader import Website7Downloader
from .Website8Downloader import Website8Downloader

Then in the main file I would from ArticleDownloader import WebsiteDownloader1 as ArticleDownloader and the rest of the code was the same for all website, since all WebsiteDownloader# classes would have the same methods and return the same things.

The "problem" is that for every Website I have to modify __init__.py to add the new website and then the main script where I import it.

With my mitmproxy project, is there a way to just add the new module and have the main script pick up all the files in proxy_modules/ and load them automatically.

If so perhaps this could help me for any future projects where I face the same "problem".

Thanks.

Improve this question
10
  • You seem to be describing GreaseMonkey. It has built up a fairly mature ecosystem over the last two decades. Commented Jan 26, 2024 at 16:21
  • Not really, I want to use mitmproxy and modify requests before they're received by the browser/after they've left the browser. Additionally my solution could go "between the router and many PCs" while Greasemonkey would need to be installed in multiple PCs. Commented Jan 26, 2024 at 16:41
  • I've been reading and what I need is mitmproxy to act as a transparent proxy (Most likely in a VM in my host machine) and then set the host machine with the VM's IP as gateway. Commented Jan 26, 2024 at 18:37
  • 1
    How exactly are you going to modify encrypted SSL requests? While in theory you can do this, do you know enough to set up your own certificate authority? Commented Jan 28, 2024 at 17:32
  • 1
    Is there a reason this won’t work? Commented Jan 29, 2024 at 4:48

1 Answer 1

Reset to default
0

Import Python modules dynamically

import importlib

module = importlib.import_module('my_package.my_module')
my_class = getattr(module, 'MyClass')
my_instance = my_class()

The time and location to load can be determined with a filesystem watchdog

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.