How to create new objects dynamically at runtime without hard coding them?

Question

I am new to python. I'm writing a testing application for another analytical tool. That tool outputs various kinds of files we call 'artifacts'. Each type of artifact is a data file of some kind.

I created a parent class called Artifact that has load() and compare() functions.

So, say I have two types of artifacts: cube and report. Each has an associated Cube and Report class, with functions that know how to read the respective data, and compare one to another (one cube to another cube and one report to another report).

Implementing that is all clear to me.

Here's my question: I'd like users to be able to implement their own new Artifacts that I'm not aware of now, and be able to load them via a config file.

The new child of Artifact would need to be a .py file, but I would like to not have to alter the code of the main test application. So I'd like the users to specify in a config file to use existing Cube and Report artifacts, as well as their newly defined artifact. For all artifacts do a load() and compare()

Can I do this without doing an explicit import of the new class, and without specify an Explicit call to it?

Can I just write

for c in Artifact classes:
    c.load()
    c.compare()

? (that's obviously simplified, I hope it's clear)

So essentially you want to reference classes (that inherit from Artifact) based on user input? Such is my_class = ./CustomArtifact.py ? And YOUR code would need to operate on that class, and not the other way around? — Collin Heist
– Collin Heist, Commented Oct 25, 2020 at 1:42
You are looking to load modules dynamiaclly; check out the importlib module. — chepner
– chepner, Commented Oct 25, 2020 at 1:47

tdelaney · Accepted Answer · 2020-10-25 03:29:01Z

1

The simplest way to do this is to add a subpackage to your project where new artifact processors are checked in as a new .py file. Add the import to __init__.py and it will load automatically.

myproject
    __init__.py
    artifacts.py
    artifactsdb
        __init__.py
        cube.py
        report.py

artifacts.py

class Artifacts:

    def load(self):
        pass

    def compare(self):
        pass

def enum_artifacts():
    for art in Artifacts.__subclasses__():
        art().load()
        art().compare()

artifactsdb/init.py

from .cube import cube
from .report import report

artifacts/cube.py

from ..artifacts import Artifacts
class cube(Artifacts):
    pass

edited Oct 25, 2020 at 3:29

answered Oct 25, 2020 at 2:43

tdelaney

77.9k6 gold badges91 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Pynchia Over a year ago

Uhm.. good idea, but as you wrote them compare and load aren't class methods and art in for art in Artifacts.__subclasses__ is a class, not an instance. Just create obj = art() and call the methods on obj

tdelaney Over a year ago

@Pynchia - Uhm, OP doesn't give a solid definition of what load and compare should be, so I just sketched in something vague. For instance, how should the code select which subclass to use? Some rule about file names? A generic loader that then selects subtypes? Try them all until one doesn't crash? I don't know, so just sketched it in. I neglected to call __subclass__ in the example. I did test but seem to have missed a few bits and bobs while pulling it together.

45th Percentile Over a year ago

Thanks to you both. I think I generally understand the suggestion here, but I'll need to try to implement this to work through it. My python isn't strong enough yet to just visualize this. But it's a very helpful suggestion.

Wups · Accepted Answer · 2020-10-25 07:57:23Z

1

The config file could be a python module, that imports the new artifact classes. If your users are able to write their own python classes, they shouldn't have a problem editing such a config file:

in userconfig.py:

# import user artifact classes:
from myartifact import NewArtifact

# add user artifacts to the list "user_artifacts"
user_artifacts = [
    NewArtifact
]

in your main code:

import userconfig

for c in userconfig.user_artifacts:
    # load and compare

answered Oct 25, 2020 at 7:57

Wups

2,5691 gold badge8 silver badges17 bronze badges

Collectives™ on Stack Overflow

How to create new objects dynamically at runtime without hard coding them?

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related