0

My loader receives part of the program from the server as .pyc files (the files themselves are NOT saving in client pc's filesystem, the byte sequence is stored in a variable).

I need to create a module from this byte sequence without creating a file.

I found a way to create a module from a string:

spec = importlib.util.spec_from_loader("modulename", loader=None) # creating spec
module = importlib.util.module_from_spec(spec) # creating module
exec(source, module.__dict__) # executing module
#  so that we can import it in other program files
sys.modules[name] = module
globals()[name] = module

But it doesn't work for me in my case.

The module is initially created with py_compile.compile("sorcefile.py", "destfile.pyc"). The answer to a similar question (How to load compiled python modules from memory?) suggests using marshal.loads(bytes[8:]) and indeed, looking at the source code of py_compile.compile function (https://svn.python.org/projects/python/trunk/Lib/py_compile.py) I saw that it uses marshal.dump(). However, as far as I understand, this code is deprecated, because looking at the source code of this function in my Python instance (3.11) (https://github.com/python/cpython/blob/3.11/Lib/py_compile.py), I saw that it no longer uses marshall.

This explains why this error appears in the following situation:

import marshal
import py_compile


py_compile.compile("source.py", "compiled.pyc")
with open("compiled.pyc", "rb") as f:
    module_bytes = f.read()

module = marshal.loads(module_bytes[8:])

Output:

Traceback (most recent call last):
  File "C:\test.py", line 9, in <module>
    module = marshal.loads(module_bytes[8:])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: bad marshal data (unknown type code)

(In this case, the pyc file is created only to emulate the data received from the server.)

So, is it possible to create a module from a sequence of bytes without creating a file in python 3.11 (as in the "string" method).

2
  • Since marshal expects to read and write NYC files, are you sure your should be skipping those initial bytes? Commented Mar 30, 2024 at 23:04
  • @JonSG, The first 8 bytes are the magic number that identifies the python version. The answer to stackoverflow.com/questions/1830727/… says they are not needed. Anyway, even with those 8 bytes, the same error appears. Commented Mar 30, 2024 at 23:09

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.