Custom 'with open()' statement in Python: generator didn't yield error

Question

I have a class for a file, from which you can parse data, write data etc. I want to use it from any application like this:

f = MyFileClass() # __init__ method puts a lot of default data in object
with f.open() as file: # where f.open() is custom MyFileClass method
    file.write("foo") # file should close automatically after this

I tried this:

# it's in MyFileClass()
from contextlib import contextmanager
@contextmanager
def open(self):
    try:
        return open(os.path.join(self.directory, self.name), self.flags)
    except Exception as e:
        print(e.__traceback__)

but I got after running first code

line 22, in fill_new_file with f.open() as file: File "C:\Python34\lib\contextlib.py", line 61, in __enter__ raise RuntimeError("generator didn't yield") from None RuntimeError: generator didn't yield

I guess it's not how contextmanagers works. How to do what I want?

I'd wouldn't call that custom context manager open to avoid confusion — mbatchkarov
– mbatchkarov, Commented Aug 22, 2014 at 10:52

Mikko Ohtamaa · Accepted Answer · 2014-08-22 10:48:38Z

9

Try this:

@contextmanager
def open(self):
    try:
        yield open(os.path.join(self.directory, self.name), self.flags)
    except Exception as e:
        print(e.__traceback__)

Context managers are generators, not functions.

answered Aug 22, 2014 at 10:48

Mikko Ohtamaa

85k63 gold badges296 silver badges479 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

metatoaster Over a year ago

Should really do something like with open(...) as f: then yield f so that the file gets closed as intended, otherwise the file will get opened multiple times if the MyFileClass.open gets called multiple times.

Mikko Ohtamaa Over a year ago

I agree: One should use class-based context manager where you close the file in __exit__ docs.python.org/dev/library/…

Andrzej Sołtysik Over a year ago

@MikkoOhtamaa I want to use file like I mentioned in my first code (what I mean by that is simple usage), eg. with f.open_file() as f:). How should I write this then? Use with statement in my contexmanager, as @metatoaster wrote?

metatoaster · Accepted Answer · 2014-08-22 12:45:43Z

Was going to comment but things turn out to be too complicated to leave there, but I do have an answer.

The corrected version of code can be reduced to essentially this

@contextmanager
def myopen(path):
    try:
        yield open(path)
    except Exception as e:
        print(e.__traceback__)

Before we try, let's get a count of open file handles using this:

>>> os.listdir('/proc/self/fd')
['0', '1', '2', '3']

Now use our context manager

>>> with myopen('/tmp/a_file') as f:
...     print(f.read())
...     print(os.listdir('/proc/self/fd'))
... 
Contents of file

['0', '1', '2', '3', '4']

Yup, file descriptor count increased, but now that we are out of our context manager, let's see

>>> print(os.listdir('/proc/self/fd'))
['0', '1', '2', '3', '4']

Uh that defeats the purpose of having a context manager for the file (we want to use the default autoclosing function, so restart the interpreter, and try this.

@contextmanager
def myopen(path):
    try:
        with open(path) as f:
            yield f
    except Exception as e:
        print(e.__traceback__)

Rerun our tests

>>> with myopen('/tmp/a_file') as f:
...     print(f.read())
...     print(os.listdir('/proc/self/fd'))
... 
Contents of file

['0', '1', '2', '3', '4']

Now outside the context manager

>>> print(os.listdir('/proc/self/fd'))
['0', '1', '2', '3']

Yay, looks like it worked (file is successfully closed), but what about a path that does not exist to see that exception handling?

>>> with myopen('/tmp/no_path') as f:
...     print(f.read())
... 
<traceback object at 0x7f6b64b618c8>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/contextlib.py", line 61, in __enter__
    raise RuntimeError("generator didn't yield") from None
RuntimeError: generator didn't yield

The exception block did fire, note that traceback object, but then since the context manager did not correctly yield, a different exception is raised like what you saw before. I don't know what to recommend, but what I recommend is log the error (using a logger) and then reraise the exception. You could consider returning a dummy object of some kind that will then raise the exception when read, or return nothing, but you need to decide on what exactly is best for your case.

Collectives™ on Stack Overflow

Custom 'with open()' statement in Python: generator didn't yield error

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related