UnicodeDecodeError on attempt to save file through django default filebased backend

Question

When i attempt to add a file with russian symbols in name to the model instance through default instance.file_field.save method, i get an UnicodeDecodeError (ascii decoding error, not in range (128) from the storage backend (stacktrace ended on os.exist). If i write this file through default python file open/write all goes right. All filenames in utf-8. I get this error only on testing Gentoo, on my Ubuntu workstation all works fine.

class Article(models.Model):
    file = models.FileField(null=True, blank=True, max_length = 300,
                            upload_to='articles_files/%Y/%m/%d/')

Traceback:
File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  100.                     response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  24.                 return view_func(request, *args, **kwargs)
File "/var/www/localhost/help/wiki/views.py" in edit_article
  338.                 new_article.file.save(fp, fi, save=True)
File "/usr/lib/python2.6/site-packages/django/db/models/fields/files.py" in save
  92.         self.name = self.storage.save(name, content)
File "/usr/lib/python2.6/site-packages/django/core/files/storage.py" in save
  47.         name = self.get_available_name(name)
File "/usr/lib/python2.6/site-packages/django/core/files/storage.py" in get_available_name
  73.         while self.exists(name):
File "/usr/lib/python2.6/site-packages/django/core/files/storage.py" in exists
  196.         return os.path.exists(self.path(name))
File "/usr/lib/python2.6/genericpath.py" in exists
  18.         st = os.stat(path)

Exception Type: UnicodeEncodeError at /edit/
Exception Value: ('ascii', u'/var/www/localhost/help/i/articles_files/2010/03/17/\u041f\u0440\u0438\u0432\u0435\u0442', 52, 58, 'ordinal not in range(128)')

It's usual class Article(models.Model): ... file = models.FileField(null=True, blank=True, max_length = 300, upload_to='articles_files/%Y/%m/%d/') — Ivan Kuznetsov
– Ivan Kuznetsov, Commented Mar 16, 2010 at 21:07
I think that problem in os difference, on all my ubuntu workstation all works fine. — Ivan Kuznetsov
– Ivan Kuznetsov, Commented Mar 16, 2010 at 21:17
Full stacktrace on my office pc, but it's not so helpful in this case (i call save method, it calls storage save method, i'll post it tomorrow). — Ivan Kuznetsov
– Ivan Kuznetsov, Commented Mar 16, 2010 at 21:21

Antony Hatchkins · Accepted Answer · 2015-09-03 15:24:15Z

4

The solution is quite simple:

In revision 12659 this bug was fixed. http://code.djangoproject.com/ticket/11030

But revision 12661 reverted it

"(In [12661]) Fixed #11030: Reverted a change that assumed the file system encoding was utf8, and changed a test to demonstrate how that assumption corrupted uploaded non-ASCII file names on systems that don't use utf8 as their file system encoding (Windows for one, specifically). Thanks for the report to vrehak."

So all I need to do is revert to 12659

edited Sep 3, 2015 at 15:24

Antony Hatchkins

34.4k13 gold badges121 silver badges114 bronze badges

answered Mar 17, 2010 at 10:26

Ivan Kuznetsov

1011 silver badge6 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tomasz Zieliński Over a year ago

In fact you need to set proper environment in Apache: code.djangoproject.com/ticket/11030#comment:5

Andrea Over a year ago

Thank you @TomaszZielinski that was indeed the right solution

Daniel Roseman · Accepted Answer · 2010-03-16 21:14:39Z

-2

I suspect it's simply a matter of making sure that the upload_to attribute is unicode:

file = models.FileField(null=True, blank=True, max_length = 300,
                        upload_to=u'articles_files/%Y/%m/%d/')

answered Mar 16, 2010 at 21:14

Daniel Roseman

602k68 gold badges910 silver badges923 bronze badges

Collectives™ on Stack Overflow

UnicodeDecodeError on attempt to save file through django default filebased backend

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related