Uploading Large Files to Dropbox using Dropbox API and Python Results in Error

Question

I am trying to upload a large zipped file to Dropbox (about 2-3GB) using Python and the Dropbox API v2. I am using the "chunked method" found here: dropbox API v2 upload large files using python. Here is the code again for convenience:

f = open(file_path)
file_size = os.path.getsize(file_path)

CHUNK_SIZE = 4 * 1024 * 1024

if file_size <= CHUNK_SIZE:

    print dbx.files_upload(f, dest_path)

else:

    upload_session_start_result = dbx.files_upload_session_start(f.read(CHUNK_SIZE))
    cursor = dropbox.files.UploadSessionCursor(session_id=upload_session_start_result.session_id,
                                               offset=f.tell())
    commit = dropbox.files.CommitInfo(path=dest_path)

    while f.tell() < file_size:
        if ((file_size - f.tell()) <= CHUNK_SIZE):
            print dbx.files_upload_session_finish(f.read(CHUNK_SIZE),
                                            cursor,
                                            commit)
        else:
            dbx.files_upload_session_append(f.read(CHUNK_SIZE),
                                            cursor.session_id,
                                            cursor.offset)
            cursor.offset = f.tell()

However, when I run it I am getting a

dropbox.exceptions.APIError

as well as

UploadSessoionLookupError

and a

UploadSessionOffsetError

I think the error might be occurring at this line specifically:

 dbx.files_upload_session_append(f.read(CHUNK_SIZE),
                                       cursor.session_id,
                                       cursor.offset)

I've tried swapping that for

dbx.files_upload_session_append_v2(
                f.read(self.CHUNK_SIZE), cursor)

but that didn't work either. Any suggestions?

APIError, UploadSessionLookupError, and UploadSessionOffsetError are just the error types. They should contain more information, so make sure you check the full error for more information on the issue. — Greg
– Greg, Commented Oct 25, 2017 at 18:36
The UploadSessionLookupError gave me "incorrect_offset" while UploadSessionOffesetError gave me "correct_offset = 10000" I still don't know what to do with that information though. — Jess
– Jess, Commented Oct 25, 2017 at 18:43
Did you also post on the Dropbox forum? I just replied there with more information on that: dropboxforum.com/t5/API-support/… — Greg
– Greg, Commented Oct 25, 2017 at 19:26

taras · Accepted Answer · 2019-03-28 11:04:55Z

0

On windows, be sure to open using binary mode

f = open(file_path, 'rb')

f.read(chunk_size) and f.tell() were off.

from python docs

On Windows, tell() can return illegal values (after an fgets()) when reading files with Unix-style line-endings. Use binary mode (‘rb’) to circumvent this problem.

edited Mar 28, 2019 at 11:04

taras

6,93510 gold badges46 silver badges54 bronze badges

answered Oct 26, 2018 at 14:54

Antonio Urquidi

1

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Uploading Large Files to Dropbox using Dropbox API and Python Results in Error

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related