1

I am trying to speed up the process of writing data to an excel file using python in my code. The PyExcelerate module has a good benchmark in terms of writing bulk data to files compared to other python modules.

  • Dimensions: Rows = 10000 Cols = 50

  • Times:

    • pyexcelerate : 10.11
    • xlwt : 15.67
    • xlsxwriter (optimised): 19.70
    • xlsxwriter : 23.50
    • openpyxl (optimised): 95.82
    • openpyxl : 95.90

Also, writing data in batches improves the time taken to write furthermore. See link below.

Speed up writing to files

Now, what I am trying to do is add data to a list as and when it is computed, and when the list size is equal to 500 or when 500 rows of data have been computed, write to the file.

wb = Workbook() ws = wb.new_sheet("test") ws.range("B2", "C3").value = [[1, 2], [3, 4]] wb.save("output.xlsx")

Is there a way that we could append the data for 500 rows in a batch?

Improve this question
2
  • 2
    If you install lxml then you'll get better results with openpyxl. As long as you're working with purely numerical data, PyExcelerate is probably the fastest module. Adding comfort usually comes with a performance cost but you might want to look at the write_row() method of xlsxwriter. Commented Apr 10, 2015 at 13:32
  • Thanks this is almost the solution that I required. I now just want to write the data in Batches of 500 using the write_row() Commented Apr 10, 2015 at 14:25

1 Answer 1

Reset to default
1

Use PyExcelerate its pretty fast then write_row from xlsxwriter and append from openpyxl .Refer https://pypi.python.org/pypi/PyExcelerate

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.