How can I plot more 10k points using matplotlib?

Question

I'm trying to plot spectrum contains ~11 000 points, so my program hangs (past my code line), below my code. How can I fix it?

import matplotlib.pyplot as plt

try:
    spectrum_file = open('external_data/spectrum.txt', 'r')
    spectrum_data = spectrum_file.read()
    print('Spectrum has imported successfully')
except:
    print('Error of importing spectrum')

try:
    spectrum_data = spectrum_data.split('\n')
    x = [row.split('\t')[0] for row in spectrum_data] # ['1', '2', ... , '11 000']
    y = [row.split('\t')[1] for row in spectrum_data]
    print('Spectrum data has separated successfully')
except:
    print('Error of separating the spectrum data')

plt.plot(x, y, label='Spectrum')
plt.show()

Maybe don't read the whole file in (spectrum_file.read()), rather iterate directly though it. This will use less RAM. — SiHa
– SiHa, Commented Jul 24, 2020 at 8:57
Related for way more points: stackoverflow.com/questions/5854515/… — Ciro Santilli OurBigBook.com
– Ciro Santilli OurBigBook.com, Commented Jan 8 at 12:06

JohanC · Accepted Answer · 2020-07-24 12:41:44Z

Before considering that matplotlib wouldn't be able to plot more than 10,000 points, you really should check your code. In fact, matplotlib can plot millions of points without problems. You could check your code by limiting the plot to the first 100 or so points and note strange behavior.

The main problem in the code is that strings are read into x and y arrays. Matplotlib tries to plot them as if they were names, not numbers. With 10,000 points the code is extremely slow in trying to cope with too many strings.

The solution is to convert the strings:

x = [float(row.split('\t')[0]) for row in spectrum_data]
y = [float(row.split('\t')[1]) for row in spectrum_data]

Instead of reading the file "by hand", it is usually recommended to use specialized functions, such as read_csv from pandas. Or loadtxt from numpy. Such functions automatically convert the fields while looping through the file.

xy = np.loadtxt('external_data/spectrum.txt')
x = xy[:,0]
y = xy[:,1]

PS: Here is an example of matplotlib with 10 million points:

import matplotlib.pyplot as plt
import numpy as np

x = np.random.normal(0, 1, 10000000)
y = np.random.normal(0, 1, 10000000)
plt.plot(x, y, ls='', marker=',', markeredgecolor='none')

plt.show()

Collectives™ on Stack Overflow

How can I plot more 10k points using matplotlib?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related