separator doesn't work while reading csv in Python

Question

I have data which consists 3004 rows without header, and each row has different number of fields (e.g. for row number 1,2,3,4 has 16,17,21,12, respectively). Here is my code when I call the csv.

 df = pd.read_csv(file,'rb', delimiter ='\t', engine='python')

here is the output:

$GPRMC,160330.40,A,1341.,N,10020.,E,0.006,,150517,,,A*7D
$GPGGA,160330.40,1341.,N,10020.,E,1,..
$PUBX,00,160330.40,1341.,N,10020.,E,...
$PUBX,03,20,2,-,056,40,,000,5,U,014,39,41,026,...
$PUBX,04,160330.40,150517,144210.39,1949,18,-6...
ÿ$GPRMC,160330.60,A,1341.,N,10020.,E...
$GPGGA,160330.60,1341.,N,10020.,E,1,...

It seemed like delimiter didn't work at all to separate the data into column by column. Hence, I tried with columns function based on number of fields from ($PUBX, 00). Here is the code when I add columns:

my_cols = ['MSG type', 'ID MSG', 'UTC','LAT', 'N/S', 'LONG', 'E/W', 'Alt', 'Status','hAcc', 'vAcc','SOG', 'COG', 'VD','HDOP', 'VDOP', 'TDOP', 'Svs', 'reserved', 'DR', 'CS', '<CR><LF>']

df = pd.read_csv(file, 'rb', header = None, na_filter = False, engine = 'python', index_col=False, names=my_cols)

and the result be like the picture below. The file becomes into one column in 'MSG type'.

the output

My purpose after success to call this csv is to read rows only with combination between $PUBX, 00,... and one column of $PUBX, 04,... and write it to csv. But, I am still struggling how to separate the file into columns. Please advice me on this matter. Thank you very much.

At a glance your actual separator is a comma, not a tab. Try removing the delimiter='\t' argument. — Ilja Everilä
– Ilja Everilä, Commented May 31, 2017 at 7:14

Mukarram Pasha · Accepted Answer · 2017-05-31 10:03:59Z

2

pd.read_csv

is used for reading CSV(comma separated values) Files hence you don't need to specify a delimiter.

If you want to read a TSV (Tab separated values) File, you can use:

pd.read_table(filepath)

The default separator is tab

answered May 31, 2017 at 10:03

Mukarram Pasha

3681 gold badge4 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

E. Ducateme · Accepted Answer · 2017-05-31 08:17:11Z

1

Hat Tip to Ilja Everilä

@Hasanah Based on your code:

df = pd.read_csv(file,'rb', delimiter ='\t', engine='python')

delimiter='\t' tells pandas to separate the data into fields based on tab characters.

The default delimiter when pandas reads in csv files is a comma, so you should not need to define a delimiter:

df = pd.read_csv(file,'rb', engine='python')

answered May 31, 2017 at 8:17

E. Ducateme

4,2962 gold badges22 silver badges31 bronze badges

Collectives™ on Stack Overflow

separator doesn't work while reading csv in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related