Python CSV error: line contains NULL byte, but no NULL byte found in the file

Question

I am trying to read some txt files, tab-delimited data and using the code below, it shows me the following error:

for row in reader:

Error: line contains NULL byte

The code is basically trying to to get some information from specific columns. An abridged version of it below:

with open('07table.txt', encoding = "utf-8", errors='ignore') as fo:
    reader = csv.reader(fo, delimiter='\t', quoting=csv.QUOTE_MINIMAL)

For the past week, I tried to investigate if the files contain NULL bytes. I've used the code below to check whether this is true, and to my surprise I didn't find any NULL bytes in the files:

if '\x00' in open('07table.txt').read():
    print("you have null bytes in your input file")
else:
    print("you don't")

Just to be on the safe side, I also ran a code that replaces NULL bytes in my txt files, but I still have the same error.

Another thing that I have tried so far is to change to an older version of Spyder and downgrade the Python version to 3.5.14 because I thought it might be an issue with the environment I have installed. This has had some effects in the sense that my code successfully reads two files that were previously supposed to contain NULL bytes. However, when it gets past these 2 files, the code renders the same error as posted above.

At this point, I am very sure that the files do not contain NULL bytes. Any thoughts on why I could get this error?

Have you tried this link? stackoverflow.com/questions/7894856/…. Which mentions trying to open the file with utf-16 encoding. — apoorva kamath
– apoorva kamath, Commented Jun 2, 2020 at 19:02
Thank you for your reply! I forgot to mention, but I actually tried that suggestion. For every file I try to decode using UTF-8, UTF-16, or latin-1. Most of the files are encoded in UTF-8. — TheWalküre
– TheWalküre, Commented Jun 2, 2020 at 19:25

TheWalküre · Accepted Answer · 2021-04-25 16:36:27Z

0

I found what the issue was. I was reading the files from an external hard drive formatted in NFTS, while the code was running on a macOS formatted in HFS.

After formatting the external drive to match the formatting on my laptop, the problem of null bytes disappeared.

answered Apr 25, 2021 at 16:36

TheWalküre

454 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python CSV error: line contains NULL byte, but no NULL byte found in the file

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related