Convert bytes to a file object in python

Question

I have a small application that reads local files using:
open(diefile_path, 'r') as csv_file
open(diefile_path, 'r') as file
and also uses linecache module

I need to expand the use to files that send from a remote server.
The content that is received by the server type is bytes.

I couldn't find a lot of information about handling IOBytes type and I was wondering if there is a way that I can convert the bytes chunk to a file-like object.
My goal is to use the API is specified above (open,linecache)
I was able to convert the bytes into a string using data.decode("utf-8"),
but I can't use the methods above (open and linecache)

a small example to illustrate

data = 'b'First line\nSecond line\nThird line\n'

with open(data) as file:
    line = file.readline()
    print(line)

output:

First line
Second line
Third line

can it be done?

Aoba K · Accepted Answer · 2023-01-22 12:23:39Z

9

The answer above that using StringIO would need to specify an encoding, which may cause wrong conversion.

from Python Documentation using BytesIO:

from io import BytesIO
f = BytesIO(b"some initial binary data: \x00\x01")

edited Jan 22, 2023 at 12:23

answered Jan 22, 2023 at 12:22

Aoba K

1111 silver badge6 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

chepner · Accepted Answer · 2020-09-16 15:10:16Z

7

open is used to open actual files, returning a file-like object. Here, you already have the data in memory, not in a file, so you can instantiate the file-like object directly.

import io


data = b'First line\nSecond line\nThird line\n'
file = io.StringIO(data.decode())
for line in file:
    print(line.strip())

However, if what you are getting is really just a newline-separated string, you can simply split it into a list directly.

lines = data.decode().strip().split('\n')

The main difference is that the StringIO version is slightly lazier; it has a smaller memory foot print compared to the list, as it splits strings off as requested by the iterator.

edited Sep 16, 2020 at 15:10

answered Sep 16, 2020 at 14:58

chepner

538k77 gold badges594 silver badges746 bronze badges

1 Comment

ItamarG Over a year ago

thank you, so from what I understand, I will have to change the API to something else than 'open'

Collectives™ on Stack Overflow

Convert bytes to a file object in python

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related