0

I am trying to write img_id (single value) in first column of csv and image features array (f_descriptor_T) in remaining columns of csv file. I have written following code: 

path=/home/dir
listing = os.listdir(path)    

for file in listing:
    path_img_ext = os.path.join(path, file) #combine filename and extension
    data_read = feature_io.ReadFromFile(path_img_ext) #read single features of single image in each iteration 

    base=os.path.basename(file) #Extract image_id from its name
    os.path.splitext(file)
    img_id = os.path.splitext(base)[0]

    f_descriptor = data_read[2] #Extract particular array, from multiple arrays of features 

    feature_flat=f_descriptor.flatten() #Make flat array

    f_descriptor_T = np.array(feature_flat).T #Transpose feature array to convert col data to row form. 

    with open("output_0.csv", "a") as f:
        writer = csv.writer(f)
        writer.writerow([img_id])
        writer.writerow(f_descriptor_T)

Unfortunately, the output shows img_id on one row and f_descriptor_T on second row, with each iteration of loop, as shown in following screenshot: Output

But i want to combine both (variable and array) in single row.

Note: I have tried zip and concatenate to combine both, but failed.

The desired output should be like this: Desired OutputNot necessary to print column header, but just img_id and f_descriptor_T array in single row.

Improve this question
7
  • You array f_descriptor_T is an array of three rows. What do you mean you want them in one row? Commented May 11, 2018 at 1:40
  • I wouldn't use a csv.writer to write arrays. Experiment with np.savetxt for writing arrays. Better yet, study its code to see how it writes rows of an array directly (with out csv module). Commented May 11, 2018 at 1:40
  • @Dyz it convert feature vector to ["[-0.00369344 -0.08348401 0.29953018 ... 0.38050437 0.01184894], and save ... in csv, not whole numpy array of features. Commented May 11, 2018 at 1:42
  • @hpaulj I want to save it in csv format for further processing, and also i have one variable (id) and one array (feature), which i want to combine. Commented May 11, 2018 at 1:44
  • For further processing? by what? And what is the csv supposed to look like when saved? Commented May 11, 2018 at 1:53

2 Answers 2

Reset to default
2

Based on your current code, f_descriptor_T appears on a different row because you are providing the f_descriptor_T in a different writerow call to the rest of the data (every call to writerow creates a new row).

Note that the writerow function takes the data-to-insert as a single parameter in the form of a list.

If you want them in the same row, you will need to provide them in the same call. To do it in the way you show in your 'Desired Output' image, you'll need to do provide the writer with a single, flat list.

Since you have two different data types (the img_id is a string, and f_descriptor_T contains numbers), I recommend doing this by creating an array containing your img_id, and then adding the contents of f_descriptor_T to it. You will need to convert f_descriptor_T into a normal Python List. For the conversion, you can use the list function.

Here is a minimal set of examples to demonstrate the behaviour.

You want to do it like example 4. It is similar to example 2, but includes an example of how to convert a numpy array into an ordinary List.

import csv
import numpy as np

#example 1
with open("output_different_rows.csv", "w") as f:
    writer = csv.writer(f)
    writer.writerow(['foo'])
    writer.writerow(['bar'])

#example 2
with open("output_same_row.csv", "w") as f:
    writer = csv.writer(f)
    writer.writerow(['foo'] + ['bar'])
#example 3
with open("output_same_row_with_array.csv", "w") as f:
    writer = csv.writer(f)
    writer.writerow([['foo', 'bar'], 'too'])

#example 4
numpy_array = np.arange(0,9)
listified_array = list(numpy_array)
with open("output_same_row_numpy.csv", "w") as f:
    writer = csv.writer(f)
    writer.writerow(['foo'] + listified_array)
Improve this answer
Sign up to request clarification or add additional context in comments.

10 Comments

writer.writerow([[img_id], f_descriptor_T]), if write this line, but still f_descriptor_T loss is values and showing ... like this ["[-0.1960862 -0.27065492 0.4554646 ... 0.07876456 -0.01247655]. I have two values, img_id and f_descriptor_T, but your example contain 3 values foo, bar and too
Ah yes of course, because you are using numpy-style arrays, it doesn't print commas. In example 3, I used three values primarily to demonstrate the pitfall with commas. SFAIK, You would need to write your own code for parsing the numpy array and thereby splitting f_descriptor_T back into several values. Does that help? If not, could you please be more specific as to what information you are missing/need clarified?
(providing a concrete example of your desired output format would help substantially)
I just want to print img_id and f_descriptor_T in single row, nothing else.
That is not enough information. There are multiple formats of output that could be described as that, potentionally an infinite number. Please give me at least one line of CSV text that shows the exact result you want, with real data, then I can show you how to reproduce it.
|
0

Attempting to reproduce your csv file

In [131]: arr = np.arange(10) # a flat array
In [132]: arr = np.array(arr).T
In [133]: arr
Out[133]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

I start with a flat file, and still have a flat one (.T on a 1d array does nothing)

In [135]: with open('out.csv','a') as f:
     ...:     writer = csv.writer(f)
     ...:     writer.writerow(['id'])
     ...:     writer.writerow(arr.T)
     ...:     
In [136]: cat out.csv
id
0,1,2,3,4,5,6,7,8,9

Each call to writerow writes a line, as the name implies.

But with savetxt I can easily write a 2d array in neat columns and rows:

In [137]: np.savetxt('out.csv', arr.reshape(5,2), delimiter=',')
In [138]: cat out.csv
0.000000000000000000e+00,1.000000000000000000e+00
2.000000000000000000e+00,3.000000000000000000e+00
4.000000000000000000e+00,5.000000000000000000e+00
6.000000000000000000e+00,7.000000000000000000e+00
8.000000000000000000e+00,9.000000000000000000e+00
In [139]: np.savetxt('out.csv', arr.reshape(5,2), delimiter=',', fmt='%5d')
In [140]: cat out.csv
    0,    1
    2,    3
    4,    5
    6,    7
    8,    9

Mixing text and numbers is a bit trickier.

I can write two variables to one row by just putting them in list:

In [143]: with open('out.csv','w') as f:
     ...:     writer = csv.writer(f)
     ...:     writer.writerow(['id', arr])
     ...: 
     ...:     
In [144]: cat out.csv
id,[0 1 2 3 4 5 6 7 8 9]

That in effect has done:

f.writeline('%s,%s'%(id, arr))

that is, format the list as strings and write.

Improve this answer

3 Comments

But i want both output in singlerow. Second thing is that, i have transposed array because initially it was stored in col, so i transposed it to save in row of csv.
I"m still guessing as to what you really want.
I just want to show img_id (string) and img_features (array) on single row. f.writeline('%s,%s'%(id, arr)), its working fine for id, but not for arr.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.