I have some data in text file as shown below image(it's just a portion of whole data). What is best way to filter out keys?
{ u'chan': 5,
u'cls': 0,
u'codr': u'4/5',
u'data': u'ABfxqqqpVVVOAAA='
}
-
don't link to a picture.. copy and paste example data in your question. Show the code you have tried? Can you read a text file with python? can you extract the dictionary? Do you know how to write a file with python?joel goldstick– joel goldstick2016-11-17 21:19:50 +00:00Commented Nov 17, 2016 at 21:19
-
Look up ast.eval python module. It will let you take your dictionary as a string, and convert it to a python dictionary.joel goldstick– joel goldstick2016-11-17 21:33:25 +00:00Commented Nov 17, 2016 at 21:33
Add a comment
|
2 Answers
Assuming the sample data is one record of your data, you can use pandas to subset and export directly to an excel workbook.
import pandas as pd
df = pd.DataFrame({ u'chan': 5,
u'cls': 0,
u'codr': u'4/5',
u'data': u'ABfxqqqpVVVOAAA=',
u'datr': u'SF10BW125',
u'freq': u'912.9',
u'lsnr': u'-8.2',
u'mhdr': u'8007000002001900',
u'modu': u'LORA',
u'opts': u'',
u'port': 5,
u'rfch': 1,
u'rssi': -111,
u'seqn': 25,
u'size': 16,
u'timestamp': u'2016-11-17T09:51:44.406724Z',
u'tmst': 2477375724L},index=[0])
df = df[['data','chan','timestamp','rssi']]
oName = #Path to desired excel workbook
df.to_excel(oName,'Sheet1')
1 Comment
James
sample data is just a portion of my text file. I do have multiple of them in it
For a series of files in the form of what you posted try
import csv
filenames=[...]
#open your output file in append mode
with open('output.csv','a+') as out:
outwriter = csv.writer(out,dialect='excel')
#write your headers
outwriter.writerow(['header1','header2','header3'])
#for every file
for fname in filenames:
with open(fname) as f:
#list that holds each row
csvlines=[]
#labels you wanna keep
keep=["u'data'","u'rssi'","u'timestamp'"]
lines=f.readlines()
print lines
#split at first : character
lines = map(lambda x:x.split(':',1),lines)
for line in lines:
if line[0].strip() in keep:
csvlines.append(line[1].strip())
#clean from unnecessary characters
csvlines=map(lambda x:x.replace("u'","").replace("'","").replace(",",""),csvlines)
#write it to csv and then reset it for the next file.
if(len(csvlines)==3):
print "writing"
print csvlines
outwriter.writerow(csvlines)
csvlines=[]
16 Comments
James
Hi, I have one file containing a lot of those data in same format. How does it work then?
themistoklik
Reset every 3 appends?
James
im getting ValueErorr : invalid mode for 'wa'
themistoklik
Also if the dile you're reading gets really big you will need a line by line solution instead of readlines()
James
I actually tried that too. But it's not writing any data from my file into csv file except headers. I also tried just a set of data, like the above sample.
|