Python get variables from JSON: Unexpected errors

Question

Im trying to create a script that will read a JSON file and use the variables to select particular folders and files and save them somewhere else.

My JSON is as follows:

{
    "source_type": "folder",
    "tar_type": "gzip",
    "tar_max_age": "10",
    "source_include": {"/opt/myapp/config", "/opt/myapp/db, /opt/myapp/randomdata"}
    "target_type": "tar.gzip",
    "target_path": "/home/user/targetA"

}

So far, I have this Python Code:

import time
import os
import tarfile
import json
source_config = '/opt/myapp/config.JSON'
target_dir = '/home/user/targetA'

def main():
    with open('source_config', "r").decode('utf-8') as f:
        data = json.loads('source_config')
        for f in data["source_include", str]:
                full_dir = os.path.join(source, f)
                tar = tarfile.open(os.path.join(backup_dir, f+ '.tar.gzip'), 'w:gz')
                tar.add(full_dir)
                tar.close()

    for oldfile in os.listdir(backup_dir):
       if str(oldfile.time) < 20:
           print("int(oldfile.time)")

My traceback is:

Traceback (most recent call last):
  File "/Users/user/Documents/myapp/test/point/test/Test2.py", line 16, in <module>
    with open('/opt/myapp/config.json', "r").decode('utf-8') as f:
AttributeError: 'file' object has no attribute 'decode'

How do I fix this?

Presumably your input JSON does not use {"/opt/myapp/config", "/opt/myapp/db, /opt/myapp/randomdata"}, but [...] instead. — Martijn Pieters
– Martijn Pieters, Commented Dec 2, 2013 at 12:13
Your input JSON is not valid; a list of items uses square brackets, but your input uses {...} for a list instead, and is missing a comma at the end. That line should use ["/opt/myapp/config", "/opt/myapp/db, /opt/myapp/randomdata"], to be valid. — Martijn Pieters
– Martijn Pieters, Commented Dec 2, 2013 at 12:42

Martijn Pieters · Accepted Answer · 2013-12-02 12:55:52Z

2

You are trying to call .decode() directly on the file object. You'd normally do that on the read lines instead. For JSON, however, you don't need to do this. The json library handles this for you.

Use json.load() (no s) to load directly from the file object:

with open(source_config) as f:
    data = json.load(f)

Next, you need to address the source_include key with:

for entry in data["source_include"]:
    base_filename = os.path.basename(entry)
    tar = tarfile.open(os.path.join(backup_dir, base_filename + '.tar.gzip'), 'w:gz')
    tar.add(full_dir)
    tar.close()

Your JSON also needs to be fixed, so that your source_include is an array, rather than a dictionary:

{
    "source_type": "folder",
    "tar_type": "gzip",
    "tar_max_age": "10",
    "source_include": ["/opt/myapp/config", "/opt/myapp/db", "/opt/myapp/randomdata"],
    "target_type": "tar.gzip",
    "target_path": "/home/user/targetA"

}

Next, you loop over filenames with os.listdir(), which are strings (relative filenames with no path). Strings do not have a .time attribute, if you wanted to read out file timestamps you'll have to use os.stat() calls instead:

for filename in os.listdir(backup_dir):
    path = os.path.join(backup_dir, filename)
    stats = os.stat(path)
    if stats.st_mtime < time.time() - 20:
        # file was modified less than 20 seconds ago

edited Dec 2, 2013 at 12:55

answered Dec 2, 2013 at 12:06

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Josh Smeaton Over a year ago

Maybe change the variable name f in the loop so it's not the same as the file object?

Brendan Over a year ago

That changes the traceback to: with open('source_config') as f: IOError: [Errno 2] No such file or directory: 'source_config'

Josh Smeaton Over a year ago

@Brendan see the with line, I removed the quotes so it uses the string in the variable source_config rather than the string "source_config" which doesn't exist as a file.

Josh Smeaton Over a year ago

No problem, also added extra info regarding the json (instead of writing another answer).

Martijn Pieters Over a year ago

@JoshSmeaton: I made that a comment on the post itself, but sure.

|

Collectives™ on Stack Overflow

Python get variables from JSON: Unexpected errors

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related