Running a python script from nodejs and returning json back

Question

I am running a python script using python-shell. The python script(named combine.py) returns data in json format. But the code works fine on my local machine but not on aws instance. I get the following error in pm2 logs:

SyntaxError: Unexpected token / in JSON at position 0
1|app      |     at JSON.parse (<anonymous>)
1|app      |     at PythonShell.asJson (/home/ubuntu/toolCaseWebsite/node_modules/python-shell/index.js:358:21)
1|app      |     at /home/ubuntu/toolCaseWebsite/node_modules/python-shell/index.js:310:42
1|app      |     at Array.forEach (<anonymous>)
1|app      |     at PythonShell.recieveInternal (/home/ubuntu/toolCaseWebsite/node_modules/python-shell/index.js:306:15)
1|app      |     at PythonShell.receiveStderr (/home/ubuntu/toolCaseWebsite/node_modules/python-shell/index.js:290:21)
1|app      |     at Socket.<anonymous> (/home/ubuntu/toolCaseWebsite/node_modules/python-shell/index.js:108:18)
1|app      |     at Socket.emit (events.js:182:13)
1|app      |     at Socket.EventEmitter.emit (domain.js:442:20)
1|app      |     at addChunk (_stream_readable.js:279:12)

I have made sure that python script returns valid json data by first putting data in file and then checking that data via online json validator.

Nodejs (javascript file)

  var ps = require('python-shell')
  noOfLines = 2
  noOfClusters = 5
  var options = {
    mode: 'json',
    pythonOptions: ['-u'], // get print results in real-time
    scriptPath: './pythonScripts/',
    args: [noOfClusters, noOfLines, processingData]
  }

  ps.PythonShell.run('combine2.py', options, function(err, results) {
    if (err) throw err
    // Results is an array consisting of messages collected during executio n
    //console.log(results)

    // Data send to index_timeline
    res.render('index_timeline', {
      results: results[0]
    })

    fs.writeFile('myOutput.txt', JSON.stringify(results, 0, 2), err => {
      // throws an error, you could also catch it here
      if (err) throw err

      // success case, the file was saved
      console.log('File saved!')
    })
  })

Python Script(combine.py)

if __name__ == "__main__":

   # getting parameters
    content = sys.argv[3]
    nclusters= int(sys.argv[1])
    noOfLines=int(sys.argv[2])


    data={"clusters":[]}

    # Data Cleaning and then splitting into sentences

    sentences = dataCleaning(content).split('.')#splitting sentences on basis of comma rather fullstop
    sentences = list(filter(None, sentences))

    temp=list()
    myDict=dict()
    summarizing=str()

    #getting clusters
    clusters = cluster_sentences(sentences, nclusters)


    for cluster in range(nclusters):
        for i,sentence in enumerate(clusters[cluster]):
            temp.append(sentences[sentence])
            summarizing+=sentences[sentence]+". "
        myDict["sentences"]=list(temp)
        sentence_tokens, word_tokens = tokenize_content(summarizing)
        sentence_ranks = score_tokens(word_tokens, sentence_tokens)
        myDict["summary"]=str(summarize(sentence_ranks, sentence_tokens,noOfLines))
        data["clusters"].append(dict(myDict))
        myDict.clear()
        del temp[:]
        summarizing=''

    print(json.dumps(data))

myOutput.txt(data that I wrote in txt file above)

[
  {
    "clusters": [
      {
        "sentences": [
          " Qoum &amp; Maa Baap K Duaon Se Award Haasil Kiya",
          " Qoum &amp; Maa Baap K Duaon Se Award Haasil Kiya",
          " Qoum &amp; Maa Baap K Duaon Se Award Haasil Kiya"
        ],
        "summary": " Qoum &amp; Maa Baap K Duaon Se Award Haasil Kiya. Qoum &amp; Maa Baap K Duaon Se Award Haasil Kiya."
      },
      {
        "sentences": [
          " Mushtaq Ahmed",
          " Mushtaq Ahmed",
          " Mushtaq Ahmed NIJAMHAMY"
        ],
        "summary": " Mushtaq Ahmed. Mushtaq Ahmed NIJAMHAMY."
      }
    ]
  }
]

Are you getting an error in line res.render('index_timeline' where you are trying to send response? — Abhas Tandon
– Abhas Tandon, Commented Apr 15, 2019 at 8:32
Test other input for JSON input without [] braces. Some JSON parsers require as top wrapper for JSON {"arr": []} to be an object. Also console.log / print out if really it's same JSON passed to this function. — Zydnar
– Zydnar, Commented Apr 15, 2019 at 8:56
@AbhasTandon I have edited logs above. You can check error now. Its not really in res.render('index_timeline'). Thanks in advance — Taha Asif
– Taha Asif, Commented Apr 15, 2019 at 9:07
Also, this error could come if some non-JSON content got written to stdout, which python-shell tried to parse as JSON. In JSON mode, all output should be JSON-encoded, with carriage returns for separating messages. Try running your python code as a standalone script and make sure that no extra output is coming from it. Most probably your python code is sending an empty line char as output which node is trying to parse as JSON.parse("/\n") leading to this error. — Abhas Tandon
– Abhas Tandon, Commented Apr 15, 2019 at 11:18

Abhas Tandon · Accepted Answer · 2019-04-15 11:30:11Z

2

This error could come if some non-JSON content got written to stdout, which python-shell tried to parse as JSON. In JSON mode, all output should be JSON-encoded, with carriage returns for separating messages.

Try running your python code as a standalone script and make sure that no extra output (Eg. new line) is coming from it.

Most probably your python code is sending an empty line char as output which node is trying to parse as JSON.parse("/\n") leading to this error.

Uncaught SyntaxError: Unexpected token / in JSON at position 0
    at JSON.parse

answered Apr 15, 2019 at 11:30

Abhas Tandon

1,88916 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Taha Asif Over a year ago

I already tried to run the script as stand alone script. It runs perfectly fine. Also I copied the result it print, and passed that result through online json validator. It prints a valid json.

Taha Asif Over a year ago

Thanks a lot. The problem was some what similar. The server was returning some warning about future depreciation of some modules. That's why python-shell was not getting data in json format. By adding command line options to ignore warnings, I was able to run this code on server as well

Taha Asif · Accepted Answer · 2019-04-15 14:52:40Z

0

Thanks a lot. The problem was some what similar. The server was returning some warning about future depreciation of some modules. That's why python-shell was not getting data in json format. By adding command line options to ignore warnings, I was able to run this code on server as well

answered Apr 15, 2019 at 14:52

Taha Asif

4139 silver badges24 bronze badges

Collectives™ on Stack Overflow

Running a python script from nodejs and returning json back

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related