Converting JavaScript object literal to JavaScript object WITHOUT 'eval'

Question

I'm making a web scraper, most of the data on the web page is in JavaScript object literal form, e.g.:

// Silly example
var user = {
    name: 'John', 
    surname: 'Doe',
    age: 21,
    family: [
        {
            name: 'Jane',
            surname: 'Doe',
            age: 37
        },
        // ...
    ]
};

So when I search for the contents in my JavaScript app the Object above would be:

"{name: 'John', surname: 'Doe', age: 21, family: [{name: 'Jane', surname: 'Doe', age: 37}]}"

Is it possible to parse those to regular JavaScript Objects without using 'eval' or making my own parser? I saw other similar questions about this but the answers are not applicable: they all suggest JSON.parse() (not applicable) and eval (I can't use it for security reasons). In this question, for example, all the answers suggest eval or new Function() which are basically the same thing.

If there are no other ways would it be a viable option to convert the literal to proper JSON and then parse it to JavaScript object?

This is what I tried right now, it worked on a simple object but I'm not sure it will work everywhere:

const literal = script.innerText.slice(script.innerText.indexOf('{'), script.innerText.lastIndexOf('}') + 1);
const json = literal.replace(/.*:.*(\".*\"|\'.*\'|\[.*\]|\{.*\}|true|false|[0-9]+).*,/g, (prev) => {
  let parts = prev.split(':');
  let key = '"' + parts.shift().trim() + '"';
  let value = parts.join(':').replace(/'.*'/, (a) => {
    return '"' + a.slice(1, a.length - 1) + '"';
  }).trim();
  return key + ':' + value;
});
const obj = JSON.parse(json);

An object literal is a JS object. If you need a JSON string, then see developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… — Teemu
– Teemu, Commented Jun 18, 2019 at 12:24
@Teemu but based on the statement that OP is using a web scraper, I'd assume that he has it in string format. I'd be interested in why it isn't possible to use JSON.parse — Tom M
– Tom M, Commented Jun 18, 2019 at 12:25
@TomM It looks like I'm not reading enough the question nowadays. Yes, the provided example is most likely a string, but it is not a JSON string, that's why JSON.parse can't parse it. — Teemu
– Teemu, Commented Jun 18, 2019 at 12:28
@TomM I'd be interested in why it isn't possible to use JSON.parse Because even if you take away var user = and the trailing ;, its not valid JSON. eg.. {hello: 'there'} is not valid JSON. — Keith
– Keith, Commented Jun 18, 2019 at 12:29

ponury-kostek · Accepted Answer · 2019-06-18 13:47:23Z

It's simple demo how you can use esprima to get globally declared variables

"use strict";

const src = `
var user = {
	name: 'John',
	surname: 'Doe',
	age: 21,
	family: [
		{
			name: 'Jane',
			surname: 'Doe',
			age: 37
		},
		// ...
	]
};`;
const src2 = `
var a = [1,2,3], b = true;
var s = "some string";
var o = {a:1}, n = null;
var some = {'realy strange' : {"object":"'literal'"}}
`;

function get_globals(src) {
  return esprima.parse(src).body
    .filter(({type}) => type === "VariableDeclaration") // keep only variables declarations
    .map(({declarations}) => declarations)
    .flat()
    .filter(({type}) => type === "VariableDeclarator")
    .reduce((vars, {id: {name}, init}) => {
      vars[name] = parse(init);
      return vars;
    }, {});
}

console.log(get_globals(src));
console.log(get_globals(src2));

/**
 * Parse expression
 * @param expression
 * @returns {*}
 */
function parse(expression) {
  switch (expression.type) {
    case "ObjectExpression":
      return ObjectExpression(expression);
    case "Identifier":
      return expression.name;
    case "Literal":
      return expression.value;
    case "ArrayExpression":
      return ArrayExpression(expression);
  }
}

/**
 * Parse object expresion
 * @param expression
 * @returns {object}
 */
function ObjectExpression(expression) {
  return expression.properties.reduce((obj, {key, value}) => ({
    ...obj,
    [parse(key)]: parse(value)
  }), {});
}

/**
 * Parse array expression
 * @param expression
 * @returns {*[]}
 */
function ArrayExpression(expression) {
  return expression.elements.map((exp) => parse(exp));
}

<script src="https://unpkg.com/esprima@~4.0/dist/esprima.js"></script>

Keith · Accepted Answer · 2019-06-19 14:52:50Z

3

For data like this you could might be able to use a couple of regex's to convert into a valid JSON object.

Below is an example..

ps. It might not be 100% foolproof for all object literals.

var str = "{name: 'John', surname: 'Doe', age: 21, family: [{name: 'Jane', surname: 'Doe', age: 37}]}";

var jstr = str
  .replace(/\'(.*?)\'/g, '"$1"')
  .replace(/([\{ ]*?)([a-z]*?)(\:)/gi, '$1"$2"$3');

var obj = JSON.parse(jstr);

console.log(obj);

As pointed out by @ponury-kostek, and by myself using regEx can be limited. Using some sort of AST parsing like Esprima is certainly a good idea, especially if your already using an AST parser.

But if an AST parser is overkill, a more robust version below using Javascript might be better. Ps. again it might not be 100% correct, but it should cope with the majority of Object literals.

var str = `{
  name: 'John:', surname: 'Doe', age: 21,
  family: [
    {name: '😊Jane\\n\\r', surname: 'Doe', age: 37},
    {'realy:strange indeed' : {"object":"'\\"literal'"}}
  ]
}`;


const objLits = [...':0123456789, \t[]{}\r\n'];

function objParse(src) {
  const input = [...src];
  const output = [];
  let inQ = false, inDQ = false, 
    inEsc = false, inVname = false;
  for (const i of input) {
    if (inEsc) {
      inEsc = false;    
      output.push(i);
    } else if (i === "\\") {
      inEsc = true;
      output.push(i);
    } else if (i === "'" && !inDQ) {
      output.push('"');
      inQ = !inQ;
    } else if (i === '"' && !inQ) {
      output.push('"');
      inDQ = !inDQ;      
    } else if (!inVname & !inQ & !inDQ & !inEsc) {
      if (objLits.includes(i)) {
        output.push(i);
      } else {
        inVname = true;
        output.push('"');
        output.push(i);
      }
    } else if (inVname) {
      if (i === ':') {
        inVname = false;
        output.push('"');
      }
      output.push(i);
    } else {
      output.push(i);
    }
  }
  const ostr = output.join('');
  return JSON.parse(ostr);
}

console.log(objParse(str));

edited Jun 19, 2019 at 14:52

answered Jun 18, 2019 at 13:06

Keith

24.4k2 gold badges32 silver badges48 bronze badges

3 Comments

ponury-kostek Over a year ago

This will crash if you use quoted keys or if you use quotes in values (escaped), try to parse something like var some = {'realy strange' : {"object":"'literal'"}}

Keith Over a year ago

@ponury-kostek Yes, I did mention it wouldn't be 100%.. I've updated with a JS version that should be a bit more robust. PS. I use AST parser's too, and I think that's the most robust way of doing this. But for something quick and dirty I'll leave these version here.

ponury-kostek Over a year ago

OP had mention that he is making web scraper, that's why I suggested using some JS parser

Tom M · Accepted Answer · 2019-06-18 12:49:37Z

2

Assuming you use node, an easy workaround would be

// scraper.js
const fs = require('fs');
const objectString = myScraper.scrape('example.com');

fs.writeFileSync('./scraped.js', objectString);

// myAppUsingTheData.js
const myObj = require('myAppUsingTheData');

However, require still involves evaluation in some way. AND you'd need separate processes in order to access your object. Also, you'd need to somehow insert module.exports. If you want to parse Objects only, try JSON5

const myObj = JSON5.parse(objectString);
console.log(myObj.name)

Using JSON5 will effectively prevent you from running malicious code that is not an object in your app and apparently can parse unquoted JSON keys.

answered Jun 18, 2019 at 12:49

Tom M

2,9172 gold badges23 silver badges48 bronze badges

1 Comment

kekekuli Oct 21 at 13:21

JSON5 is magic, which solved my problem!

Slai · Accepted Answer · 2019-06-18 16:11:09Z

-1

A script tag can be added with the script text :

var JS = `var user = {
    name: 'John', 
    surname: 'Doe',
    age: 21,
    family: [
        {
            name: 'Jane',
            surname: 'Doe',
            age: 37
        },
    ]
};`;

var script = document.createElement('script');
script.textContent = JS
document.head.appendChild(script);

console.log( user )

answered Jun 18, 2019 at 16:11

Slai

23k5 gold badges49 silver badges55 bronze badges

1 Comment

AksLolCoding Over a year ago

This has the same issue as eval() does, anybody can put any code they want in the text.

Collectives™ on Stack Overflow

Converting JavaScript object literal to JavaScript object WITHOUT 'eval'

4 Answers 4

Comments

3 Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

3 Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related