xTuple.com xTupleU Blog & News Customer Support

My problems with Xtuple 5 and JSON

Greetings
I’m piloting an xtuple 4.11.1 upgrade to 5.0 beta project. My final unsolved issue regards some custom extension js code. It works on xt4 but fails on xt5. I receive an "“Unable to parse JSON string” error. The exact same code has worked in previous xt versions for years. The source of the returned data doesn’t seem to matter. We use our same libraries to pull from a number of different API providers and each give the same error when used in xtuple 5. The returned data seems to be valid JSON and passes other parsers. Here is an example of a returned string that fails:

{“id”:10729,“shortname”:“BEFO-SCBA-IFO-FF1 Blended Learning - Niagara County Spring 2019”,“categoryid”:1734,“categorysortorder”:7300002,“fullname”:“BEFO-SCBA-IFO-FF1 Blended Learning - Niagara County Spring 2019”,“displayname”:“BEFO-SCBA-IFO-FF1 Blended Learning - Niagara County Spring 2019”,“idnumber”:"",“summary”:“

New York Office of Fire Prevention & Control - Firefighter I 2016 Edition

”,“summaryformat”:1,“format”:“topics”,“showgrades”:1,“newsitems”:0,“startdate”:1553749200,“enddate”:0,“numsections”:28,“maxbytes”:104857600,“showreports”:0,“visible”:1,“hiddensections”:1,“groupmode”:0,“groupmodeforce”:0,“defaultgroupingid”:0,“timecreated”:1361292846,“timemodified”:1475599473,“enablecompletion”:0,“completionnotify”:0,“lang”:"",“forcetheme”:"",“courseformatoptions”:[{“name”:“hiddensections”,“value”:1},{“name”:“coursedisplay”,“value”:0}]}

Here is the function that processes the “finished” signal of the qnetworkrequestmanager:

_netmgr.finished.connect(function(res) {  
  var res_json, res_str;
  if (qtv5 !== true) {
     if (res.error()) {
       if (typeof error !== "undefined") {
         if (typeof scope !== "undefined") {
           error.call(scope, res);
         } else {
           error(res);
         }
       } else {
         QMessageBox.critical(mywindow, "Error", "A network error occurred while contacting server." + "<br />Error " + res.error() + ": " + res.errorString());
       }
     }
   }                
   try {     
      res_str = res.readAll();
      res_json = JSON.parse(res_str);
   } catch (e) {
      print(e);
   }
    if (typeof scope !== "undefined") {
      success.call(scope, res_json);
    } else {
      success(res_json);
    }
  
  try {
    return _netmgr.finished.disconnect(arguments.callee);
  } catch (e) {
    return print(e);
  }

} );

The line “res_json = JSON.parse(res_str);” is what throws the error.

I guess my question would be two fold. 1) Has anyone else used qtscript and the JSON parsing routines successfully in xtuple 5.0 beta. I hate to be troubleshooting something that just doesn’t work. and 2) Does anyone see any errors or have any ideas on how to get this string to parse?

Thanks for any ideas. I started off by writing this question as an email to Gil and David but then got to feeling guilty for bothering them directly so I opted to post my stuggles here instead.

Jim

Version 5.0 includes an upgrade core Qt version (5.11) which may explain some differences. I’m not aware of anything specific changing around JSON - in fact version 5.0 makes much greater use of json in many places.

I copied your json example into a JSON parser and it was saying there is an error in the “summary” element. Looks like you might have a special character in the value string.

Line feeds are not valid in JSON. if you need to include it, it has to be escaped (\\n instead of \n).

I think my “copy and paste” into the forum editor may have inserted line feeds and corrupted the data. When I copy and pasted directly into the https://jsonformatter.curiousconcept.com" validator the source of my post example shows as valid json. I believe the forum input screen may have changed it.

I think my problem is showing up because of a typecasting issue. If I execute:
res_str = res.readAll();
res_json = JSON.parse(res_str); Then I get an error.

If I do res_json = JSON.parse(‘the string’) (where ‘the string’ is the json code I pasted) then it works. So my guess is that res_json is not of the correct data type that JSON.parse needs. I read that the readAll() function returns a QByteArray. However I think JSON.parse is wanting a text (string?) value. Is there any easy way to cast the qbytearray to the correct type?

Maybe qt5 has eliminated that implicit cast since the exact same code was developed on the previous qt and still works as it always had.

I am using the same code in Qt5.11 and Qt5.12, with no manipulating the response after being read. If you add a line to print(res_str), what shows up in the database log?

David, I added the “print(res_str)” statement: I copy and pasted this from the database log:
{“id”:10680,“shortname”:“Massachusetts Firefighting Academy - 078 - MFA”,“categoryid”:1028,“categorysortorder”:6390004,“fullname”:“Massachusetts Firefighting Academy - 078 - MFA”,“displayname”:“Massachusetts Firefighting Academy - 078 - MFA”,“idnumber”:"",“summary”:"",“summaryformat”:1,“format”:“topics”,“showgrades”:1,“newsitems”:0,“startdate”:1553061600,“enddate”:0,“numsections”:26,“maxbytes”:41943040,“showreports”:0,“visible”:1,“hiddensections”:1,“groupmode”:0,“groupmodeforce”:0,“defaultgroupingid”:0,“timecreated”:1361292846,“timemodified”:1553260226,“enablecompletion”:1,“completionnotify”:0,“lang”:"",“forcetheme”:"",“courseformatoptions”:[{“name”:“hiddensections”,“value”:1},{“name”:“coursedisplay”,“value”:0}]}
Thu May 30 16:30:20 2019 Debug:
SyntaxError: Unable to parse JSON string

I’ve found the problem between xt4 and xt5. The data returned by the read_all function is a QByteArray. However the data used by JSON.parse() is expected to be of type string.

Works in xt4 but parse error in xt5:
try {
var raw_data = res.readAll();
res_json = JSON.parse(raw_data); // error at this point
} catch (e) {
print(e);
}

Works in xt4 AND xt5:
try {
var raw_data = res.readAll();
var res_str = raw_data.toString(); //Convert qbytearray to qstring
res_json = JSON.parse(res_str);
} catch (e) {
print(e);
}

David said he was using the same code I was having problems with and it was working for him. I don’t know how that might be but possibly it is a platform issue. I think David uses a Mac and I’m working on a PC. Just guessing.

We use the networkrequestmanager in a number of our modifications so this is a great relief to get it working in xt5. In my piloting I believe this issue was the final major problem. I’m eager to the the release candidate and get this new offering into a production environment.

Jim

Jim