Not all scripted queries can be safely converted to an encoded query

[James Fricker]

I would like to point out the issue that not all scripted queries can be safely converted to encoded queries.

In this wiki article it suggests using a single encoded query instead of using a series of addQuery calls for complex GlideRecord queries.

The danger with this approach is when any of the text used to construct the encoded query is user supplied.

For example you would expect these two scripts to be equivalent, but they are not.

var gr = new GlideRecord("incident");
gr.addQuery("short_description", current.u_field);
gr.query();

var gr = new GlideRecord("incident");
gr.addEncodedQuery("short_description=" + current.u_field);
gr.query();

This is a very simple unusual example designed just to prove my point.

If the value of current.u_field is "avalue^NQanyfieldANYTHING"

then the first example above would only return incidents with that exact string as the short description.

Whereas in the second example the encoded query becomes...

short_description=avalue^NQanyfieldANYTHING

and it would return all incidents in the system

In other words, in the second example the user who entered the value of current.u_field has just successfully injected extra query terms into your code and changed its behavior.

Unfortunately there is no way I know of to escape the carat character in encoded query search values.

If you could, then you could safely create encoded query strings with user supplied text.

For example, if we could use the back slash character to escape the caret character in a search value, the encoded query above would become

short_description=avalue\^NQanyfieldANYTHING

And the addEncodedQuery method would need to know the difference between '^' and '\^'

The carat character on its own would be a search term separator, whereas backslash-carat would mean include the carat in the search string.

[James Fricker]

> so ^NQANYTHING will return all records.

Firstly ^NQANYTHING is meaningless as the field name is missing. If you mean "^NQanyfieldANYTHING" then not for addQuery.

If you pass an encoded query to an addQuery call, it will search for the explicit text without parsing the encoded query. Try it if you don't believe me.

[Yifeng Zhang]

Hi all 

I wonder is the issue has been resolved yet? 

is there escape caret in the encoded query yet, this post has been 3 years old.. user can inject query and that's pretty big deal aint it

[James Fricker]

Hi Yifeng,

I agree it is a pretty big deal, and as far as I know there is still no way to easily escape the caret character. One option is to convert the encoded query version of the code to this...

var gr = new GlideRecord('incident');
gr.addEncodedQuery('short_description=javascript:"' + String(current.u_field).replace('\\', '\\\\').replace('^', '\\x5E') + '"');
gr.query();

Here is a link to the original wiki article that recommended using addEncodedQuery instead of addQuery which I think can be a dangerous thing to do because of this query injection issue.

[Yifeng Zhang]

Yeah this query thing is a interesting topic, I landed on this post from another issue that I found out... tell you a secret... try query a ACL protected field in the UI... you will soon find out the underlining query engine honors condition put on protected field, even if you don't have the rights to access it.... and with a STARTWITH query you can linearly guess the value of any ACL protected field as long as you have the rights to access the record itself... we reported this to HI portal and still waiting on a response.. but I kind of get the feeling that this was not a known issue..

[TylerTeter · Accepted solution]

I ran into this same issue, and had to find a solution for it. After a lot of digging, it turns out ServiceNow has a function for this!

Effectively their form of escaping is like '^' to '^^'. It's probably best practice to use this method for future expand-ability.

I haven't fully tested it to make sure it catches all special search terms (ie %*<>!= and CONTAINS, STARTSWITH, etc)... so YMMV

Example:

var gr = new GlideRecord("incident");

gr.addEncodedQuery("short_description=" + GlideStringUtil.escapeQueryTermSeparator( current.u_field ) );

gr.query();

https://developer.servicenow.com/app.do#!/api_doc?v=london&id=GSUS-escapeQTermSeparator_S