I know it should be simple, but I can't figure out how to escape the double quotes inside mysql query for json manipulation.
I have the following example:
SET #j = '{"4": 1, "0as#x"" : [1, 2, 3, 4], "b": [2, 3]}';
Please pay attention to the double quotes inside the second key: 0as#x"
If i run this query: SELECT JSON_ARRAY_APPEND(#j, '$."0as#x\"', '2');
I get the following error:
Error Code: 3141. Invalid JSON text in argument 1 to function json_array_append: "Missing a colon after a name of object member." at position 16.
All I want is to know how to escape the double quotes inside variable name of json object key.
I also have tried doubling the quotes """, with two backslashes \\"...
Could you please help me?
Thank you!
Later EDIT
In the set statement I escaped the double quotes with \". This is done behind if you use JSON_OBJECT.
In the end I escaped with \\ the double quotes and it worked.
The final code that is working:
SET #j = JSON_OBJECT('4', 1, '0as#x"', '[1, 2, 3, 4]', 'b', '[2, 3]');
SELECT JSON_ARRAY_APPEND(#j, '$."0as#x\\""', 2);
Use the JSON_OBJECT function to build the object, this way:
SET #j = JSON_OBJECT('4', 1, '0as#x"', '[1, 2, 3, 4]', 'b', '[2, 3]');
Then the function (no extra quotes around the key name):
SELECT JSON_ARRAY_APPEND(#j, '$.0as#x"', 2);
Hope it helped.
Related
This works:
select json_value('{ "a": "b" }', '$.a')
This doesn't work:
select json_value('{ "a": "b" }', '$["a"]')
and neither does this:
select json_value('{ "a": "b" }', '$[''a'']')
In JSON, these are the same:
foo = { "a": "b" }
console.log(foo.a)
console.log(foo["a"])
What am I missing? I get an error trying to use bracket notation in SQL Server:
JSON path is not properly formatted. Unexpected character '"' is found at position 2
No sooner do I ask, than I stumble on an answer. I couldn't find this in any documentation anywhere, but select json_value('{ "a": "b" }', '$."a"') works. Bracket notation is not supported, but otherwise invalid keys can be escaped with quotation marks, e.g. select json_value('{ "I-m so invalid][": "b" }', '$."I-m so invalid]["') when in JavaScript that would be foo["I-m so invalid]["]
MsSql reserves this for array index. SQL parses all JSON as a string literal, instead of as an object(JSON or ARRAY) with any hidden key.
Some of what SQL can do will vary with version. Here's a crash course on the annoying (but also really powerful, and fast once in place) requirements. I'm posting more than you need because a lot of the documentation for JSON through MsSql is lacking, and doesn't do justice to how strong it is with JSON.
MsDoc here: https://learn.microsoft.com/en-us/sql/relational-databases/json/json-data-sql-server?view=sql-server-ver15
In this example, we are working with a JSON "object" to separate the data into columns. Note how calling a position inside of an array is weird.
declare #data nvarchar(max) = N'[{"a":"b","c":[{"some":"random","array":"value"},{"another":"random","array":"value"}]},{"e":"f","c":[{"some":"random","array":"value"},{"another":"random","array":"value"}]}]'
--make sure SQL is happy. It will not accept partial snippets
select ISJSON(#data)
--let's look at the data in tabular form
select
json1.*
, json2.*
from openjson(#data)
with (
a varchar --note there is no "path" specified here, as "a" is a key in the first layer of the object
, c nvarchar(max) as JSON --must use "nvarchar(max)" and "as JSON" or SQL freaks out
, c0 nvarchar(max) N'$.c[0]' as JSON
) as json1
cross apply openjson(json1.c) as json2
You can also pull out the individual values, if needed
select oj.value from openjson(#data) as oj where oj.[key] = 1;
select
oj.value
, JSON_VALUE(oj.value,N'$.e')
, JSON_VALUE(oj.value,N'$.c[0].some')
, JSON_VALUE(#data,N'$[1].c[0].some') --Similar to your first example, but uses index position instead of key value. Works because SQL views the "[]" brackets as an array while trying to parse.
from openjson(#data) as oj
where oj.[key] = 1
In my hive table "ticket_full" I have a json type column named "service_id" that I would like to extract in 3 columns, which is like this
[{"position":"1","typeid":"ROUTNAME","value":"PWAW13197"},{"position":"2","typeid":"CDCNAME","value":null},{"position":"3","typeid":"SVCNAME","value":"Business"},{"position":"4","typeid":"USID","value":"FI021MLQE4"}]
[{"position":"1","typeid":"ROUTNAME","value":"KHLA30076"},{"position":"2","typeid":"CDCNAME","value":"eff-e-rjh-sw-cs2"},{"position":"3","typeid":"SVCNAME","value":"Managed LAN"},{"position":"4","typeid":"USID","value":"SA00BNGH0E"}]
[{"position":"1","typeid":"NUMLIAPTT","value":"0492212984"},{"position":"2","typeid":null,"value":null},{"position":"3","typeid":null,"value":null},{"position":"4","typeid":null,"value":null}]
I used the code below:
SELECT get_json_object(single_json_table.identifiant_produit, '$.position') AS position,
get_json_object(single_json_table.identifiant_produit, '$.typeid') AS typeid,
get_json_object(single_json_table.identifiant_produit, '$.value') AS value
FROM
(SELECT explode(split(regexp_replace(substr(serviceid, 2, length(serviceid)-2),
'"},\\{"', '"},,,,{"'), ',,,,') ) as identifiant_produit
FROM ticket_full) single_json_table
it works but every time there is a value at NULL, it ignores what follows and goes to the next field:
example:
Does anyone know how to fix this please ?
It is because null has no double-quotes and you are replacing this '"},\\{"' with this '"},,,,{"'
Try to remove double-quote before } in the regex pattern and replacement string accordingly, then it will work with quoted values and nulls also:
split(regexp_replace(substr(serviceid, 2, length(serviceid)-2),
'},\\{"', '},,,,{"'), ',,,,')
I'm looking for a groovy regex to be able to parse CSV file while ignoring commas insider double quotes.
The following regex works well in Java but not in Groovy:
it.split(",(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)")
Would you please help me to solve this issue.
I want to validate a CSv file format, for example for following example, the format is correct or not:
Header1, Header2, Header3
1, 2, 3
4, "5, 6", 7
But in this case, the format is not valid:
Header1, Header2, Header3
1, 2
I check Groovy Split CSV but it didn't solve my problem. Because the solution shown in that article, after parsing following csv:
Header1, Header2, Header3
1, "2, 3", 4, 5
will match:
Header1: 1
Header2: "2, 3"
Header3: 4
and it ignores 5! But me, I want to print out a message that format is not correct.
Thanks in advance.
try to change it like this:
it.split(",(?=(?:[^\"]\"[^\"]\")[^\"]\${1})")
Let me know.
EDIT 3 Problem below exists for Coldfusion 9.0, updating to 9.0.1 does indeed fix this
I have an application that is using SerializeJSON to encode query results:
#SerializeJSON('Ok works fine')#
Unfortunately it trims the trailing zeroes from numbers:
#SerializeJSON(12345.50)#
manually if i was to make the same value a string, same thing occurs
#SerializeJSON('12345.50')#
How can I prevent this from happening?
EDIT - my scenario specifics
Database (Oracle) has these example values stored on a row
benefactor_id : 0000729789 varchar2(10)
life_gift_credit_amt : 12345.50 number(14,2)
When I query using Coldfusion 9.0.1 (cfscript if it matters) , here is an RC dump, notice the id string retains leading zeroes, but the number column has removed trailing zero.
While that is interesting, it doesnt matter to the Original issue as i can create a query manually to retain that trailing zero like below, it still gets lost in the serializeJSON
I take the query results, and encode the values using serializeJSON. The JSON is consumed by jquery Datatables ajax. Notice the id string has become a number, and has added the '.0' as Miguel-F mentioned
<cfscript>
...
rc.sql = q.setsql;
rc.qResult = q.execute().getresult();
savecontent variable="rc.aaData" {
for (i=1; i <= rc.qResult.RecordCount; i++) {
writeOutput('{');
for (col=1; col <= iColumnsLen; col++) {
// the following line contains a conditional specific to this example
writeOutput('"#aColumns[col]#":#SerializeJSON(rc.qResult[aColumns[col]][i])#');
//former statement, discarded due to not being able to handle apostrophe's ... writeOutput('"#jsStringFormat(rc.qResult[aColumns[col]][i])#"');
writeOutput((col NEQ iColumnsLen) ? ',' : '');
}
writeOutput('}');
writeOutput((i NEQ rc.qResult.RecordCount) ? ',' : '');
}
};
</cfscript>
I was oringially using jsStringFormat instead of serializeJSON, but this would return invalid JSON due to the comments text area containing apostrophe's ect
{
"sEcho": 1,
"iTotalRecords": 65970,
"iTotalDisplayRecords": 7657,
"aaData": [
{
"nd_event_id": 525,
"benefactor_id": 729789.0,
"seq_number": 182163,
"life_gift_credit_amt": 12345.5,
"qty_requested": 2,
"b_a_comment": "#swap",
"pref_mail_name": "Jay P. Rizzi"
}
]
}
EDIT 2
a quick sidenote, if i change my serialization line to
writeOutput('"#aColumns[col]#": "#SerializeJSON(rc.qResult[aColumns[col]][i])#"');
then my result set changes to placing records in double quoting , but also double double quotes strings, while still removing the trailing zero; It leads me to believe serializeJSON is casting the value as a type?
"aaData": [
{
"nd_event_id": "525",
"benefactor_id": "729789.0",
"seq_number": "182163",
"life_gift_credit_amt": "12345.5",
"qty_requested": "2",
"b_a_comment": ""#swap"",
"pref_mail_name": ""JayP.Rizzi""
},
This is a bit baffling... I tested in CF 9 as well. Not really knowing what you are doing with the serialized data (passing as a service, outputting on a page, etc.), I put together some test patterns. One possible solution is if only trying to serialize a sing value - don't. You can actually run deserialize against your numeric value without serializing, and all it does is strip the trailing 0. Otherwise, if you must serialize a single value and don't want the trailing 0 stripped, set the variable to contain the quotation marks
<cfset manualserial = '"111.10"'>
<cfdump var="#DeSerializeJson(manualserial)#">
At this point you can us Deserialize and see that it maintains the 0, with output of 111.10
Below is some additional testing, so you can see what happens when serializing an array while trying to keep the trailing 0... no luck. However when I forwent the built in CF serialize and just created a serialized string, the trailing 0 is maintained (refer to var customarr and d_customarr in WriteDump example below).
Hope that helps a little.
<cfscript>
/*initial testing*/
string = SerializeJSON('Ok works fine');
numericstring = SerializeJSON('12345.50');
numeric = SerializeJSON(12345.50);
arr = SerializeJSON([12345.50,12345.10,'12345.20']);
arrFormat = SerializeJSON([NumberFormat(12345.50,'.00') & ' ',12345.10,'12345.20']);
d_string = DeSerializeJSON(string);
d_numericstring = DeSerializeJSON(numericstring);
d_numeric = DeSerializeJSON(numeric);
d_arr = DeSerializeJSON(arr);
d_arrFormat = DeSerializeJSON(arrFormat);
/*technically, there is no need to serialize a single string value, as running through DeSerialize just trims the trailing 0
if you need to do so, you would want to pass in as a string with quotation marks*/
customstring = '"12345.50"';
d_customstring = DeSerializeJSON(customstring);
customarr = '["12345.50","12345.10","12345.20"]'; //--you can format your own array instead of using CF to serialize
d_customarr = DeSerializeJSON(customarr);
WriteDump(variables);
</cfscript>
=======appended possible solution b========
I think that manually serializing your records may be the most stable option, try this example, and if it works you should be able to add the function to a cfc or create a udf for re-use. Hope it helps.
<cfscript>
q = QueryNew('nd_event_id,benefactor_id,seq_number,life_gift_credit_amt,qty_requested,b_a_comment,pref_mail_name',
'Integer,VarChar,Integer,Decimal,Integer,VarChar,VarChar');
r = queryaddrow(q,2);
querysetcell(q, 'nd_event_id', 525, 1);
querysetcell(q, 'benefactor_id', 0000729789, 1);
querysetcell(q, 'seq_number', 182163, 1);
querysetcell(q, 'life_gift_credit_amt', 12345.50, 1);
querysetcell(q, 'qty_requested', 2, 1);
querysetcell(q, 'b_a_comment', '##swap', 1);
querysetcell(q, 'pref_mail_name', 'Jay P. Rizzi', 1);
querysetcell(q, 'nd_event_id', 525, 2);
querysetcell(q, 'benefactor_id', 0000729790, 2);
querysetcell(q, 'seq_number', 182164, 2);
querysetcell(q, 'life_gift_credit_amt', 12345.90, 2);
querysetcell(q, 'qty_requested', 10, 2);
querysetcell(q, 'b_a_comment', '##swap', 2);
querysetcell(q, 'pref_mail_name', 'Jay P. Rizzi', 2);
WriteDump(q);
s = membershipManualSerializer(q);
public string function membershipManualSerializer(required query q){
var jsonString = '{"aaData":[';
var cols = listtoarray(q.columnList,',');
for(var i=1; i lte q.recordcount; i++){
jsonString &= "{";
for(var c=1;c lte arraylen(cols);c++){
jsonString &= '"' & cols[c] & '":"' & q[cols[c]][i] & '"';
jsonString &= (c lt arraylen(cols))? ",":"";
}
jsonString &= (i lt q.recordcount)? "},":"}]";
}
jsonString &="}";
return jsonString;
}
WriteOutput(s);
WriteDump(DeserializeJson(s));
</cfscript>
Taken from the comments
The original poster (OP) of this question initially reported that they were having this issue with ColdFusion 9.0.1. As it turned out they were actually running ColdFusion 9.0.0. This is significant because Adobe had made changes to how the SerializeJSON() function treats numbers in version 9.0.1. When the server was upgraded to version 9.0.1 these issues were resolved.
This blog post by Raymond Camden discusses the changes made in 9.0.1 - Not happy with the CF901 JSON Changes?
In that blog post he references bug 83638 that had been entered and then fixed in HotFix 1 for version 9.0.1 - Cumulative Hotfix 1 (CHF1) for ColdFusion 9.0.1
If you search the BugBase for JSON under version 9.0.1 there are several reporting the same issue as the OP.
Those reported bugs also mentioned another issue that the OP had not initially reported, that a .0 was being appended to integers as well. Later in the discussion the OP confirmed that they too were seeing this behavior. This lead them to verify the ColdFusion version being utilized and found that it was not 9.0.1.
The following throws a syntax error, "unexpected EOF while parsing":
${foo({'bar':'baz'})}
which I guess is from the inner closing curly brace.
This works fine:
${foo(dict(bar='baz'))}
but what's the syntax for using a dictionary literal?
From Brian Rue on the Mako Templates Google Group:
This is a long-outstanding bug; just use dict(). If you need a dictionary
with keys that aren't strings, convert a list of tuples into a dict.
e.g. instead of this:
${foo({1: 'a', 2: 'b'})}
do this:
${foo(dict([(1, 'a'), (2, 'b')]))}