I am querying a nested json file, the structure looks something like as follows
{"user_id":1234,
"text":"example text"
"first_nested":{
"field1":"dummy string 1",
"field2":"dummy string 2"
},
"user":{
"field3":"dummy string 3",
"field4":"dummy string 4"
},
"last":1}
I have a nested json structure named "user", and when query the following:
SELECT tbl.user AS us FROM dfs.`/filepath/trial.json` as tbl WHERE user_id=221
or
SELECT tbl.user.field1 AS us FROM dfs.`/filepath/trial.json` as tbl WHERE tbl.user_id=221
I get the following error:
UserRemoteException: PARSE ERROR: Encountered ". user" at line 1, column 11. Was expecting one of: "FROM" ... "," ... "AS" ... ... ... ... ... ... "." ... "NOT" ... "IN" ... "BETWEEN" ... "LIKE" ... "SIMILAR" ... "=" ... ">" ... "<" ... "<=" ... ">=" ... "<>" ... "+" ... "-" ... "*" ... "/" ... "||" ... "AND" ... "OR" ... "IS" ..
and when I simply use user with dereferencing with tbl. , the query return the name of the user who owns the current drill profile.
I can't change the name of the column from this json file, how do I get around this ?
I looked this up in the docs. Use backticks for reserved words.
Because the column alias contains the special space character, also enclose the alias in back ticks, as shown in the following example
From https://drill.apache.org/docs/lexical-structure/
In your case:
SELECT tbl.`user`.field1 AS us FROM ...
Related
I have a json file filled with a list of data where each element has one field called url.
[
{ ...,
...,
"url": "us.test.com"
},
...
]
In a different file I have a list of mappings that I need to replace the affected url fields with, formatted like this:
us.test.com test.com
hello.com/se hello.com
...
So the end result should be:
[
{ ...,
...,
"url": "test.com"
},
...
]
Is there a way to do this in Vim or do I need to do it programmatically?
Well, I'd do this programmatically in Vim ;-) As you'll see it's quite similar to Python and many other scripting languages.
Let's suppose we have json file open. Then
:let foo = json_decode(join(getline(1, '$')))
will load json into VimScript variable. So :echo foo will show [{'url': 'us.test.com'}, {'url': 'hello.com/se'}].
Now let's switch to a "mapping" file. We're going to split all lines and make a Dictionary like that:
:let bar = {}
:for line in getline(1, '$') | let field = split(line) | let bar[field[0]] = field[1] | endfor
Now :echo bar shows {'hello.com/se': 'hello.com', 'us.test.com': 'test.com'} as expected.
To perform a substitution we do simply:
:for field in foo | let field.url = bar->get(field.url, field.url) | endfor
And now foo contains [{'url': 'test.com'}, {'url': 'hello.com'}] which is what we want. The remaining step is to write the new value into a buffer with
:put =json_encode(foo)
You could…
turn those lines in your mappings file (/tmp/mappings for illustration purpose):
us.test.com test.com
hello.com/se hello.com
...
into:
g/"url"/s#us.test.com#test.com#g
g/"url"/s#hello.com/se#hello.com#g
...
with:
:%normal Ig/"url"/s#
:%s/ /#
The idea is to turn the file into a script that will perform all those substitutions on all lines matching "url".
If you are confident that those strings are only in "url" lines, you can just do:
:%normal I%s#
:%s/ /#
to obtain:
%s#us.test.com#test.com#g
%s#hello.com/se#hello.com#g
...
write the file:
:w
and source it from your JSON file:
:source /tmp/mappings
See :help :g, :help :s, :help :normal, :help :range, :help :source, and :help pattern-delimiter.
I want to pass the value from "processContext.targetUpdate" from managed bean to 'update' tag in commandButton, if it is not null.
So I did:
<p:commandButton
update="#(.painelInfoDer)
#(.msgsDialogAltDer)
#(.opValorAtualDer)
#(.outputPanel)
#(.#{processContext.targetUpdate != null?processContext.targetUpdate:#none})"
But I received an error:
Caused by: com.sun.el.parser.ParseException: Encountered "#" at line 1, column 157.
Was expecting one of:
<INTEGER_LITERAL> ...
<FLOATING_POINT_LITERAL> ...
<STRING_LITERAL> ...
"true" ...
"false" ...
"null" ...
"(" ...
"!" ...
"not" ...
"empty" ...
"-" ...
<IDENTIFIER> ...
What's wrong with this expression? Will pass parameter like this work as I expect?
I think the problems is the #none, now I change to:
<p:commandButton
update="#(.painelInfoDer)
#(.msgsDialogAltDer)
#(.opValorAtualDer)
#(.outputPanel)
#(.#{processContext.targetUpdate != null?processContext.targetUpdate:''})"
and now the button disappear.
I try to load some data from a table containing json rows.
There is one field that can contain special chars as \t and \r, and I want to keep them as is in the new table.
Here is my file:
{"text_sample": "this is a\tsimple test", "number_sample": 4}
Here is what I do:
Drop table if exists temp_json;
Drop table if exists test;
create temporary table temp_json (values text);
copy temp_json from '/path/to/file';
create table test as (select
(values->>'text_sample') as text_sample,
(values->>'number_sample') as number_sample
from (
select replace(values,'\','\\')::json as values
from temp_json
) a);
I keep getting this error:
ERROR: invalid input syntax for type json
DETAIL: Character with value 0x09 must be escaped.
CONTEXT: JSON data, line 1: ...g] Objection to PDDRP Mediation (was Re: Call for...
How do I need to escape those characters?
Thanks a lot
Copy the file as csv with a different quoting character and delimiter:
drop table if exists test;
create table test (values jsonb);
\copy test from '/path/to/file.csv' with (format csv, quote '|', delimiter ';');
select values ->> 'text_sample', values ->> 'number_sample'
from test;
?column? | ?column?
-----------------------------+----------
this is a simple test | 4
As mentioned in Andrew Dunstan's PostgreSQL and Technical blog
In text mode, COPY will be simply defeated by the presence of a backslash in the JSON. So, for example, any field that contains an embedded double quote mark, or an embedded newline, or anything else that needs escaping according to the JSON spec, will cause failure. And in text mode you have very little control over how it works - you can't, for example, specify a different ESCAPE character. So text mode simply won't work.
so we have to turn around to the CSV format mode.
copy the_table(jsonfield)
from '/path/to/jsondata'
csv quote e'\x01' delimiter e'\x02';
In the official document sql-copy, some Parameters list here:
COPY table_name [ ( column_name [, ...] ) ]
FROM { 'filename' | PROGRAM 'command' | STDIN }
[ [ WITH ] ( option [, ...] ) ]
[ WHERE condition ]
where option can be one of:
FORMAT format_name
FREEZE [ boolean ]
DELIMITER 'delimiter_character'
NULL 'null_string'
HEADER [ boolean ]
QUOTE 'quote_character'
ESCAPE 'escape_character'
FORCE_QUOTE { ( column_name [, ...] ) | * }
FORCE_NOT_NULL ( column_name [, ...] )
FORCE_NULL ( column_name [, ...] )
ENCODING 'encoding_name'
FORMAT
Selects the data format to be read or written: text, csv (Comma Separated Values), or binary. The default is text.
QUOTE
Specifies the quoting character to be used when a data value is quoted. The default is double-quote. This must be a single one-byte character. This option is allowed only when using CSV format.
DELIMITER
Specifies the character that separates columns within each row (line) of the file. The default is a tab character in text format, a comma in CSV format. This must be a single one-byte character. This option is not allowed when using binary format.
NULL
Specifies the string that represents a null value. The default is \N (backslash-N) in text format, and an unquoted empty string in CSV format. You might prefer an empty string even in text format for cases where you don't want to distinguish nulls from empty strings. This option is not allowed when using binary format.
HEADER
Specifies that the file contains a header line with the names of each column in the file. On output, the first line contains the column names from the table, and on input, the first line is ignored. This option is allowed only when using CSV format.
cast json as text, instead of getting text value from json. Eg:
t=# with j as (
select '{"text_sample": "this is a\tsimple test", "number_sample": 4}'::json v
)
select v->>'text_sample' your, (v->'text_sample')::text better
from j;
your | better
-----------------------------+--------------------------
this is a simple test | "this is a\tsimple test"
(1 row)
and to avoid 0x09 error, try using
replace(values,chr(9),'\t')
as in your example you replace backslash+t, not the actual chr(9)...
I am getting back a JSON string from a MySQL 5.7 query in ColdFusion 9.0.1. Here is my query:
SELECT (
SELECT GROUP_CONCAT(
JSON_OBJECT(
'nrtype', nrt.nrtype,
'number', nr.number
)
)
) AS nrJSON
FROM ...
The returned data looks like this:
{"nrtype": "Phone 1", "number": "12345678"},{"nrtype": "E-Mail 1", "number": "some#email.com"}
But as soon as I try to use DeserializeJSON() on it I am getting the following error:
JSON parsing failure at character 44:',' in {"nrtype": "Phone 1", "number": "12345678"},{"nrtype": "E-Mail 1", "number": "some#email.com"}
I am a little confused. What I want to get is a structure created by the DeserializeJSON() function.
What can I do?
That is not valid JSON as the parser is describing. If you wrap that JSON within square brackets '[' and ']' it would be valid (or at least parsable). They will make it an array of structures. Not sure how to make MySQL return the data within those brackets?
I guess you could add the brackets using ColdFusion but I would prefer to have the source do it correctly.
jsonhack = '[' & queryname.nrJSON & ']';
datarecord = DeserializeJSON(jsonhack);
writeDump(datarecord);
I created an example with your data that you can see here - trycf.com gist
From the comments
The solution indeed was [to add the following to the SQL statement]:
CONTACT('[',
GROUP_CONCAT(
JSON_OBJECT(...)
),
']')
If you have columns with some already containing JSON format String, try this : https://stackoverflow.com/a/45278722/2282880
Portion of code with JSON_MERGE() :
...
CONCAT(
'{"elements": [',
GROUP_CONCAT(
JSON_MERGE(
JSON_OBJECT(
'type', T2.`type`,
'data', T2.`data`
),
CONCAT('{"info": ', T2.`info`, '}')
)
),
']}'
) AS `elements`,
...
I'm trying to run a query on Cassandra through spark.
When running this command:
val test = sc.cassandraTable[Person](keyspace,table)
.where("name=?","Jane").collect
I get the appropriate output for the query.
When I try to use the where statement to enter the query as a whole string I get an error.
I receive the query as a json:
{"clause": " name = 'Jane' "}
then turn it into a string.
When running
val query = (json \ "clause").get.as[String]
//turns json value into a string
val test = sc.cassandraTable[Person](keyspace,table)
.where(query).collect
I get the following error:
java.io.IOException: Exception during preparation of SELECT "uuid", "person", "age" FROM "test"."users" WHERE token("uuid") > ? AND token("uuid") <= ? AND name = Jane ALLOW FILTERING: line 1:232 no viable alternative at input 'ALLOW' (...<= ? AND name = [Jane] ALLOW...)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.createStatement(CassandraTableScanRDD.scala:288)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD.com$datastax$spark$connector$rdd$CassandraTableScanRDD$$fetchTokenRange(CassandraTableScanRDD.scala:302)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD$$anonfun$18.apply(CassandraTableScanRDD.scala:328)
at com.datastax.spark.connector.rdd.CassandraTableScanRDD$$anonfun$18.apply(CassandraTableScanRDD.scala:328)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at com.datastax.spark.connector.util.CountingIterator.hasNext(CountingIterator.scala:12)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at com.datastax.spark.connector.util.CountingIterator.foreach(CountingIterator.scala:4)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
I suspect that when I turn the json value " name = 'Jane' " into a string, I lose the single quotes hence I get " name = Jane " which of course raises an error. I tried escaping the single quotes with \ and with a second pair of single quotes around the name Jane {"clause": " name = ''Jane'' "}. It doesn't solve the issue.
Edit: After further testing it's definitely the json that loses the single quotes and CQL needs them to perform the query. Can anyone suggest a way to escape/save the presence of the single quotes? I tried escaping with \ double single quotes '' . Is there a way to use JSON to provide proper whole CQL statements?
Please use Unicode character \u0027.