I'd like to CONCAT two column value into one string in string in JSON format. However I have problem with the quote and double quote in the query. How do I fix my query so it success produce the expected result?
$concat = "CONCAT('{"CODE":"pm_r.CODE","NAME":"pm_r.NAME"}') AS `JSON`"
$query = $this->db->query(
'SELECT pm_r.ID_REQUIREMENT, '.$concat.'FROM `pm_requirement` `pm_r`'
);
The expected out should be:
ID_REQUIREMENT JSON
ID001 {"CODE":"001","NAME":"Shane"}
To avoid quoting clashes, a solution is to use a php HEREDOC string.
I also fixed you CONCAT, where the names of the column to concatenate should be separated from the fixed parts of the string.
$query = $this->db->query(<<<EOT
SELECT
pm_r.ID_REQUIREMENT,
CONCAT('{"CODE":"', pm_r.CODE, '","NAME":"', pm_r.NAME, '"}') AS `JSON`
FROM `pm_requirement` `pm_r`
EOT
);
Related
First week having to deal with a MYSQL database and JSON field types and I cannot seem to figure out why values are encoded automatically and then returned in encoded format.
Given the following SQL
-- create a multiline string with a tab example
SET #str ="Line One
Line 2 Tabbed out
Line 3";
-- encode it
SET #j = JSON_OBJECT("str", #str);
-- extract the value by name
SET #strOut = JSON_EXTRACT(#J, "$.str");
-- show the object and attribute value.
SELECT #j, #strOut;
You end up with what appears to be a full formed JSON object with a single attribute encoded.
#j = {"str": "Line One\n\tLine 2\tTabbed out\n\tLine 3"}
but using JSON_EXTRACT to get the attribute value I get the encoded version including outer quotes.
#strOut = "Line One\n\tLine 2\tTabbed out\n\tLine 3"
I would expect to get my original string with the \n \t all unescaped to the original values and no outer quotes. as such
Line One
Line 2 Tabbed out
Line 3
I can't seem to find any JSON_DECODE or JSON_UNESCAPE or similar functions.
I did find a JSON_ESCAPE() function but that appears to be used to manually build a JSON object structure in a string.
What am I missing to extract the values to the original format?
I like to use handy operator ->> for this.
It was introduced in MySQL 5.7.13, and basically combines JSON_EXTRACT() and JSON_UNQUOTE():
SET #strOut = #J ->> '$.str';
You are looking for the JSON_UNQUOTE function
SET #strOut = JSON_UNQUOTE( JSON_EXTRACT(#J, "$.str") );
The result of JSON_EXTRACT() is intentionally a JSON document, not a string.
A JSON document may be:
An object enclosed in { }
An array enclosed in [ ]
A scalar string value enclosed in " "
A scalar number or boolean value
A null — but this is not an SQL NULL, it's a JSON null. This leads to confusing cases because you can extract a JSON field whose JSON value is null, and yet in an SQL expression, this fails IS NULL tests, and it also fails to be equal to an SQL string 'null'. Because it's a JSON type, not a scalar type.
I have inserted some test data like below:
INSERT INTO tblDataInfo (lookupkey, lookupvalue, scope)
VALUES ('diskname', '/dev/sdb', 'common')
yours sincerely
I wanted to query this data and like to obtain the query output in JSON format.
I have used the query
select lookupvalue as 'disk.name'
from tblDataInfo
where lookupkey = 'diskname' FOR JSON PATH;
This query returns
[{"disk":{"name":"\/dev\/sdb"}}]
which is escaping all my forward slashes (/) using the escape character (\). How can I have my output not to put escape-character (\)?
This query returns result that you need:
select json_query ('{"name":"' + lookupvalue + '"}') as 'disk'
from tblDataInfo
where lookupkey = 'diskname'
for json path;
I'm not a specialist with regular expression and I'm facing issues using regexp_replace in Hive.
I would like to load a CSV file into Hive, which contains rows like that:
AAA,1234,BBB,,,"""CC,CCC""","""DDD""","""EE"EEE""",,
"""AAA""",1234,BBB,,,CCCC,"""DD,DD""",,"""FFFF""",
As you can see, the format isn't perfect
There are non-escaped commas into string fields
Some string fields are enclosed by """ (3 double-quotes)
There are non-escaped double-quotes into string fields
There are empty fields
When I try to import it into a Hive table, the columns are not well parsed because of the non-escaped commas.
So I imported the raw data as rows into a Hive table like this:
CREATE EXTERNAL TABLE MyRawTable
(
RAW_DATA STRING
)
STORED AS TEXTFILE
LOCATION '/path/to/hdfs/file'
And i'm trying to use the regexp_replace function to transform the rows:
Escape the commas, the double and simple quotes in the string fields
Not enclose string fields by double quotes
So data will look like that:
AAA,1234,BBB,,,CC\,CCC,DDD,EE\"EEE,,
AAA,1234,BBB,,,CCCC,DD\,DD,,FFFF,
I don't find the solution for this regex, any ideas? Thanks a lot!
Forget about the regexp, you don't need it. The commas aren't escaped, but they are surrounded by double-quotes. You can simply use the OpenCSVSerde :
CREATE EXTERNAL TABLE yourtable(foo int, bar string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
WITH SERDEPROPERTIES (
"separatorChar" = ",",
"quoteChar" = "\"",
"escapeChar" = "\""
)
LOCATION '/your/folder/containing/csv/files/';
I need an sql select statement to retrieve 04:30 and test.zip from this string:
{"TIME":"04:30","DATE":"11\/25\/2013","FILENAME":["test.zip"]}
use this \[(.*?)\]
it return value between [ and ]
and for 04:30 use TIME":(.*?),
it return value after "TIME":
Can't you just decode it and use PHP? (assuming you can't change the way it's stored in the db)
<?php
$str = '{"TIME":"04:30","DATE":"11/25/2013","FILENAME":["test.zip"]}';
$o = json_decode($str);
$time = $o->TIME;
$file = $o->FILENAME[0];
var_dump($time); //"04:30"
var_dump($file); //"test.zip"
Regex replaces etc in MySQL require a UDF (user-defined function) mysql-udf-regexp
If none of the above are viable solutions (change DB structure, do it with PHP, use a MySQL UDF), you'll need to get creative. It would require a known, static format of that string, but you could replace some parts and substring others. For example:
SELECT SUBSTRING(REPLACE(`column_name`,'{"TIME":"',''),1,5) AS `time` FROM `table_name`
File is more complex, this example assuming only one filename in the array
SELECT REPLACE(SUBSTRING(`column_name`,LOCATE('"FILENAME":["',`column_name`)+13),'"]}','') AS `file` FROM `table_name`
Those two field selections get 04:30 and test.zip respectively (you can of course use those functions in the same statement, rather than separately like I have, by comma separating them)
My query is throwing up this error. Can anyone see why?
$query = "SELECT * FROM Units WHERE ID = `$uniqueUnits[a]`";
Unknown column '' in 'where clause'
Two problems.
You're using backticks to delimit a string. Backticks delimit fields, so MySQL thinks you're trying to give it a column name.
The error message indicates that, in fact, this value that it thinks is a column name, is empty. So your value $uniqueUnits[a] is probably broken, or not being interpolated correctly.
You should do the following:
Interpolate your variables explictly with the "complex syntax" to be sure that the string forms properly;
Check the value of $query so that you can see what's going on:
print $query;
Use actual quotation marks to delimit strings:
$query = "SELECT * FROM Units WHERE ID = '{$uniqueUnits[a]}'";
// ^ quote
// ^ PHP variable interpolation
try
$query = "SELECT * FROM Units WHERE ID = '$uniqueUnits[a]'";
^--- ^---
Backticks are for escaping reserved words, so mysql is translating your variable's contents into a field name.
Because apparently $uniqueUnits[a] resolves to the empty string. And there is no column like this in the database.
Try surrounding your array with {}, like this:
$query = "SELECT * FROM Units WHERE ID = `{$uniqueUnits[a]}`";
Also, is column ID actually in your table?