I have an old database, where some columns have comma separated strings stored like this technician,director,website designer
I want to convert them to a JSON array, so I can use the MySQL JSON array type and the methods associated with it.
So basically I am looking for a method to convert technician,director,website designer to ["technician","director","website designer"] in SQL.
The length of the list is arbitrary.
The biggest struggle I am having is how to apply a SQL function on each element in the comma separated string, (So I for example can run the JSON_QUOTE() on each element) as adding the brackets are just a simple CONCAT.
The solution should be for MySQL 5.7.
You can use REPLACE to get the expected string:
SELECT CONCAT('["', REPLACE('technician,director,website designer', ',', '","'), '"]')
-- ["technician","director","website designer"]
Using JSON_VALID you can check if the result of the conversion is a valid JSON value:
SELECT JSON_VALID(CONCAT('["', REPLACE('technician,director,website designer', ',', '","'), '"]'))
-- 1
demo on dbfiddle.uk
Related
Using PostgreSQL 9.3, the json_array_elements function returns each string element in an array as json strings.
select value from json_array_elements('["a", "b"]');
value
-------
"a"
"b"
I would like to convert these to regular Postgres TEXT values but I'm at a loss. I tried value::TEXT but they are still double quoted, i.e. json strings.
As simple as:
select value from json_array_elements_text('["a", "b"]');
I think u want this.
select REPLACE(value::TEXT,'"','') from json_array_elements('["a", "b"]');
I am using some native JSON fields to store information about some application entities in a MySQL 5.7.10 database. I can have 'N' rows per "entity" and need to roll-up and merge the JSON objects together, and any conflicting keys should replace instead of merge. I can do this through code, but if I can do it natively and efficiently in MySQL even better.
I have attempted this using a combination of GROUP_CONCAT and JSON_MERGE, but I've run into two issues:
JSON_MERGE won't take the results of GROUP_CONCAT as a valid argument
JSON_MERGE combines conflicting keys instead of replacing them. What I really need is more of a JSON_SET but with 'N' number of JSON docs instead of "key, value" notation.
Is this possible with the current MySQL JSON implementation?
First of all, GROUP_CONCAT only returns a string, so you have to cast it. Second of all, there is a function doing exactly what you want called JSON_MERGE_PATCH(). Try the following:
SELECT
JSON_MERGE_PATCH(
yourExistingJson,
CAST(
CONCAT(
'[',GROUP_CONCAT(myJson),']'
)
AS JSON)
) AS myJsonArray
....
Just realized your version. You would have to upgrade to 5.7.22 or higher. Is it possible in your case? If not, there may be other ways but they wont be elegant :(
You could do something like the following:
SELECT
CAST(CONCAT(
'[',
GROUP_CONCAT(
DISTINCT JSON_OBJECT(
'foo', mytable.foo,
'bar', mytable.bar
)
),
']'
) AS JSON) AS myJsonArr
FROM mytable
GROUP BY mytable.someGroup;
JSON_MERGE won't take the results of GROUP_CONCAT as a valid argument
GROUP_CONCAT gives a,b,c,d, not a JSON array. Use JSON_ARRAYAGG (introduced in MySQL 5.7.22), which works just like group_concat, but gives a correct array ["a", "b", "c", "d"], that should be accepted with JSON functions.
Prior to 5.7.22, you need to use a workaround:
cast(
concat('["', // begin bracket and quote
group_concat(`field` separator '", "'), // separator comma and quotes
'"]' // end quote and bracket
) as json
)
JSON_MERGE combines conflicting keys instead of replacing them. What I really need is more of a JSON_SET but with 'N' number of JSON docs instead of "key, value" notation.
Use JSON_MERGE_PATCH instead, as introduced in MySQL 5.7.22. JSON_MERGE is a synonym for JSON_MERGE_PRESERVE.
See https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html.
Read my Best Practices for using MySQL as JSON storage.
Aggregation of JSON Values
For aggregation of JSON values, SQL NULL values are ignored as for other data types. Non-NULL values are converted to a numeric type and aggregated, except for MIN(), MAX(), and GROUP_CONCAT(). The conversion to number should produce a meaningful result for JSON values that are numeric scalars, although (depending on the values) truncation and loss of precision may occur. Conversion to number of other JSON values may not produce a meaningful result.
I just found this in mysql docs
I have a Postgres JSON column where some columns have data like:
{"value":90}
{"value":99.9}
...whereas other columns have data like:
{"value":"A"}
{"value":"B"}
The -> operator (i.e. fields->'value') would cast the value to JSON, whereas the ->> operator (i.e. fields->>'value') casts the value to text, as reported by pg_typeof. Is there a way to find the "actual" data type of a JSON field?
My current approach would be to use Regex to determine whether the occurrence of fields->>'value' in fields::text is surrounded by double quotes.
Is there a better way?
As #pozs mentioned in comment, from version 9.4 there are available json_typeof(json) and jsonb_typeof(jsonb) functions
Returns the type of the outermost JSON value as a text string. Possible types are object, array, string, number, boolean, and null.
https://www.postgresql.org/docs/current/functions-json.html
Applying to your case, an example of how this could be used for this problem:
SELECT
json_data.key,
jsonb_typeof(json_data.value) AS json_data_type,
COUNT(*) AS occurrences
FROM tablename, jsonb_each(tablename.columnname) AS json_data
GROUP BY 1, 2
ORDER BY 1, 2;
I ended up getting access to PLv8 in my environment, which made this easy:
CREATE FUNCTION value_type(fields JSON) RETURNS TEXT AS $$
return typeof fields.value;
$$ LANGUAGE plv8;
As mentioned in the comments, there will be a native function for this in 9.4.
So I have a column with strings that are multiples of 5 characters, such as "12345" or "abcde12345" or "asdfghjkli12345". What I'm trying to do is write a query to split each of these strings into 5 character chunks and return the distinct ones.
So with "12345" , "abcde12345" , "asdfghjkli12345"
I would get back "12345" "abcde" "asdfg" and "hjkli"
Is this possible?
MySQL does not include a function to split a delimited string. Although separated data would normally be split into separate fields within a relation data, spliting such can be useful either during initial data load/validation or where such data is held in a text field.
The following formula can be used to extract the Nth item in a delimited list, in this case the 3rd item "ccccc" in the example comma separated list.
select replace(substring(substring_index('aaa,bbbb,ccccc', ',', 3), length(substring_index('aaa,bbbb,ccccc', ',', 3 - 1)) + 1), ',', '') ITEM3
The above formula does not need the first item to be handled as a special case and returns empty strings correctly when the item count is less than the position requested.
You can also create your own split function and use it. Split value from one field to two
Source: http://dev.mysql.com/doc/refman/5.0/en/string-functions.html
This sounds like it would be outside the limitations of SQL and would certainly require a programming language. It would be VERY easy in any 3GL.
I have a SQL query where I'm calling SUBSTRING_INDEX on a comma delimited string. In all cases the string has two commas. Is there any way to get at the center element only? That is to say, besides doing something like this:
SUBSTRING_INDEX(SUBSTRING_INDEX(foo, ',', 2), ',', -1)
You can't do a regular expression substitution with MySQL but you can take a look at those third party libs for MySQL:
https://launchpad.net/mysql-udf-regexp