I have the following table in my database:
Type | Name
-------------------------------------------------
INT(10) UNSIGNED | id
LONGTEXT | settings
The settings column holds JSON strings such as the following:
'[
{"value":"1","label":"user_type"},
{"value":"2","label":"email_type"}
]'
I have some corrupt data that doesn't correspond to the required format as the requirements have now changed.
'[
{"value": 8,"label":"should_receive_notifications"},
]'
Notice how the value is unquoted compared to the first example which is how I need them.
Is there a way I can do a find and replace on all JSON strings within the settings column to update all unquoted values in the JSON string and wrap them in quotes?
You may use the next procedure:
CREATE PROCEDURE quote_value(max_amount INT)
BEGIN
REPEAT
UPDATE test
SET settings = JSON_REPLACE(settings, CONCAT('$[', max_amount, '].value'), CAST(JSON_UNQUOTE(JSON_EXTRACT(settings, CONCAT('$[', max_amount, '].value'))) AS CHAR));
SET max_amount = max_amount - 1;
UNTIL max_amount < 0 END REPEAT;
END
max_amount parameter defines the amount of objects in the array to be updated (do not forget that the array elements are counted from zero). So set it to max objects per array amount value.
https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=166f43d44e57b62da034bd9530713beb
This is under assumption that there are no spaces between the characters in json string, simple but data needs to be verified for this.
update tablename
set settings = replace(settings, '"value\":' , '"value":\"')
where settings not like '%"value":"%'
update tablename
set settings = replace(settings, ',"' , '","')
where settings not like '%","%'
Related
I have a requirement where I need to mask all but characters in position 1,4,8,12,16.. for a variable length string with 'X'
For example:
Input string - 'John Doe'
Output String - 'JXXn xxE'
SPACE between the two strings must be retained.
Kindly help or reach out for more details if required.
I think maybe an external function would be best here, but if that's too much to bite off, you can get crafty with strtok_split_to_table, xml_agg and regexp_replace to rip the string apart, replace out characters using your criteria, and stitch it back together:
WITH cte AS (SELECT REGEXP_REPLACE('this is a test of this functionality', '(.)', '\1,') AS fullname FROM Sys_Calendar.calendar WHERE calendar_date = CURRENT_DATE)
SELECT
REGEXP_REPLACE(REGEXP_REPLACE((XMLAGG(tokenout ORDER BY tokennum) (VARCHAR(200))), '(.) (.)', '\1\2') , '(.) (.)', '\1\2')
FROM
(
SELECT
tokennum,
outkey,
CASE WHEN tokennum = 1 OR tokennum mod 4 = 0 OR token = ' ' THEN token ELSE 'X' END AS tokenout
FROM TABLE (strtok_split_to_table(cte.fullname, cte.fullname, ',')
RETURNS (outkey VARCHAR(200), tokennum integer, token VARCHAR(200) CHARACTER SET UNICODE)) AS d
) stringshred
GROUP BY outkey
This won't be fast on a large data set, but it might suffice depending on how much data you have to process.
Breaking this down:
WITH cte AS (SELECT REGEXP_REPLACE('this is a test of this functionality', '(.)', '\1,') AS fullname FROM Sys_Calendar.calendar WHERE calendar_date = CURRENT_DATE)
This CTE is just adding a comma between every character of our incoming string using that regexp_replace function. Your name will come out like J,o,h,n, ,D,o,e. You can ignore the sys_calendar part, I just put that in so it would spit out exactly 1 record for testing.
SELECT
tokennum,
outkey,
CASE WHEN tokennum = 1 OR tokennum mod 4 = 0 OR token = ' ' THEN token ELSE 'X' END AS tokenout
FROM TABLE (strtok_split_to_table(cte.fullname, cte.fullname, ',')
RETURNS (outkey VARCHAR(200), tokennum integer, token VARCHAR(200) CHARACTER SET UNICODE)) AS d
This subquery is the important bit. Here we create a record for every character in your incoming name. strtok_split_to_table is doing the work here splitting that incoming name by comma (which we added in the CTE)
The Case statement just runs your criteria swapping out 'X' in the correct positions (record 1, or a multiple of 4, and not a space).
SELECT
REGEXP_REPLACE(REGEXP_REPLACE((XMLAGG(tokenout ORDER BY tokennum) (VARCHAR(200))), '(.) (.)', '\1\2') , '(.) (.)', '\1\2')
Finally we use XMLAGG to combine the many records back into one string in a single record. Because XMLAGG adds a space in between each character we have to hit it a couple of times with regexp_replace to flip those spaces back to nothing.
So... it's ugly, but it does the job.
The code above spits out:
tXXs XX X XeXX oX XhXX fXXXtXXXaXXXy
I couldn't think of a solution, but then #JNevill inspired me with his idea to add a comma to each character :-)
SELECT
RegExp_Replace(
RegExp_Replace(
RegExp_Replace(inputString, '(.)(.)?(.)?(.)?', '(\1(\2[\3(\4', 2)
,'(\([^ ])', 'X')
,'(\(|\[)')
,'this is a test of this functionality' AS inputString
tXXs XX X XeXX oX XhXX fXXXtXXXaXXXy
The 1st RegExp_Replace starts at the 2nd character (keep the 1st character as-is) and processes groups of (up to) 4 characters adding either a ( (characters #1,#2,#4, to be replaced by X unless it's a space) or [ (character #3, no replacement), which results in :
t(h(i[s( (i(s[ (a( (t[e(s(t( [o(f( (t[h(i(s( [f(u(n(c[t(i(o(n[a(l(i(t[y(
Of course this assumes that both characters don't exists in your input data, otherwise you have to choose different ones.
The 2nd RegExp_Replace replaces the ( and the following character with X unless it's a space, which results in:
tXX[s( XX[ X( X[eXX( [oX( X[hXX( [fXXX[tXXX[aXXX[y(
Now there are some (& [ left which are removed by the 3rd RegExp_Replace.
As I still consider me as a beginner in Regular Expressions, there will be better solutions :-)
Edit:
In older Teradata versions not all parameters were optional, then you might have to add values for those:
RegExp_Replace(
RegExp_Replace(
RegExp_Replace(inputString, '(.)(.)?(.)?(.)?', '(\1(\2[\3(\4', 2, 0 'c')
,'(\([^ ])', 'X', 1, 0 'c')
,'(\(|\[)', '', 1, 0 'c')
I received a valid JSON string from client side, it contains an array of integer values:
declare #JSON nvarchar(max) = N'{"Comments": "test", "Markets": [3, 151]}'
How to select the market IDs correctly?
If I use a query like this: select * from openjson(#JSON) j, it returns
The type of Markets is 4, which means an object,
but the query below returns null value:
select j.Markets from openjson(#JSON) with(Markets nvarchar(max)) j
My goal is to update Market table based on these IDs, eg:
update Market set Active = 1 where MarketID in (3, 151)
Is there a way to do this?
Any built-in function compatible with SQL server 2016 can be used.
Note:
Thanks to #johnlbevan
SELECT VALUE FROM OPENJSON(#JSON, '$.Markets') works perfectly for this problem.
Just for the completeness, here is how I created the JSON integer array ("Markets": [3, 151]) from SQL server.
Since there is no array_agg function out of the box in 2016, I did this:
SELECT (
JSON_QUERY('[' + STUFF(( SELECT ',' + CAST(MarketID AS VARCHAR)
FROM Market
FOR XML PATH('')),1,1,'') + ']' ) AS Markets)
To expand the Markets array alongside other columns you can do this:
SELECT Comments, Market
FROM OPENJSON('{"Comments": "test", "Markets": [3, 151]}')
WITH (Comments nvarchar(32), Markets NVARCHAR(MAX) AS JSON) AS a
CROSS APPLY OPENJSON (a.Markets) WITH (Market INT '$') AS b
Convert the string to json
Map the first field returned to the Comments column with type nvarchar(32)
Map the second field to Markets column with type nvarchar(max), then use as json to say that the contents is json (see https://learn.microsoft.com/en-us/sql/t-sql/functions/openjson-transact-sql#arguments for a more detailed description - search the page for as json; the key paragraph starts at the 4th occurrence)
Use a cross apply to apply the OPENJSON function to the Markets column so we can fetch values from that property.
Finally use the WITH statement to map the name Market to the returned value, and assign it a data type of INT.
However, to just get the list of values needed to do the update, you can do this:
UPDATE Market
SET Active = 1
WHERE MarketID IN
(
SELECT value
FROM OPENJSON('{"Comments": "test", "Markets": [3, 151]}','$.Markets')
);
Again OPENJSON lets us query the string as JSON
However this time we specify a path to point at the Markets value directly (see https://learn.microsoft.com/en-us/sql/t-sql/functions/openjson-transact-sql)
We now return the values returned and filter our UPDATE on those, as we would were we dealing with any other subquery.
Is it possible to store a JSON list field in MariaDB like the sample below using dynamic columns:
{
"myfield": "value1",
"myList": ["item1" , "item2", "item3"]
}
CAST can be replaced with
JSON_MERGE('{"key": "val"}', JSON_OBJECT());
It's a pity that MariaDB dynamic column doesn't support JSON array list natively although that's a fix for that and work better with INTEGERS but you can try to use a string.
ALTER TABLE `your_table` ADD COLUMN `dynamic_column` BLOB NULL DEFAULT
NULL ;
You will store the array list as string in the dynamic column
INSERT INTO
your_table (dynamic_column)
VALUES
(COLUMN_CREATE('myfield','value1'
,'myList','["item1" , "item2", "item3"]')
);
OR
UPDATE
your_table
SET
dynamic_column = COLUMN_CREATE('myfield','value1'
,'myList','["item1" , "item2", "item3"]'
)
WHERE
id = your_id ;
So far so good now there's the tricky part. How you will return that without breaking the format. Using a replace function:
SELECT REPLACE(REPLACE(COLUMN_JSON(dynamic_column),'"[','['),']"',']') FROM your_table ;
You also can test the output quickly without creating any table.
SET #tmp= COLUMN_CREATE('myfield','value1','myList','[ "item1" , "item2" , "item3" ]' ) ;
SELECT REPLACE(REPLACE(COLUMN_JSON(#tmp),'"[','['),']"',']') json_string;
If it outputs breaking the json format with '\' :
+----------------------------------------------------------------------+
| json_string |
+----------------------------------------------------------------------+
| {"myList":[ \"item1\" , \"item2\" , \"item3\" ],"myfield":"value1"} |
+----------------------------------------------------------------------+
then try something like
SELECT REPLACE(REPLACE(REPLACE(COLUMN_JSON(#tmp),'"[','['),']"',']'),'\\','') json_string;
+----------------------------------------------------------------+
| json_string |
+----------------------------------------------------------------+
| {"myList":[ "item1" , "item2" , "item3" ],"myfield":"value1"} |
+----------------------------------------------------------------+
GIVE IT A GO !!
** Don't forget to validate the json string output on any online validator or text editor like textwrangler or notepad++
** Beware sometimes dynamic columns can be a dynamic pain
Cheers
I have a database which has a field titled 'address1'. If there is only 1 string in this field for a record, I am able to correct the case from eg 'PAULSTOWN' to 'Paulstown', or 'bishopslough' to 'Bishopslough'.
I have done this by creating a function:
CREATE FUNCTION init_cap (s VARCHAR(255))
RETURNS VARCHAR(255) DETERMINISTIC
RETURN CONCAT( UPPER( SUBSTRING( s, 1, 1 ) ) , LOWER( SUBSTRING( s FROM 2 ) ) );
Then using:
UPDATE customer SET address1 = init_cap(address1);
To correct records.
However, this does not fully correct records that contain more than one string, eg 'dalesfort road' will only be corrected to 'Dalesfort road' and not 'Dalesfort Road'. There are also some entries with more than 2 strings.
How could I change the above function to cater for 2 or more strings? Also is that function declared correctly, or should I be using begin and end sections?
It's ok I found the answer at artfulsoftware.com
Now I just need to analyse the code and learn how it works!
I have html content in the post_content column.
I want to search and replace A with B but only the first time A appears in the record as it may appear more than once.
The below query would obviously replace all instances of A with B
UPDATE wp_posts SET post_content = REPLACE (post_content, 'A', 'B');
This should actually be what you want in MySQL:
UPDATE wp_post
SET post_content = CONCAT(REPLACE(LEFT(post_content, INSTR(post_content, 'A')), 'A', 'B'), SUBSTRING(post_content, INSTR(post_content, 'A') + 1));
It's slightly more complicated than my earlier answer - You need to find the first instance of the 'A' (using the INSTR function), then use LEFT in combination with REPLACE to replace just that instance, than use SUBSTRING and INSTR to find that same 'A' you're replacing and CONCAT it with the previous string.
See my test below:
SET #string = 'this is A string with A replace and An Answer';
SELECT #string as actual_string
, CONCAT(REPLACE(LEFT(#string, INSTR(#string, 'A')), 'A', 'B'), SUBSTRING(#string, INSTR(#string, 'A') + 1)) as new_string;
Produces:
actual_string new_string
--------------------------------------------- ---------------------------------------------
this is A string with A replace and An Answer this is B string with A replace and An Answer
Alternatively, you could use the functions LOCATE(), INSERT() and CHAR_LENGTH() like this:
INSERT(originalvalue, LOCATE('A', originalvalue), CHAR_LENGTH('A'), 'B')
Full query:
UPDATE wp_posts
SET post_content = INSERT(originalvalue, LOCATE('A', originalvalue), CHAR_LENGTH('A'), 'B');
With reference to https://dba.stackexchange.com/a/43919/200937 here is another solution:
UPDATE wp_posts
SET post_content = CONCAT( LEFT(post_content , INSTR(post_content , 'A') -1),
'B',
SUBSTRING(post_content, INSTR(post_content , 'A') +1))
WHERE INSTR(post_content , 'A') > 0;
If you have another string, e.g. testing then you need to change the +1 above to the according string length. We can use LENGTH() for this purpose. By the way, leave the -1 untouched.
Example: Replace "testing" with "whatever":
UPDATE wp_posts
SET post_content = CONCAT( LEFT(post_content , INSTR(post_content , 'testing') -1),
'whatever',
SUBSTRING(post_content, INSTR(post_content , 'testing') + LENGTH("testing"))
WHERE INSTR(post_content , 'testing') > 0;
By the way, helpful to see how many rows will be effected:
SELECT COUNT(*)
FROM post_content
WHERE INSTR(post_content, 'A') > 0;
If you are using an Oracle DB, you should be able to write something like :
UPDATE wp_posts SET post_content = regexp_replace(post_content,'A','B',1,1)
See here for more informations : http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions130.htm
Note : you really should take care of post_content regarding security issue since it seems to be an user input.
Greg Reda's solution did not work for me on strings longer than 1 character because of how the REPLACE() was written (only replacing the first character of the string to be replaced). Here is a solution that I believe is more complete and covers every use case of the problem when defined as How do I replace the first occurrence of "String A" with "String B" in "String C"?
CONCAT(LEFT(buycraft, INSTR(buycraft, 'blah') - 1), '', SUBSTRING(buycraft FROM INSTR(buycraft, 'blah') + CHAR_LENGTH('blah')))
This assumes that you are sure that the entry ALREADY CONTAINS THE STRING TO BE REPLACED! If you try replacing 'dog' with 'cat' in the string 'pupper', it will give you 'per', which is not what you want. Here is a query that handles that by first checking to see if the string to be replaced exists in the full string:
IF(INSTR(buycraft, 'blah') <> 0, CONCAT(LEFT(buycraft, INSTR(buycraft, 'blah') - 1), '', SUBSTRING(buycraft FROM INSTR(buycraft, 'blah') + CHAR_LENGTH('blah'))), buycraft)
The specific use case here is replacing the first instance of 'blah' inside column 'buycraft' with an empty string ''. I think a pretty intuitive and natural solution:
Find the index of the first occurrence of the string that is to be replaced.
Get everything to the left of that, not including the index itself (thus '-1').
Concatenate that with whatever you are replacing the original string with.
Calculate the ending index of the part of the string that is being replaced. This is easily done by finding the index of the first occurrence again, and adding the length of the replaced string. This will give you the index of the first char after the original string
Concatenate the substring starting at the ending index of the string
An example walkthrough of replacing "pupper" in "lil_puppers_yay" with 'dog':
Index of 'pupper' is 5.
Get left of 5-1 = 4. So indexes 1-4, which is 'lil_'
Concatenate 'dog' for 'lil_dog'
Calculate the ending index. Start index is 5, and 5 + length of 'pupper' = 11. Note that index 11 refers to 's'.
Concatenate the substring starting at the ending index, which is 's_yay', to get 'lil_dogs_yay'.
All done!
Note: SQL has 1-indexed strings (as an SQL beginner, I didn't know this before I figured this problem out). Also, SQL LEFT and SUBSTRING seem to work with invalid indexes the ideal way (adjusting it to either the beginning or end of the string), which is super convenient for a beginner SQLer like me :P
Another Note: I'm a total beginner at SQL and this is pretty much the hardest query I've ever written, so there may be some inefficiencies. It gets the job done accurately though.
I made the following little function and got it:
CREATE DEFINER=`virtueyes_adm1`#`%` FUNCTION `replace_first`(
`p_text` TEXT,
`p_old_text` TEXT,
`p_new_text` TEXT
)
RETURNS text CHARSET latin1
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT 'troca a primeira ocorrencia apenas no texto'
BEGIN
SET #str = p_text;
SET #STR2 = p_old_text;
SET #STR3 = p_new_text;
SET #retorno = '';
SELECT CONCAT(SUBSTRING(#STR, 1 , (INSTR(#STR, #STR2)-1 ))
,#str3
,SUBSTRING(#STR, (INSTR(#str, #str2)-1 )+LENGTH(#str2)+1 , LENGTH(#STR)))
INTO #retorno;
RETURN #retorno;
END
Years have passed since this question was asked, and MySQL 8 has introduced REGEX_REPLACE:
REGEXP_REPLACE(expr, pat, repl[, pos[, occurrence[, match_type]]])
Replaces occurrences in the string expr that match the regular
expression specified by the pattern pat with the replacement string
repl, and returns the resulting string. If expr, pat, or repl is NULL,
the return value is NULL.
REGEXP_REPLACE() takes these optional arguments:
pos: The position in expr at which to start the search. If omitted, the default is 1.
occurrence: Which occurrence of a match to replace. If omitted, the default is 0 (which means “replace all occurrences”).
match_type: A string that specifies how to perform matching. The meaning is as described for REGEXP_LIKE().
So, assuming you can use regular expressions in your case:
UPDATE wp_posts SET post_content = REGEXP_REPLACE (post_content, 'A', 'B', 1, 1);
Unfortunately for those of us on MariaDB, its REGEXP_REPLACE flavor is missing the occurrence parameter. Here's a regex-aware version of Andriy M's solution, conveniently stored as a reusable function as suggested by Luciano Seibel:
DELIMITER //
DROP FUNCTION IF EXISTS replace_first //
CREATE FUNCTION `replace_first`(
`i` TEXT,
`s` TEXT,
`r` TEXT
)
RETURNS text CHARSET utf8mb4
BEGIN
SELECT REGEXP_INSTR(i, s) INTO #pos;
IF #pos = 0 THEN RETURN i; END IF;
RETURN INSERT(i, #pos, CHAR_LENGTH(REGEXP_SUBSTR(i, s)), r);
END;
//
DELIMITER ;
It's simpler
UPDATE table_name SET column_name = CONCAT('A',SUBSTRING(column_name, INSTR(column_name, 'B') + LENGTH('A')));
For MYSQL version pre-5.6 and 8.0, I've used this pattern to fix my issue, it's a bit gross, but I hope it helps some of you guys:
SET #string = 'I love shop it is a terrific shop, I love eveything about it';
SET #shop_code = 'shop';
SET #shop_date = CONCAT(#shop_code, '__', DATE_FORMAT(NOW(), '%Y_%m_%d__%Hh%im%ss'));
SET #part1 = SUBSTRING_INDEX(#string, #shop_code, 1);
SET #shop_nb = ROUND( (LENGTH(#string) - LENGTH(REPLACE(#string, #shop_code,''))) / LENGTH(#shop_code) );
SET #part2 = SUBSTRING_INDEX(#string, #shop_code, -#shop_nb);
SET #string = CONCAT(#part1, #shop_date, #part2);
SELECT #string;
To keep the sample of gjreda a bit more simple use this:
UPDATE wp_post
SET post_content =
CONCAT(
REPLACE(LEFT(post_content, 1), 'A', 'B'),
SUBSTRING(post_content, 2)
)
WHERE post_content LIKE 'A%';