mariadb json decode - json

Question

I got a text column with JSON arrays.
+-------------------------------------------------------------------------+
| code |
+-------------------------------------------------------------------------+
| [] |
| ["5020","6420"] |
| ["4645","4775"] |
| ["6820"] |
| ["2442","2511"] |
| ["4675"] |
| ["1814","6201","6203","6209","6312","6399","7022","7311","7312","7320"] |
| ["6910","6920"] |
| ["4321","4329","4652","7112"] |
| ["4332","7739"] |
+-------------------------------------------------------------------------+
I need to get these codes to another table as single integer values. As soon as there are millions of records I would not like to fetch these records to php/ruby/etc for this task. I'd like to make it inside MariaDB with SQL.
The question is what could be a nice approach for such query?

Answer 1

In addition to the first posted answer about reading table in chunks, some other possible solutions:
1) If it needs to be done once or rarely, I would go with the low-tech approach:
select code into an outfile;
run a simple sed (remove square brackets and replace commas with new lines, thus getting one number per row, quote marks don't matter);
load the file into a table.
2) If you really want to stick with SQL only, there should be a procedure:
use cursor to go through each row;
use substring replacements to convert the array-like string ["1","2"] into the values-like string (1),(2) -- it's simple, although a bit ugly;
append the resulting string to INSERT INTO <table name> VALUES;
run it as a prepared statement;
3) If you want to go fancy and experiment (assuming it's not production), try JSON functions in MariaDB 10.2, also through a procedure:
use cursor to go through each row;
use JSON_LENGTH to get a number of elements in the array;
use a loop (e.g. WHILE) to JSON_EXTRACT each value in turn and either insert it immediately, or again build a prepared statement.

Answer 2

No. The SQL code would be so messy as to be worse overall.
I would read the table in chunks of 100-1000 rows, go through the arrays to get the numbers, de-dup (in php/etc) then do INSERT or INSERT IGNORE into the target table.
Tips on chunking (aimed at DELETE, but adaptable for SELECT): http://mysql.rjweb.org/doc.php/deletebig#deleting_in_chunks

Answer 3

For PHP. u can use it
$re = '/\d+/m';
$str = '+-------------------------------------------------------------------------+
| code |
+-------------------------------------------------------------------------+
| [] |
| ["5020","6420"] |
| ["4645","4775"] |
| ["6820"] |
| ["2442","2511"] |
| ["4675"] |
| ["1814","6201","6203","6209","6312","6399","7022","7311","7312","7320"] |
| ["6910","6920"] |
| ["4321","4329","4652","7112"] |
| ["4332","7739"] |
+-------------------------------------------------------------------------+';
preg_match_all($re, $str, $matches);
// Print the entire match result
print_r($matches);
Result:
Array
(
[0] => Array
(
[0] => 5020
[1] => 6420
[2] => 4645
[3] => 4775
[4] => 6820
[5] => 2442
[6] => 2511
[7] => 4675
[8] => 1814
[9] => 6201
[10] => 6203
[11] => 6209
[12] => 6312
[13] => 6399
[14] => 7022
[15] => 7311
[16] => 7312
[17] => 7320
[18] => 6910
[19] => 6920
[20] => 4321
[21] => 4329
[22] => 4652
[23] => 7112
[24] => 4332
[25] => 7739
)
)
u should do
$str = Result (MYSQL JSON)
Enjoy.