Is there a way to take out part of a json?
I just want the common object without zona and central.
I have the following table
CREATE TABLE FB_TAB
( COL CLOB COLLATE USING_NLS_COMP,
ID NUMBER,
TYPE VARCHAR2(20 BYTE) COLLATE USING_NLS_COMP,
COLOR VARCHAR2(20 BYTE) COLLATE USING_NLS_COMP,
AMOUNT NUMBER,
APP VARCHAR2(20 BYTE) COLLATE USING_NLS_COMP,
CONSTRAINT JSON_CON_1 CHECK (col IS JSON) ENABLE
)
and its insert
insert into fb_tab
values('
{"common":{"contrato":{"id":"1","codigo":"054AKSDJ","nombre":"BUCLE"},"servicio":"","actividad":"Apertura","tipo_actividad":"BAJA","numero_administrativo":"","estado_origen":"Pendiente","provincia":{"id":"24","nombre":"León"},"aplicacion_origen":{"id":"1","nombre":"VISORD"},"zona":{"pais":"ES","cliente":"TL","div_geo":"2410002"},"central":{"codigo":"2410002","nombre":"Leon-Torre"},"clave":{"act_domiciliaria":"","prioridad":""}},"app_log":{"app_name":"client_mobile"}}
', 23, 'Ball', 'Red', 15, 'Mobile');
commit;
I want to get the next JSON as a result
{"Type":"Ball","Color":"Red","App":"Mobile","Amount":"15","my_json":{"contrato":{"id":"1","codigo":"054AKSDJ","nombre":"BUCLE"},"servicio":"","actividad":"Apertura","tipo_actividad":"BAJA","numero_administrativo":"","estado_origen":"Pendiente","provincia":{"id":"24","nombre":"León"},"aplicacion_origen":{"id":"1","nombre":"VISORD"},"clave":{"act_domiciliaria":"","prioridad":""}}}
I'm trying with this query
SELECT JSON_OBJECT (
'Type' value to_char(a.Type),
'Color' value to_char(a.Color),
'App' value to_char(a.App),
'Amount' value to_char(a.Amount),
'my_json' VALUE treat ( JSON_QUERY(a.col, '$.common' WITHOUT WRAPPER) as json )
)
--into json_output
FROM FB_TAB a
where a.id = :id;
but my actual result is this
{"Type":"Ball","Color":"Red","App":"Mobile","Amount":"15","my_json":{"contrato":{"id":"1","codigo":"054AKSDJ","nombre":"BUCLE"},"servicio":"","actividad":"Apertura","tipo_actividad":"BAJA","numero_administrativo":"","estado_origen":"Pendiente","provincia":{"id":"24","nombre":"León"},"aplicacion_origen":{"id":"1","nombre":"VISORD"},"zona":{"pais":"ES","cliente":"TL","div_geo":"2410002"},"central":{"codigo":"2410002","nombre":"Leon-Torre"},"clave":{"act_domiciliaria":"","prioridad":""}}}
I don't want to see zona and central
Is there a way to do this ?
Best regards
You have a mistake inside the JSON you're trying to insert into the table
Here's the revised insert statement
I've broke it down in pieces trying to fix it, feel free to "unwrap" it in a single line
insert into fb_tab
values('
{"common":
{"contrato":{"id":"1","codigo":"054AKSDJ","nombre":"BUCLE"},
"servicio":"",
"actividad":"Apertura",
"tipo_actividad":"BAJA",
"numero_administrativo":"",
"estado_origen":"Pendiente",
"provincia":{"id":"24","nombre":"León"},
"aplicacion_origen":{"id":"1","nombre":"VISORD"},
"clave":{"act_domiciliaria":"","prioridad":""}
},
"zona":{"pais":"ES","cliente":"TL","div_geo":"2410002"},
"central":{"codigo":"2410002","nombre":"Leon-Torre"},
"app_log":{"app_name":"client_mobile"}
}',
23, 'Ball', 'Red', 15, 'Mobile');
Basically you were placing clave after central and closing it with two curly brackets (}) making the common to include zona and central, that you were trying to exclude from the result of the query.
Now when you query the table
SELECT JSON_OBJECT (
'Type' value to_char(a.Type),
'Color' value to_char(a.Color),
'App' value to_char(a.App),
'Amount' value to_char(a.Amount)
,'my_json' VALUE treat ( JSON_QUERY(a.col, '$.common' WITHOUT WRAPPER) as json )
)
FROM FB_TAB a
where a.id = 23;
Removed the :id bind to ease the debugging process
You get the desired result
{"Type":"Ball","Color":"Red","App":"Mobile","Amount":"15","my_json":{"contrato":{"id":"1","codigo":"054AKSDJ","nombre":"BUCLE"},"servicio":"","actividad":"Apertura","tipo_actividad":"BAJA","numero_administrativo":"","estado_origen":"Pendiente","provincia":{"id":"24","nombre":"León"},"aplicacion_origen":{"id":"1","nombre":"VISORD"},"clave":{"act_domiciliaria":"","prioridad":""}}}
Related
I have a table with about 900,000 rows. One of the columns has a 7 character value that I need to search for client-side validation of user input. I'm currently using ajax as the user types, but most of the users can out-run the ajax round trips and end up having to wait until the all the validation calls return. So I want to shift the wait time to the initial load of the app and take advantage of browser caching. So I'll bundle minify and gzip the json file with webpack. I'll probably make it an entry that I can then require/ensure as the app loads.
To make the validation super fast on the client side I want to produce a json file that has a single json structure with the first two characters of the 7 character column as the key to the object with an array of all values that start with the first two characters as an array in the value for said key (see example below). I can then use indexOf to find the value within this segmented list and it will be very quick.
As mentioned above I'm currently using ajax as the user types.
I'm not going to show my code because it's too complex. But I basically keep track of the pending ajax requests and when the request that started with the last value the user entered returns (the value currently sitting in the text box), then I can show the user if the entry exists or not. That way if the request return out of order, I'm not giving false positives.
I'm using SQL Server 2016 so I want to use for json to produce my desired output. But here's what I want to produce:
{
"00": [ "0000001", "0000002", ... ],
"10": [ "1000000", "1000001", ... ],
//...
"99": [ "9900000", "9900001", ...]
}
So far I have been unable to figure out how to use substring( mySevenDigitCol, 1, 2 ) as the key in the json object.
I'm not sure if this can be done using FOR JSON AUTO (["0000001", "0000002", ... ] is the difficult part), but next approach based on string manipulation is one possible solution to your problem:
Input:
CREATE TABLE #Data (
SevenDigitColumn varchar(7)
)
INSERT INTO #Data
(SevenDigitColumn)
VALUES
('0000001'),
('0000002'),
('0000003'),
('0000004'),
('0000005'),
('1000001'),
('1000002'),
('9900001'),
('9900002')
T-SQL:
;WITH JsonData AS (
SELECT
SUBSTRING(dat.SevenDigitColumn, 1, 2) AS [Key],
agg.StringAgg AS [Values]
FROM #Data dat
CROSS APPLY (
SELECT STUFF(
(
SELECT CONCAT(',"', SevenDigitColumn, '"')
FROM #Data
WHERE SUBSTRING(SevenDigitColumn, 1, 2) = SUBSTRING(dat.SevenDigitColumn, 1, 2)
FOR XML PATH('')
), 1, 1, '') AS StringAgg
) agg
GROUP BY SUBSTRING(dat.SevenDigitColumn, 1, 2), agg.StringAgg
)
SELECT CONCAT(
'{',
STUFF(
(
SELECT CONCAT(',"', [Key], '": [', [Values], ']')
FROM JsonData
FOR XML PATH('')
), 1, 1, ''),
'}')
Output:
{"00": ["0000001","0000002","0000003","0000004","0000005"],"10": ["1000001","1000002"],"99": ["9900001","9900002"]}
Notes:
With SQL Server 2017+ you can use STRING_AGG() function:
SELECT
CONCAT(
'{',
STRING_AGG(KeyValue, ','),
'}'
)
FROM (
SELECT CONCAT(
'"',
SUBSTRING(dat.SevenDigitColumn, 1, 2),
'": [',
STRING_AGG('"' + SevenDigitColumn + '"', ','),
']'
) AS KeyValue
FROM #Data dat
GROUP BY SUBSTRING(dat.SevenDigitColumn, 1, 2)
) JsonData
Notes:
If your column data is not only digits, you should use STRING_ESCAPE() with 'json' as second parameter to escape special characters.
I am trying to Insert/update a string (JSON). While inserting/Updating the value I get data truncation error.
I have tried making a JSON type column and passing an JSON_OBJECT() type but that fails as well.
select '''[{"id":"1202","title":"Asian","notes":"","active":"1"}]''';
CREATE TABLE mktesttable (
id int NOT NULL,
s VARCHAR(34530) NOT NULL
);
INSERT INTO mktesttable
select 1, '''[{"id":"1202","title":"Asian","notes":"","active":"1"}]''';
select * from mktesttable;
// That Works
INSERT INTO mktesttable
SELECT
patient_data.id,
CONCAT(
'''[{"id":"', patient_data.race,
'","title":"', list_options.title,
'","notes":"', list_options.notes,
'","active":"', list_options.active,
'"}]'''
) as s
FROM
patient_data
INNER JOIN list_options
ON patient_data.race = list_options.id order by 1 desc
Yields same result (Id's and data varies) but doesn't work
Result Set
If you want to store JSON object, you should use the JSON datatype instead of strings. To create a JSON object, you can use JSON_OBJECT.
CREATE TABLE mktesttable (
id int NOT NULL,
s JSON NOT NULL
);
INSERT INTO mktesttable
SELECT
patient_data.id,
JSON_OBJECT(
'id', patient_data.race,
'title', list_options.title,
'notes', list_options.notes,
'active', list_options.active,
)
FROM
patient_data
INNER JOIN list_options
ON patient_data.race = list_options.id
ORDER BY patient_data.id desc
If you need a JSON array as shown in your sample data, then :
JSON_ARRAY(
JSON_OBJECT(
'id', patient_data.race,
'title', list_options.title,
'notes', list_options.notes,
'active', list_options.active,
)
)
How can I update an object in an array based on a unique value in the object?
Let's say this is my json object stored in a table called objects and in a column called content
table: objects
id: 7383
content: { data:[{id: 111, active: 1 }, {id: 222, active: 1 }, {id: 333, active: 0 }] }
I can update objects if I know the position of the element in the array with
SET content = JSON_REPLACE(content,'$.data[1].active', 0)
Where id = 7383
However, if I don't know the position of the array, but I do know the value of id (for example 222) in the object, how can I update active to 0 for the object that has id: 222 ?
Currently, it's complicated to look up numerical values with MySQL JSON functions. In a JSON like the following, it would be simple:
{"id": "222", "active": 1}
There are many ways to get what you need, I present one that can give you ideas (modify everything that is necessary):
UPDATE `objects`
SET `objects`.`content` =
JSON_REPLACE(`objects`.`content`, CONCAT('$.data',
(SELECT
JSON_UNQUOTE(
REPLACE(
JSON_SEARCH(
REPLACE(
REPLACE(
REPLACE(
`der`.`content` ->> '$.data[*].id',
', ',
'","'),
']',
'"]'),
'[',
'["'),
'one',
'222'),
'$',
'')
)
FROM (SELECT `objects`.`content`
FROM `objects`
WHERE `objects`.`id` = 7383) `der`
), '.active'), 0)
WHERE `objects`.`id` = 7383;
Beware of possible performance problems.
See dbfiddle.
In the most recent version of MySQL (>= 8.0.4), the sentence would be much simpler:
UPDATE `objects`
INNER JOIN JSON_TABLE(
`objects`.`content`,
'$.data[*]' COLUMNS(
`rowid` FOR ORDINALITY,
`id` INT PATH '$.id'
)
) `der` ON `der`.`id` = 222
SET `objects`.`content` =
JSON_REPLACE(
`objects`.`content`,
CONCAT('$.data[', `der`.`rowid` - 1, '].active'),
0)
WHERE
`objects`.`id` = 7383;
See db-fiddle.
It can be achieved by combining the functions JSON_SEARCH, which returns a dirty json path to the item you need, and then, extract the value of the jsonpath with an array index, concatenate it with subpath we want to update and use JSON_SET to set a new value to the final json path (tested with MySQL 5.7.32):
-- INPUT ------------------------------------------------
-- unique value for an object in the array
SET #unique_value = "12345";
-- object field we want to update
SET #field_to_update = '.myField';
-- new value
SET #new_value = 1;
-- PROCESSING ------------------------------------------
-- Get json path to the item with specified #unique_value
-- RESULT: $.data[6].id
SET #temp_path = ( TRIM(BOTH '"' FROM ( SELECT JSON_SEARCH(json, 'one', #unique_value, NULL, "$.data")
FROM `my-table`
WHERE `column1` = "abcd" ) ));
-- We are looking for the bracket that delimits index within the array of documents: [11]
SET #closing_bracket_index = (SELECT LOCATE(']', #temp_path));
-- Get json path with index of an object for #unique_value
-- in MySQL, string indexing starts from position 1, not a zero
-- RESULT: $.data[6]
SET #item_path = ( SELECT SUBSTRING(#temp_path, 1, #closing_bracket_index) );
-- $.data[6].myFIeld
SET #item_path_to_update = ( SELECT CONCAT(#item_path, #field_to_update) );
-- UPDATE JSON STATEMENT
UPDATE `my-table`
SET json = JSON_SET(json-column, #item_path_to_update, #new_value)
WHERE `column1` = "abcd";
Using only the native JSON fuctions (no PHP, etc) in MySQL version 5.7.12 (section 13.16 in the manual) I am trying to write a query to generate a JSON document from relational tables that contains a sub object. Given the following example:
CREATE TABLE `parent_table` (
`id` int(11) NOT NULL,
`desc` varchar(20) NOT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `child_table` (
`id` int(11) NOT NULL,
`parent_id` int(11) NOT NULL,
`desc` varchar(20) NOT NULL,
PRIMARY KEY (`id`,`parent_id`)
);
insert `parent_table` values (1,'parent row 1');
insert `child_table` values (1,1,'child row 1');
insert `child_table` values (2,1,'child row 2');
I am trying to generate a JSON document that looks like this:
[{
"id" : 1,
"desc" : "parent row 1",
"child_objects" : [{
"id" : 1,
"parent_id" : 1,
"desc" : "child row 1"
}, {
"id" : 2,
"parent_id" : 1,
"desc" : "child row 2"
}
]
}]
I am new to MySQL and suspect there is a SQL pattern for generating nested JSON objects from one to many relationships but I'm having trouble finding it.
In Microsoft SQL (which I'm more familiar with) the following works:
select
[p].[id]
,[p].[desc]
,(select * from [dbo].[child_table] where [parent_id] = [p].[id] for json auto) AS [child_objects]
from [dbo].[parent_table] [p]
for json path
I attempted to write the equivalent in MySQL as follows:
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',(select json_object('id',id,'parent_id',parent_id,'desc',`desc`)
from child_table where parent_id = p.id)
)
from parent_table p;
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',json_array((select json_object('id',id,'parent_id',parent_id,'desc',`desc`)
from child_table where parent_id = p.id))
)
from parent_table p
Both attempts fail with the following error:
Error Code: 1242. Subquery returns more than 1 row
The reason you are getting these errors is that the parent json object is not expecting a result set as one of its inputs, you need to have simple object pairs like {name, string} etc bug report - may be available in future functionality... this just means that you need to convert your multi row results into a concatination of results separated by commas and then converted into a json array.
You almost had it with your second example.
You can achieve what you are after with the GROUP_CONCAT function
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',json_array(
(select GROUP_CONCAT(
json_object('id',id,'parent_id',parent_id,'desc',`desc`)
)
from child_table
where parent_id = p.id))
)
from parent_table p;
This almost works, it ends up treating the subquery as a string which leaves the escape characters in there.
'{\"id\": 1,
\"desc\": \"parent row 1\",
\"child_objects\":
[\"
{\\\"id\\\": 1,
\\\"desc\\\": \\\"child row 1\\\",
\\\"parent_id\\\": 1
},
{\\\"id\\\": 2,
\\\"desc\\\": \\\"child row 2\\\",
\\\"parent_id\\\": 1}\"
]
}'
In order to get this working in an appropriate format, you need to change the way you create the JSON output as follows:
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',(select CAST(CONCAT('[',
GROUP_CONCAT(
JSON_OBJECT(
'id',id,'parent_id',parent_id,'desc',`desc`)),
']')
AS JSON) from child_table where parent_id = p.id)
) from parent_table p;
This will give you the exact result you require:
'{\"id\": 1,
\"desc\": \"parent row 1\",
\"child_objects\":
[{\"id\": 1,
\"desc\": \"child row 1\",
\"parent_id\": 1
},
{\"id\": 2,
\"desc\": \"child row 2\",
\"parent_id\": 1
}]
}'
For MariaDb, CAST AS JSON does not work. But JSON_EXTRACT may be used to convert a string to a JSON object:
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',JSON_EXTRACT(IFNULL((select
CONCAT('[',GROUP_CONCAT(
json_object('id',id,'parent_id',parent_id,'desc',`desc`)
),']')
from child_table where parent_id = p.id),'[]'),'$')
) from parent_table p;
I tried group_concat solution but I found it's problems larger string because of group_concat limitations (group_concat_max_len).
I wrote the new function resolve the problem about converting a string to JSON object as bellow and how to use it.
Tested on MariaDB 10.5.12
Usage: https://i.stack.imgur.com/cWfd7.jpg
CREATE FUNCTION `ut_tf_array`(input_json longtext) RETURNS longtext CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci
COMMENT 'Function for transform json array agg'
BEGIN
DECLARE transformed_data_list longtext ;
DECLARE record longtext ;
DECLARE i_count int ;
DECLARE i_count_items int ;
SET i_count = 0;
SET i_count_items = JSON_LENGTH(JSON_EXTRACT(input_json,'$'));
SET transformed_data_list = '[]';
-- return array with length = zero
IF input_json is NULL THEN
RETURN transformed_data_list;
END IF;
WHILE i_count < i_count_items DO
-- fetch into record
SELECT JSON_EXTRACT( JSON_EXTRACT( input_json ,'$') , CONCAT('$[',i_count,']')) INTO record;
-- append to transformed_data_list
SELECT JSON_ARRAY_APPEND(transformed_data_list, '$', JSON_EXTRACT(record, '$')) into transformed_data_list;
SET i_count := i_count + 1;
END WHILE;
-- done
RETURN transformed_data_list;
END
Below Query works for me.
SELECT JSON_ARRAYAGG(JSON_OBJECT('Id', p.id, 'desc', p.`desc`, 'child_objects', temp_json)) AS json_value
FROM (
SELECT p.id, p.`desc`,
JSON_ARRAYAGG(JSON_OBJECT('id', p.id, 'parent_id', p.parent_id, 'desc', p.`desc`)) AS temp_json
FROM parent_table p
GROUP BY p.id
) t;
(NOTE: This question is not about escaping queries, it's about escaping results)
I'm using GROUP_CONCAT to combine multiple rows into a comma delimited list. For example, assume I have the two (example) tables:
CREATE TABLE IF NOT EXISTS `Comment` (
`id` int(11) unsigned NOT NULL auto_increment,
`post_id` int(11) unsigned NOT NULL,
`name` varchar(255) collate utf8_unicode_ci NOT NULL,
`comment` varchar(255) collate utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
KEY `post_id` (`post_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=6 ;
INSERT INTO `Comment` (`id`, `post_id`, `name`, `comment`) VALUES
(1, 1, 'bill', 'some comment'),
(2, 1, 'john', 'another comment'),
(3, 2, 'bill', 'blah'),
(4, 3, 'john', 'asdf'),
(5, 4, 'x', 'asdf');
CREATE TABLE IF NOT EXISTS `Post` (
`id` int(11) NOT NULL auto_increment,
`title` varchar(255) collate utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=7 ;
INSERT INTO `Post` (`id`, `title`) VALUES
(1, 'first post'),
(2, 'second post'),
(3, 'third post'),
(4, 'fourth post'),
(5, 'fifth post'),
(6, 'sixth post');
And I want to list all posts along with a list of each username who commented on the post:
SELECT
Post.id as post_id, Post.title as title, GROUP_CONCAT(name)
FROM Post
LEFT JOIN Comment on Comment.post_id = Post.id
GROUP BY Post.id
gives me:
id title GROUP_CONCAT( name )
1 first post bill,john
2 second post bill
3 third post john
4 fourth post x
5 fifth post NULL
6 sixth post NULL
This works great, except that if a username contains a comma it will ruin the list of users. Does MySQL have a function that will let me escape these characters? (Please assume usernames can contain any characters, since this is only an example schema)
Actually, there are ascii control characters specifically designed for separating database fields and records:
0x1F (31): unit (fields) separator
0x1E (30): record separator
0x1D (29): group separator
Read more: about ascii characters
You will never have them in usernames and most probably never in any other non-binary data in your database so they can be used safely:
GROUP_CONCAT(foo SEPARATOR 0x1D)
Then split by CHAR(0x1D) in whatever client language you want.
If there's some other character that's illegal in usernames, you can specify a different separator character using a little-known syntax:
...GROUP_CONCAT(name SEPARATOR '|')...
... You want to allow pipes? or any character?
Escape the separator character, perhaps with backslash, but before doing that escape backslashes themselves:
group_concat(replace(replace(name, '\\', '\\\\'), '|', '\\|') SEPARATOR '|')
This will:
escape any backslashes with another backslash
escape the separator character with a backslash
concatenate the results with the separator character
To get the unescaped results, do the same thing in the reverse order:
split the results by the separator character where not preceded by a backslash. Actually, it's a little tricky, you want to split it where it isn't preceded by an odd number of blackslashes. This regex will match that:
(?<!\\)(?:\\\\)*\|
replace all escaped separator chars with literals, i.e. replace \| with |
replace all double backslashes with singe backslashes, e.g. replace \\ with \
REPLACE()
Example:
... GROUP_CONCAT(REPLACE(name, ',', '\\,'))
Note you have to use a double-backslash (if you escape the comma with backslash) because backslash itself is magic, and \, becomes simply ,.
I'd suggest GROUP_CONCAT(name SEPARATOR '\n'), since \n usually does not occur. This might be a little simpler, since you don't need to escape anything, but could lead to unexpected problems. The encodeing/regexp decoding stuff as proposed by nick is of course nice too.
If you're going to be doing the decoding in your application, maybe just use hex:
SELECT GROUP_CONCAT(HEX(foo)) ...
or you could also put the length in them:
SELECT GROUP_CONCAT(CONCAT(LENGTH(foo), ':', foo)) ...
Not that I tested either :-D
what nick said really, with an enhancement - the separator can be more than one character too.
I've often used
GROUP_CONCAT(name SEPARATOR '"|"')
Chances of a username containing "|" are fairly low i'd say.
You're getting into that gray area where it might be better to postprocess this outside the world of SQL.
At least that's what I'd do: I'd just ORDER BY instead of GROUP BY, and loop through the results to handle the grouping as a filter done in the client language:
Start by initializing last_id to NULL
Fetch the next row of the resultset (if there aren't more rows go to step 6)
If the id of the row is different than last_id start a new output row:
a. if last_id isn't NULL then output the grouped row
b. set the new grouped row = the input row, but store the name as a single element array
c. set last_id to the value of the current ID
Otherwise (id is the same as last_id) append the row name onto the existing grouped row.
Go back to step 2
Otherwise you have finished; if the last_id isn't NULL then output the existing group row.
Then your output ends up including names organized as an array and can decide how you want to handle/escape/format them then.
What language/system are you using? PHP? Perl? Java?
Jason S: This is exactly the issue I'm dealing with. I'm using an PHP MVC framework and was processing the results like you describe (multiple rows per result and code to group the results together). However, I've been working on two functions for my models to implement. One returns a list of all necessary fields needed to recreate the object and the other is a function that given a row with the fields from the first function, instantiate a new object. This lets me request a row from the database and easily turn it back into the object without knowing the internals of the data needed by the model. This doesn't work quite as well when multiple rows represent one object, so I was trying to use GROUP_CONCAT to get around that problem.
Right now I'm allowing any character. I realize a pipe would be unlikely to show up, but I'd like to allow it.
How about a control character, which you should be stripping out of application input anyway? I doubt you need eg. a tab or a newline in a name field.
Just to expand on some of the answers, I implemented #derobert 's second suggestion in PHP and it works well. Given MySQL such as:
GROUP_CONCAT(CONCAT(LENGTH(field), ':', field) SEPARATOR '') AS fields
I used the following function to split it:
function concat_split( $str ) {
// Need to guard against PHP's stupid multibyte string function overloading.
static $mb_overload_string = null;
if ( null === $mb_overload_string ) {
$mb_overload_string = defined( 'MB_OVERLOAD_STRING' )
&& ( ini_get( 'mbstring.func_overload' ) & MB_OVERLOAD_STRING );
}
if ( $mb_overload_string ) {
$mb_internal_encoding = mb_internal_encoding();
mb_internal_encoding( '8bit' );
}
$ret = array();
for ( $offset = 0; $colon = strpos( $str, ':', $offset ); $offset = $colon + 1 + $len ) {
$len = intval( substr( $str, $offset, $colon ) );
$ret[] = substr( $str, $colon + 1, $len );
}
if ( $mb_overload_string ) {
mb_internal_encoding( $mb_internal_encoding );
}
return $ret;
}
I also initially implemented #ʞɔıu 's suggestion, using one of #Lemon Juice 's separators. It worked fine but apart from its complication it was slower, the main problem being that PCRE only allows fixed length lookbehind so using the suggested regex to split requires capturing the delimiters, otherwise doubled backslashes at the end of strings will be lost. So given MySQL such as (note 4 PHP backslashes => 2 MySQL backslashes => 1 real backslash):
GROUP_CONCAT(REPLACE(REPLACE(field, '\\\\', '\\\\\\\\'),
CHAR(31), CONCAT('\\\\', CHAR(31))) SEPARATOR 0x1f) AS fields
the split function was:
function concat_split( $str ) {
$ret = array();
// 4 PHP backslashes => 2 PCRE backslashes => 1 real backslash.
$strs = preg_split( '/(?<!\\\\)((?:\\\\\\\\)*+\x1f)/', $str, -1, PREG_SPLIT_DELIM_CAPTURE );
// Need to add back any captured double backslashes.
for ( $i = 0, $cnt = count( $strs ); $i < $cnt; $i += 2 ) {
$ret[] = isset( $strs[ $i + 1 ] ) ? ( $strs[ $i ] . substr( $strs[ $i + 1 ], 0, -1 ) ) : $strs[ $i ];
}
return str_replace( array( "\\\x1f", "\\\\" ), array( "\x1f", "\\" ), $ret );
}