I am using MySQL 5.7 and have a table with following schema
CREATE TABLE `Test` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`created_by` varchar(45) COLLATE utf8_unicode_ci DEFAULT NULL,
`status` varchar(45) COLLATE utf8_unicode_ci DEFAULT NULL,
`metadata` blob COMMENT 'to capture the custom metadata',
`created_at` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci'
And the sample row data for the table looks like this
1234,user1,open,"{'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}",2021-05-18 16:01:25
I want to select rows from this table based on the keys in json blob field metadata; for example, let's say where key1 = 'value1'. So I tried something like this
select * from `test` where metadata->>"$.key1" = "value1";
But I got this error Cannot create a JSON value from a string with CHARACTER SET 'binary'. So I casted it to json first by something like below
select JSON_EXTRACT(CAST(metadata as JSON), "$") as meta from test;
The problem is this returns base64 encoded string and when I try to decode the same using FROM_BASE64 like below I get null values in the column.
select FROM_BASE64(JSON_EXTRACT(CAST(metadata as JSON), "$")) as meta from test;
So I think I have two problems here: the first one being how to decode the base64 encoded data which I get after casting blob as json, and second how to filter the rows based on keys in the metadata field.
I do feel this as a design error where the most ideal data type should have been json but since this is how it is now, I need some way to workaround this.
Edit
I also tried following as suggested in one of the comments
select cast(convert(cast(metadata as char) using utf8) as json) from test;
but I get this error
Data truncation: Invalid JSON text in argument 1 to function cast_as_json: "Missing a name for object member." at position 1
Is there any way I can work around this ?
Related
I use MySQL5.7 and store json in mysql, now i need to do some analysis on this json.
Such as below table:
CREATE TABLE `json_test1` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(128) NOT NULL,
`json` json DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB
insert test data:
insert into json_test1 (name, json)
values ('lily', '{"age": 22, "lily": "ldfd"}'),
('sam','{"sam":"dfsf"}'),
('k','{"arm":"aaa"}'),
('cd','{"xl":"bbb"}');
Now, i need to check if json key contains name, for example: ('sam','{"sam":"sam"}'), if json contains key 'sam', so for this single one, i can use below sql:
select json_contains_path(json, 'one','$.sam') jsonC
from json_test1 where name='sam';
Output is 1, so json contains key 'sam', but how can i check all these data to see if key exist in json, in this condition, i need pass a variable to function json_contains_path(json, 'one','$.{name}') , but i don't know how to set variable in this function.
Expected result:
Anyone know that?
You just need to add a CONCAT() function such as
SELECT name, JSON_CONTAINS_PATH(json, 'one',CONCAT('$."',name,'"')) AS jsonExists
FROM json_test1
Demo
I'm having issues executing the following MySQL statement with the ocaml-mysql (latest version) library:
let dump_to_db text =
let insert = P.create db (s "INSERT INTO Temperature VALUES (?,?,?,?)") in
ignore (P.execute insert [| "NULL"; "xxx.xxx.xxx.xxx"; text ; "CURRENT_TIMESTAMP" |])
My problem is that the timestamp field is not set correctly. Apparently, whatever I write into the last field (might it be "CURRENT_TIMESTAMP" or simply "NULL"), I'm not able to get the proper outcome.
What happens is that the query get executed and what I see inside the table is a NULL-filled Timestamp field (even tho I have no idea how it can be possible because is defined as a NOT NULL column).
My table structure follows:
CREATE TABLE `Temperature` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`IP` varchar(100) DEFAULT NULL,
`Value` varchar(100) DEFAULT NULL,
`Timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1
Prepared.execute takes array with values not arbitrary expressions (function calls). String value "CURRENT_TIMESTAMP" gets converted into timestamp and as it is not recognized as valid number is turned into zero.
Pass NULL properly and it will be default initialized according to CREATE TABLE :
P.execute_null insert [| None; Some "xxx.xxx.xxx.xxx"; Some text ; None |]
I have a table that looks like this:
CREATE TABLE `testtable` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`data` json DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The json field has data that looks like this:
{'a1': 'Title', 'b17': 'Message'}
I want to select id and a1(json). I don't want b17. Is there a way to do this?
If you are using MySQL 5.7.8 or newer, you can take advantage of the JSON data type supported there:
https://dev.mysql.com/doc/refman/5.7/en/json.html
You can then use JSON Path expressions to extract values from the field.
Otherwise, you are stuck with either extracting the value through SUBSTR() and POSITION() functions - hokey example below - assuming the formatting of the values is sufficiently predictable, or else processing the result outside of SQL.
SELECT id, SUBSTR(hackedJSON, 1, POSITION("'" IN hackedJSON) - 1) a1
FROM (
SELECT id, SUBSTR(data, POSITION("'a1':" IN data) + 7) hackedJSON
FROM testtable) a
i'm relaunching a website a found these entries in one of the database tables. I'm wondering what format it is and how i can easily recreate data of a form into this format:
CREATE TABLE IF NOT EXISTS `phpwcms_formresult` (
`formresult_id` int(11) NOT NULL AUTO_INCREMENT,
`formresult_pid` int(11) NOT NULL DEFAULT '0',
`formresult_createdate` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`formresult_ip` varchar(50) NOT NULL DEFAULT '',
`formresult_content` mediumtext NOT NULL,
PRIMARY KEY (`formresult_id`),
KEY `formresult_pid` (`formresult_pid`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
One entry looks like this:
(3179, 7, '2013-10-02 09:35:48', ':11', 'a:14:{s:13:"veranstaltung";s:4:"name";s:11:"Nick Habbel";s:22:"erziehungsberechtigter";s:12:"Petra Habbel".....;}');
In front of every string there is the length of the string "s:22: ...". What kind of format is this?
Any hint is much appreciated. Thanks!
An example of this value is done by the serialization of an array in php:
$arr = array("name"=>"Nick Habbel");
echo serialize($arr);
# output:
# a:1:{s:4:"name";s:11:"Nick Habbel";}
So, the application that inserts data in your table seems to serialize an array before the insertion. In your example an array (a) with a cardinality of 14 elements (a:14:) plus key/values with lengths and type "s" for strings, "i" for integers etc (s:13:"veranstaltung")
I have a weird problem with a MySQL users table. I have quickly created a simplified version as a testcase.
I have the following table
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`identity` varchar(255) NOT NULL,
`credential` varchar(255) NOT NULL,
`credentialSalt` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=ucs2 AUTO_INCREMENT=2 ;
INSERT INTO `users` (`id`, `identity`, `credential`, `credentialSalt`) VALUES
(1, 'test', '7288edd0fc3ffcbe93a0cf06e3568e28521687bc', '123');
And I run the following query
SELECT id,
IF (credential = SHA1(CONCAT('test', credentialSalt)), 1, 0) AS dynamicSaltMatches,
credentialSalt AS dynamicSalt,
SHA1(CONCAT('test', credentialSalt)) AS dynamicSaltHash,
IF (credential = SHA1(CONCAT('test', 123)), 1, 0) AS staticSaltMatches,
123 AS staticSalt,
SHA1(CONCAT('test', 123)) AS staticSaltHash
FROM users
WHERE identity = 'test'
Which gives me the following result
The dynamic salt does NOT match while the static salt DOES match.
This is blowing my mind. Can someone help me point out the cause of this?
My MySQL version is 5.5.29
It's because of the default character set of your table. You appear to be running this on a UTF8 database and something in SHA1() is having problems with the differing character sets.
If you change your table declaration to the following it will match again:
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`identity` varchar(255) NOT NULL,
`credential` varchar(255) NOT NULL,
`credentialSalt` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2 ;
SQL Fiddle
As robertklep commented explicitly casting your string to a character will also work, basically ensure you're using the same characterset when doing comparisons using SHA1()
As the encryption functions documentation says:
Many encryption and compression functions return strings for which the result might contain arbitrary byte values. If you want to store these results, use a column with a VARBINARY or BLOB binary string data type. This will avoid potential problems with trailing space removal or character set conversion that would change data values, such as may occur if you use a nonbinary string data type (CHAR, VARCHAR, TEXT).
This was changed in version 5.5.3:
As of MySQL 5.5.3, the return value is a nonbinary string in the connection character set. Before 5.5.3, the return value is a binary string; see the notes at the beginning of this section about using the value as a nonbinary string.