I am using mysql 5.6 and it will not be feasible for me to upgrade it to 5.7. I have a table which stores json as an attribute. Attaching screenshot for reference.
Here, the column policy_status contains status and values of different policies as json for each user.
How can I find the list of users, say, with appVersion' status as success and value = 1437.
I got a few references online but as I am new to stored procedures I am not able to reach a solution. I will appreciate any help. Thanks in advance.
It is not efficient at all but may can help you with ideas:
SELECT *
FROM data
WHERE
(LOCATE('"employmentType":["status":"success"]', policy_status) > 0
AND
LOCATE('"value": 1', policy_status) > 0);
Using the LOCATE function you can see whether the field contains your desired appVersion and value strings. See sqlfiddle demo here.
Where the simple test data:
CREATE TABLE data (
id INT UNSIGNED NOT NULL,
policy_status TEXT
);
INSERT INTO data (id, policy_status) VALUES
(1,'{"employmentType":["status":"success"], "value": 1}'),
(2,'{"employmentType":["status":"no"], "value": 1}'),
(3,'{"employmentType":["status":"no"], "value": 0}'),
(4,'{"employmentType":["status":"success"], "value": 0}'),
(5,'{"employmentType":["status":"no"], "value": 1}');
gives the result:
{"employmentType":["status":"success"], "value": 1}
Where both strings are found.
UPDATE:
Also if you can add FULLTEXT index for your policy_status column than you can use fulltext search in the WHERE clause:
...
WHERE
MATCH (policy_status) AGAINST ('+"employmentType:[status:success]" +"value: 1"' IN BOOLEAN MODE)
Note the + and " characters in AGAINST(...). They are special boolean full-text search operators. See here.
A leading or trailing plus sign indicates that this word must be
present in each row that is returned
and
A phrase that is enclosed within double quote (") characters matches
only rows that contain the phrase literally, as it was typed. The
full-text engine splits the phrase into words and performs a search in
the FULLTEXT index for the words. Nonword characters need not be
matched exactly.
If it is not an option in your case, you can use LIKE for matching the substrings:
...
WHERE
(policy_status LIKE '%"employmentType":["status":"success"]%'
AND
policy_status LIKE '%"value": 1%');
See sqlfiddle demo for both.
Related
I use full text indexing to find results faster and it works well except when the term i search for is attached to an underscore inside the database record.
My database records:
article.title
++++++++++++++++++++++++++++++
My article 123456 created
------------------------------
My article new_123456 created
------------------------------
My article 123456_new created
My match against query:
MATCH(article.title) AGAINST ( "123456*" IN BOOLEAN MODE )
This query return only the first record and ignore the others since the term "123456" is attached to an underscore ( _ ), either before or after the term, the query will ignore the records.
What is the thing I did wrong and how to fix this problem?
There are many things that can mess up FULLTEXT:
Punctuation
"stop words"
min word "length"
Language
It is sometimes best to edit the data before storing it. In your case, replacing "_" with " " might be the 'right' solution. That could be done either in your application code as you insert strings, or by using MySQL's REPLACE() as the string is INSERTed.
I was trying MySQL secondary indexing referring to MySQL Documentation, and weird thing happened.
Firstly, I created a table with small modification per the example in the document
create table jemp(
c JSON,
g VARCHAR(20) GENERATED ALWAYS AS (c->"$.name"),
INDEX i (g)
)
Secondly, I inserted values per the example in the document
INSERT INTO jemp (c) VALUES
('{"id": "1", "name": "Fred"}'), ('{"id": "2", "name": "Wilma"}'),
('{"id": "3", "name": "Barney"}'), ('{"id": "4", "name": "Betty"}');
And then, I tried to perform a fuzzy search with "like" and "wildcard". This doesn't work because index doesn't support prefix %, but it can get result.
select c->"$.name" as name from jemp where g like "%F%"
Here is the weird thing, I removed the prefix %, and index did work. However, I didn't get any results. Per my poor understanding of MySQL, this should work.
select c->"$.name" as name from jemp where g like "F%"
I would be so much appreciate if anyone could help me with it.
For your query to work, you want a generated column that extracts the name as text rather than JSON. That is, use ->> instead of ->:
g VARCHAR(20) GENERATED ALWAYS AS (c ->> '$.name')
Then: the index may help for both following conditions:
where g like 'F%'
where g = 'F'
Whether MySQL decides to use it or not is another story; basically the databases assesses whether using the index will be faster than a full scan. If it believes that the condition will match on a large number of rows, it will probably choose to full scan.
Note that I consistently use single quotes for string literals; although MySQL tolerates otherwise, this is what the SQL standard specifies. In some other databases, double quotes stand for identifiers (this also is compliant with the standard).
I have sort of cache column in mysql table column. Lets call it cacheCol It is structured as json.
cacheCol example
{
"23": {
"variationOption": "23",
"productCode": "322992-015",
"price": "150",
"qnt": ""
},
"25": {
"variationOption": "25",
"productCode": "322992-015",
"price": "150",
"qnt": "0"
},
"26": {
"variationOption": "26",
"productCode": "322992-015",
"price": "150",
"qnt": "7"
}
}
I want to select myslq row if specific json part qnt is > 0. In this example part with key 26 is only one to match.
I have worked out regex to check those values:
https://www.regextester.com/?fam=109762
But when i run my query
SELECT * FROM "tbl" WHERE ("cacheCol" REGEXP ('(?<=\"26":\{)[^\{]*"qnt":"[1-9]\d*"(?=.*\})'))
Error shows up:
Got error 'repetition-operator operand invalid' from regexp
Here is sqlfiddle to play around: http://www.sqlfiddle.com/#!9/30335a/1
(Updated sqlfiddle to cover more variations in DB)
Is there any way to work around my regex to be compatible with mysql.
Found some info about mysql 5.7 having json data type maybe where is way to get my desired result avoiding regex?
For Schema like this :
CREATE TABLE IF NOT EXISTS `tbl` (
`id` int(6) unsigned NOT NULL,
`cacheCol` TEXT NOT NULL,
PRIMARY KEY (`id`)
) DEFAULT CHARSET=utf8;
INSERT INTO `tbl` (`id`, `cacheCol`) VALUES
('1', '{"23":{"variationOption":"23","productCode":"322992-015","price":"150","qnt":""}}'),
('2', '{"25":{"variationOption":"25","productCode":"322992-015","price":"150","qnt":"0"}}'),
('3', '{"26":{"variationOption":"26","productCode":"322992-015","price":"150","qnt":"7"}}');
Use the following query :
SELECT *
FROM `tbl`
where cacheCol REGEXP '.?"qnt":"[1-9]\d*"';
Since the field is always going to be JSON format it is enough to search for the substring alone.
You can play around here
Also it is good idea to explore JSON datatype in Mysql if you would have many updates and reads to this object, if only for simple querying, then you can treat it as string and work with REGEX itself.
Hope it helps
You can use JSON functions like JSON_SEARCH() to find a value that is an exact match, but not a inequality match.
In MySQL 8.0, you can use the JSON_TABLE() function if your JSON were structured as an array, but not with the JSON as you have structured it as an object.
Really, if you need to use inequality expressions to search for specific fields within your data, you should not use JSON at all. You should store the data in normal rows and columns.
The more I see people misusing JSON in complex ways in MySQL, the more I am convinced it was a bad idea for MySQL to implement a JSON data type.
My rule for JSON in MySQL is: references to a JSON column anywhere but the SELECT-list are a code smell. You should use normal columns instead of JSON.
MySQL REGEXP does not support lookaheads, but you can try to achieve the same logic using something like this:
SELECT * FROM "tbl" WHERE ("cacheCol" REGEXP ('(\{[\"\:,\-a-zA-Z0-9]+\"qnt\"\:\"[1-9][0-9]*\"\})'))
Hope it helps.
$qstring = "SELECT titulo as value, id FROM blogs WHERE titulo LIKE '%".$term."%' LIMIT 5";
$qstring = "SELECT titulo as value, id FROM blogs WHERE MATCH(titulo) AGAINST ('.$term.') LIMIT 5";
The first one will return results but not really related to the query
the second will return:
Can't find FULLTEXT index matching the column list
why?
check value in $term that should be greater then 3 in case of FULLTEXT index search otherwise it will return null
The minimum and maximum lengths of words to be indexed are defined by the ft_min_word_len and ft_max_word_len system variables. The default minimum value is four characters. If you change either value, you must rebuild your FULLTEXT indexes. For example, if you want three-character words to be searchable, you can set the ft_min_word_len variable by putting the following lines in an option file:
match() only works on field which have a FULLTEXT on them, exactly as the error message says. You'd have to do:
ALTER TABLE blogs ADD FULLTEXT INDEX tituolo_ft (titulo);
before you can use fulltext operations on the field.
As the error message implies, you can't use MATCH ... AGAINST unless there is a FULLTEXT index on the field you are comparing.
The LIKE statement should work though. I think the problem may be the double quotes in your pattern which are superfluous and will require corresponding quotes in the database value. Please show what database data you are trying to match.
In addition to the FULLTEXT index mentioned by others it looks like you are not properly quoting your text in the AGAINST clause. I think it should be:
AGAINST ('".$term."')
Or else, since you already have double quotes around your query just embed the variable:
AGAINST ('$term')
I have keywords like "some-or-other" where the hyphens matter in the search through my mysql database. I'm currently using the fulltext function.
Is there a way to escape the hyphen character?
I know that one option is to comment out #define HYPHEN_IS_DELIM in the myisam/ftdefs.h file, but unfortunately my host does not allow this. Is there another option out there?
Here's the code I have right now:
$search_input = $_GET['search_input'];
$keyword_safe = mysql_real_escape_string($search_input);
$keyword_safe_fix = "*'\"" . $keyword_safe . "\"'*";
$sql = "
SELECT *,
MATCH(coln1, coln2, coln3) AGAINST('$keyword_safe_fix') AS score
FROM table_name
WHERE MATCH(coln1, coln2, coln3) AGAINST('$keyword_safe_fix')
ORDER BY score DESC
";
From here http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
One solution to find a word with a dashes or hyphens in is to use FULL TEXT SEARCH IN BOOLEAN MODE, and to enclose the word with the hyphen / dash in double quotes.
Or from here http://bugs.mysql.com/bug.php?id=2095
There is another workaround. It was recently added to the manual:
"
Modify a character set file: This requires no recompilation. The true_word_char() macro
uses a “character type” table to distinguish letters and numbers from other
characters. . You can edit the contents in one of the character set XML
files to specify that '-' is a “letter.” Then use the given character set for your
FULLTEXT indexes.
"
Have not tried it on my own.
Edit: Here is some more additional info from here http://dev.mysql.com/doc/refman/5.0/en/fulltext-boolean.html
A phrase that is enclosed within double quote (“"”) characters matches only rows that contain the phrase literally, as it was typed. The full-text engine splits the phrase into words and performs a search in the FULLTEXT index for the words. Prior to MySQL 5.0.3, the engine then performed a substring search for the phrase in the records that were found, so the match must include nonword characters in the phrase. As of MySQL 5.0.3, nonword characters need not be matched exactly: Phrase searching requires only that matches contain exactly the same words as the phrase and in the same order. For example, "test phrase" matches "test, phrase" in MySQL 5.0.3, but not before.
If the phrase contains no words that are in the index, the result is empty. For example, if all words are either stopwords or shorter than the minimum length of indexed words, the result is empty.
Some people would suggest to use the following query:
SELECT id
FROM texts
WHERE MATCH(text) AGAINST('well-known' IN BOOLEAN MODE)
HAVING text LIKE '%well-known%';
But by that you need many variants depending on the used fulltext operators. Task: Realize a query like +well-known +(>35-hour <39-hour) working week*. Too complex!
And do not forget the default len of ft_min_word_len so a search for up-to-date returns only date in your results.
Trick
Because of that I prefer a trick so constructions with HAVING etc aren't needed at all:
Instead of adding the following text to your database table: "The Up-to-Date Sorcerer" is a well-known science fiction short story. copy the hyphen words without hypens to the end of the text inside a comment: "The Up-to-Date Sorcerer" is a well-known science fiction short story.<!-- UptoDate wellknown -->
If the users searches for up-to-date remove the hyphen in the sql query:
MATCH(text) AGAINST('uptodate ' IN BOOLEAN MODE)
By that you're user can find up-to-date as one word instead of getting all results that contain only date (because ft_min_word_len kills up and to).
Of course before you echo the texts you should remove the <!-- ... --> comments.
Advantages
the query is simpler
the user is able to use all fulltext operators as usual
the query is faster.
If a user searches for -well-known +science MySQL treats that as not include *well*, could include *known* and must include *science*. This isn't what the user expected. The trick solves that, too (as the sql query searches for -wellknown +science)
Maybe simpler to use the Binary operator.
SELECT *
FROM your_table_name
WHERE BINARY your_column = BINARY "Foo-Bar%AFK+LOL"
http://dev.mysql.com/doc/refman/5.0/en/cast-functions.html#operator_binary
The BINARY operator casts the string following it to a binary string. This is an easy way to force a column comparison to be done byte by byte rather than character by character. This causes the comparison to be case sensitive even if the column is not defined as BINARY or BLOB. BINARY also causes trailing spaces to be significant.
My preferred solution to this is to remove the hyphen from the search term and from the data being searched. I keep two columns in my full-text table - search and return. search contains sanitised data with various characters removed, and is what the users' search terms are compared to, after my code has sanitised those as well.
Then I display the return column.
It does mean I have two copies of the data in my database, but for me that trade-off is well worth it. My FT table is only ~500k rows, so it's not a big deal in my use case.