MySql 5.7 json_extract by key - mysql

I have a table and it looks like below:
Table data
id params
1 {"company1X":{"price":"1124.55"},"company2X":{"price":"1,124.55"},"company3X":{"price":""},"company4X":{"price":""},"company5X":{"price":"1528.0"}}
I don't know the name of "company" to use in my request.
How can I fetch my data ordered by price?
Thanks!
P.S I have tried select json_extract(params, '$[*].price') from data but it doesn't work (return nulls).

$[*] gets all elements of a JSON array, not an object. This is an object, so you get NULL.
$.* will get you all elements in a JSON object, so $.*.price gets you a JSON array of all prices.
mysql> select json_extract(params, '$.*.price') from foo;
+-------------------------------------------+
| json_extract(params, '$.*.price') |
+-------------------------------------------+
| ["1124.55", "1,124.55", "", "", "1528.0"] |
+-------------------------------------------+
Now there's a problem. As far as SQL is concerned, this is a single row. It can't be sorted with a normal order by, that works on rows.
MySQL has no function for sorting JSON... so you're stuck. You can return the JSON array and let whatever is receiving the data do the sorting. You might be able to write a stored procedure to sort the array... but that's a lot of work to support a bad table design. Instead, change the table.
The real problem is this is a bad use of a JSON column.
JSON columns defeat most of the point of a SQL database (less so in PostgreSQL which has much better JSON support). SQL databases work with rows and columns, but JSON shoves what would be multiple rows and columns into a single cell.
For this reason, JSON columns should be used sparingly; typically when you're not sure what sort of data you'll be needing to store. Important information like "price" that's going to be searched and sorted should be done as normal columns.
You'll want to change your table to be a normal SQL table with columns for the company name and price. Then you can use normal SQL features like order by and performance will benefit from indexing. There isn't enough information in your question to suggest what that table might look like.

Related

How to manage JSON query performance in MySQL DB

I have a Mysql8 DB which contains JSON data. Unfortunately, the content is not always the same. To make it simple, the hierarchy is always the same, but sometimes part of the "tree" is missing or slightly different. For instance:
$.bilan.victimes.*.preview.TAGSAU (I use a star, since sometimes, it's '1', '2', etc... and sometimes it is only '$.bilan.victimes' (without further subkeys)
Now, I am using queries to lookup information in the JSON like:
SELECT
COUNT(fiche_id) AS USAGE_DSA,
JSON_VALUE(content, '$.bilan.victimes.*.preview.DSA') AS DSA
FROM bilan_json
WHERE STR_TO_DATE(JSON_VALUE(content, '$.bilan.victimes.*.preview.TAGSAU'),'%e/%c/%Y %H%#%i') >= '2021-01-01'
GROUP BY DSA;
This is working fine, but since there is a lot of records, and JSON could be very long, it takes an awful bunch of time to display the result. In this example, this is only key... I am supposed to retrieve multiples values from the JSON, sometimes in a single query.
I've read about virtual columns (https://stackoverflow.com/questions/68118107/how-to-create-a-virtual-column-to-index-json-column-in-mysql#:~:text=if%20table%20is%20already%20created%20and%20you%20want,%60jval%60%3B%20Dont%20forget%20to%20index%20the%20Generated%20Columns) and also improving performance for JSON object (https://blogit.create.pt/goncalomelo/2018/12/20/query-performance-for-json-objects-inside-sql-server/) but I can't really figure out if I should create a virtual column per key ? And, how can I create a virtual column with a transform ? In above case, I would create something like :
ALTER TABLE bilan_json
ADD COLUMN tagsau DATETIME
GENERATED ALWAYS AS STR_TO_DATE(JSON_VALUE(content, '$.bilan.victimes.*.preview.TAGSAU'),'%e/%c/%Y %H%#%i')
AFTER content;
What would be your advice ?
Simply put, If you expect to need a field in JSON for a WHERE or ORDER BY clause, that field should be in its own column.
3 approaches:
Redundantly store it in a column as you INSERT the rows.
Use a Virtual ("Generated") column (as you suggest).
Remove it from JSON as you put it in its own column.
Once it is in a column, it can be indexed. (It is unclear how useful an index would be for the SELECT you show.)
Did you try that ALTER? Did it work? We need SHOW CREATE TABLE in order to advise further.

Merging nested tables in linq to sql

I have a linq query that gets data from an OData Reporting service
So far so good, but when I return my data like this :
select new {TimesheetActual , TimesheetLine,Timesheet, TimesheetProject,TimesheetTask, subTv, TimesheetResource, subRes, pLeft}
It returns as a collection of nested collections.
For my service I need one big table with every column from every record.
I know this is possible by explicitly naming every column in the select statement like this:
select new { TimesheetActual.Column1, TimesheetActual.Column2, .., TimesheetLine.Column1,.., TimesheetProject.Column1,..}
But due to the massive amount of columns I'm a little reluctant to do it this way.
So my question, is there any way to either merge the collections or another way to get the same result without having to specify 100+ columns?

One column or separate columns for extra data - mysql

I was thinking what if I have a table with columns for meta_description (varchar 300), meta_tags (varchar 300), and meta_title (varchar 200)... can I "join" all this columns just into one column "extra_information" (longtext) and save here the same information but maybe in JSON format?
Is this convenient or not and why :)?
This fields are not very important for me, I will never make any query to search or sort the results trough this information. The metatags for example are only a comma separated text I don't need to do some kind of relation table on this.
What I want to know is this will save space on my database or will be working a little bit faster, or things like that... But if you tell me that have 5 columns instead of just one is the same for MySQL of course I will have the 5 columns...
Thanks a lot!
The answer boils down on: Does MySQL have to work with your data?
If all date is concatenated in one column, be it as JSON or comma-seperated or what not, it is nearly off limits for any MySQL operation. You can surely SELECT it, but it is very hard to search, group or sort by anything inside that column. So, it you are absolutly sure MySQL soes never have to see the data itself and will only return some column with data in it, go for it.
Benefits are that the table structure does not have to be changed because your data changes. and column structure is very clean
if you need to filter, sort, group or do whatever operation on it within a SQL query, leave it in seperate columns.

Select decoded JSON data from joined MySQL tables

Could you write me please how to make selection from MySQL database if I have two tables with JSON data. One of them has following structure:
Table Trees
(id, name, value) - three columns
which includes following data
1, trees, [{"name":"Oaktree","value":1,"target":null},{"name":"Appletree","value":2,"target":null},{"name":"Plumtree","value":3,"target":null}]
2, length, [{"name":"10m","value":1,"target":null},{"name":"15m","value":2,"target":null},{"name":"20m","value":3,"target":null}]
3, age, [{"name":"5y","value":1,"target":null},{"name":"10y","value":2,"target":null},{"name":"20y","value":3,"target":null}]
The second table has the following structure:
Table SelectedTrees
(properties) - only one column
which includes the following data
[{"id":"1","value":["1","3"]},{"id":"2","value":["1", "2", "3"]},{"id":"3","value":["2"]}]
it means selected data from Trees tables. id in properties column from selectedTrees coresponds to id column from Trees table. I would like to select from database real (json_decoded) values like:
Trees = Oaktree, Plumtree
Length = 10m, 15m, 20m
Age = 10y
How could I make this?
Thanks in advance.
Jan
In a nutshell, this is not possible. Relational databases are built for quickly comparing constant values that they can index. JSON is just a string to MySQL, and any kind of partial string matching triggers a so-called table scan, which is essentially going to become freaking slow when you get serious amounts of data.
You COULD get it to work like this:
SELECT * FROM Trees
JOIN SelectedTrees
ON properties LIKE CONCAT('"id":"', Trees.id, '"')
This is however just a hack that you should never want to use in any production system, and I advise against using it in a test system. Instead refactor your database so there's never going to be any JSON in there that you are going to match on in your queries. It's fine to store secondary data as JSON, just make sure the IDs and names are extracted before insertion, and then insert in separate columns in the database tables so the DB engine can do its relational magic.

Storing array with values in database

I have the following data which I want to save in my DB (this is used for sending text messages via a 3rd party API)
text_id, text_message, text_time, (array)text_contacts
text_contacts contains a normal array with all the contact_id's
How should I properly store the data in a MySQL database?
I was thinking myself either on 2 ways:
Make the array with contact_id's in a json_encoded (no need for serializing since it's not multi-dimensional) string, and store it in a text field in the DB
Make a second table with the text_id and all contact_id's on a new row..
note: The data stored in the text_contacts array does not need to be changed at any time.
note2: The data is used as individual contact_id to get the phone number from the contact, and check whether the text message has actually been sent.. (with a combination of text_id, and phonenumber)
What is more efficiƫnt, and why?
This is completely dependent upon your expected usage characteristics. If you will have a near-term need to query based upon the contact_ids, then store them independently as in your second solution. If you're storing them for archival purposes, and don't expect them to be used dynamically, you're as well off saving the time and storing them in a JSON string. It's all about the usage.
IMO, go with the second table, mapping text-ids to contact-ids. Will be easier to manipulate than storing all the contacts in one field
This topic will bring in quite a few opinions, but my belief: second table, by all means.
If you ever have a case where you actually need to search by that data, it will not require you to parse it before using it.
It is a heck of a lot easier to debug (for the same reason)
json_encode and json_decode (or equivalent) take far more time than a join does.
Lazy loading is easier, even if not necessary in most cases.
Others will find it more readable and, with a good schema definition, easier to conceptualize and maintain.
Almost all implementations would use one table for storing each text_contacts, and then a second table would use a foreign key to reference the text_contacts table. So, if say you had a table text_contacts that looked like this:
contact_id | name
1 | someone
2 | someone_else
And a text message table that looked like this:
text_id | text_message | text_time | text_contact
1 | "Hey" | 12:48 | 1
2 | "Hey" | 12:48 | 2
Each contact that has been sent a message would have a new entry in the text message table, with the last column referencing the contact_id field of the text_contacts table. This way makes it much easier to retrieve messages by contact, because you can say "select * from text_messages where text_contact = 1" instead of searching through each of the arrays on the single table to find the messages sent by a specific user.