I was thinking what if I have a table with columns for meta_description (varchar 300), meta_tags (varchar 300), and meta_title (varchar 200)... can I "join" all this columns just into one column "extra_information" (longtext) and save here the same information but maybe in JSON format?
Is this convenient or not and why :)?
This fields are not very important for me, I will never make any query to search or sort the results trough this information. The metatags for example are only a comma separated text I don't need to do some kind of relation table on this.
What I want to know is this will save space on my database or will be working a little bit faster, or things like that... But if you tell me that have 5 columns instead of just one is the same for MySQL of course I will have the 5 columns...
Thanks a lot!
The answer boils down on: Does MySQL have to work with your data?
If all date is concatenated in one column, be it as JSON or comma-seperated or what not, it is nearly off limits for any MySQL operation. You can surely SELECT it, but it is very hard to search, group or sort by anything inside that column. So, it you are absolutly sure MySQL soes never have to see the data itself and will only return some column with data in it, go for it.
Benefits are that the table structure does not have to be changed because your data changes. and column structure is very clean
if you need to filter, sort, group or do whatever operation on it within a SQL query, leave it in seperate columns.
Related
I have a Mysql8 DB which contains JSON data. Unfortunately, the content is not always the same. To make it simple, the hierarchy is always the same, but sometimes part of the "tree" is missing or slightly different. For instance:
$.bilan.victimes.*.preview.TAGSAU (I use a star, since sometimes, it's '1', '2', etc... and sometimes it is only '$.bilan.victimes' (without further subkeys)
Now, I am using queries to lookup information in the JSON like:
SELECT
COUNT(fiche_id) AS USAGE_DSA,
JSON_VALUE(content, '$.bilan.victimes.*.preview.DSA') AS DSA
FROM bilan_json
WHERE STR_TO_DATE(JSON_VALUE(content, '$.bilan.victimes.*.preview.TAGSAU'),'%e/%c/%Y %H%#%i') >= '2021-01-01'
GROUP BY DSA;
This is working fine, but since there is a lot of records, and JSON could be very long, it takes an awful bunch of time to display the result. In this example, this is only key... I am supposed to retrieve multiples values from the JSON, sometimes in a single query.
I've read about virtual columns (https://stackoverflow.com/questions/68118107/how-to-create-a-virtual-column-to-index-json-column-in-mysql#:~:text=if%20table%20is%20already%20created%20and%20you%20want,%60jval%60%3B%20Dont%20forget%20to%20index%20the%20Generated%20Columns) and also improving performance for JSON object (https://blogit.create.pt/goncalomelo/2018/12/20/query-performance-for-json-objects-inside-sql-server/) but I can't really figure out if I should create a virtual column per key ? And, how can I create a virtual column with a transform ? In above case, I would create something like :
ALTER TABLE bilan_json
ADD COLUMN tagsau DATETIME
GENERATED ALWAYS AS STR_TO_DATE(JSON_VALUE(content, '$.bilan.victimes.*.preview.TAGSAU'),'%e/%c/%Y %H%#%i')
AFTER content;
What would be your advice ?
Simply put, If you expect to need a field in JSON for a WHERE or ORDER BY clause, that field should be in its own column.
3 approaches:
Redundantly store it in a column as you INSERT the rows.
Use a Virtual ("Generated") column (as you suggest).
Remove it from JSON as you put it in its own column.
Once it is in a column, it can be indexed. (It is unclear how useful an index would be for the SELECT you show.)
Did you try that ALTER? Did it work? We need SHOW CREATE TABLE in order to advise further.
I have a table in MySQL database with about 30 text fields and about 10 joined N-N tables.
My client wants one form input field to search through all the data.
Is there an easy way to do it?
My assumption is that if I do so many joins, the query is going to take ages.
So an idea I had is to create a column called "ALL". After each edit/add action I would dump all the other columns' date into this ALL column and do a search like this:
Select * From Table WHERE all like "%search"
Is it possible to do it like this? Anyone knows the right way to do it?
Thank you, Mike.
Yes, right
Commonly, there is another (distinct) column 'all' that is a tuple of all values of all columns and then you search through that column.
Another option is to add a different database just for a sake of fulltext
https://www.elastic.co/
https://www.algolia.com/
Apologies if this is redundant, and it probably is, I gave it a look but couldn't find a question here that fell in with what I wanted to know.
Basically we have a table with about ~50000 rows, and it's expected to grow much bigger than that. We need to be able to allow admin users to add in custom data to an item based on its category, and users can just pick which fields defined by the administrators they want to add info to.
Initially I had gone with an item_categories_fields table which pairs up entries from item_fields to item_categories, so admins can add custom fields and reuse them across categories for consistency. item_fields has a relationship to item_field_values which links values with fields, which is how we handled things in .NET. The project is using CAKEPHP though, and we're just learning as we go, so it can get a bit annoying at times.
I'm however thinking of maybe just adding an item_custom_fields table that is essentially the item_id and a text field that stores XMLish formatted data. This is just for the values of the custom fields.
No problems if I want to fetch the item by its id as the required data is stored in the items table, but what if I wanted to do a search based on a custom field? Would a
SELECT * FROM item_custom_fields
WHERE custom_data LIKE '%<material>Plastic</material>%'
(user input related issues aside) be practical if I wanted to fetch items made of plastic in this case? Like how slow would that be?
Thanks.
Edit: I was afraid of that as realistically this thing will be around 400k rows for that one table at launch, thanks guys.
Any LIKE query that starts with % will not use any indexes you have on the column, so the query will scan the whole table to find the result.
The response time for that depends highly on your machine and the size of the table, but it definitely won't be efficient in any shape or form.
Your previous/existing solution (if well indexed) should be quite a bit faster.
I have a huge data set. The structure looks something like this:
K_Field1, K_Field2, K_Field3, K_Field4, D_Field5, D_Field6, D_Field7, D_field8
The problem is that only the first 4 field (K_Field1,K_Field2,K_Field3,K_Field4) together identify a row uniquely. I created one table, using these fields as its fields.
Let's say I have 1 million rows in the table using that structure. If I import a new record, I have to decide if it's already in the database. If it is, then I have to update it, if not, then I need to insert a new row.
To do that, I need to put a multiple field index on the first 4 columns, which is - I'm afraid - not the best solution. Is there a better database structure to store and search in that data, or I have to live with the four-fielded index?
I'm using MySQL
There's nothing wrong with creating an index on the first 4 columns, in fact you should:
create unique index mytable_key on mytable(K_Field1,K_Field2,K_Field3,K_Field4);
because that is the reality of your situation.
It is also the "correct" solution.
please i have a modified_by column in mysql table with string like this "40,1280825613|40,1280825617". the "40" is the id of the user that made changes to the record and "1280825613" is the date string. The | separates different periods of change.
How do i change the entire column to a readable date without affecting the user id. either mysql or php solution is welcome. Thank you so much.
I'd recommend a PHP script. You'll need to make two columns modified_by to retain the user id and modified for the timestamp. If there are multiple modified_by for each record you'll probably want to make a separate table, i.e. revisions. This would be the best way to store the data relationship. I'd also recommend not storing formatted data. You should already see why that's not a good idea. The database is the raw data, use PHP to format it.
Once you have that setup, just:
Select the data from the old table.
Loop over the records
explode() the column on |
Loop over the array
explode() the element on ,
Insert into new columns/table
Forgive me, but I'd rather teach you how to fish.