Update: Question refined, I still need help!
I have the following table structure:
table reports:
ID | time | title | (extra columns)
1 | 1364762762 | xxx | ...
Multiple object tables that have the following structure
ID | objectID | time | title | (extra columns)
1 | 1 | 1222222222 | ... | ...
2 | 2 | 1333333333 | ... | ...
3 | 3 | 1444444444 | ... | ...
4 | 1 | 1555555555 | ... | ...
In the object tables, on an object update a new version with the same objectID is inserted, so that the old versions are still available. For example see the entries with objectID = 1
In the reports table, a report is inserted but never updated/edited.
What I want to be able to do is the following:
For each entry in my reports table, I want to be able to query the state of all objects, like they were, when the report was created.
For example lets look at the sample report above with ID 1. At the time it was created (see the time column), the current version of objectID 1 was the entry with ID 1 (entry ID 4 did not exist at that point).
ObjectID 2 also existed with it's current version with entry ID 2.
I am not sure how to achieve this.
I could use a query that selects the object versions by the time column:
SELECT *
FROM (
SELECT *
FROM objects
WHERE time < [reportTime]
ORDER BY time DESC
)
GROUP BY objectID
Lets not talk about the performance of this query, it is just to make clear what I want to do. My problem is the comparison of the time columns. I think this is no good way to make sure that I got the right object versions, because the system time may change "for any reason" and the time column would then have wrong data in it, which would lead to wrong results.
What would be another way to do so?
I thought about not using a time column for this, but instead a GLOBAL incremental value that I know the insertion order across the database tables.
If you are interting new versions of the object, and your problem is the time column(I assume you are using this column to sort which one is newer); I suggest you to use an auto-incremental ID column for the versions. Eventually, even if the time value is not reliable for you, the ID will be.Since it is always increasing. So higher ID, newer version.
Related
This question already has answers here:
How to store data with dynamic number of attributes in a database
(8 answers)
Storing JSON in database vs. having a new column for each key
(10 answers)
Closed 3 years ago.
I am working on a project, that has multiple processes and each process has different data items to process. Data items (For different processes) have different columns (but always the same columns for the same process).
At first, I assumed that it would be fine to have a table for all of the processes and then, whenever a new process is created, another table with the item data could be created as well, but it turns out, that there would be a new process way to often to create new tables all the time. Then I was looking into nested tables but found out that there is no concept of the nested tables in MySQL. (I've heard that this could be done with MariaDB. Has anyone worked with it?)
To make it a bit more clear here is the current concept (columns and values here are only approximate to make the concept more clear):
process_table:
ID | process_name | item_id | ...
---------------------------------
1 | some_process | 111 | ...
2 | other_process| 222 | ...
3 | third_process| 333 | ...
4 | third_process| 444 | ...
...
item_tables:
item_table_1:
ID | Column1 | Column2 | process_name | ...
--------------------------------------
111| val1 | val2 | some_process | ...
...
item_table_2:
ID | Column4 | Column5 | process_name | ...
--------------------------------------
333| val1 | val2 | third_process| ...
444| val3 | val4 | third_process| ...
...
So then for each new process, there would be new item_table and for each process, it needs to have different column names, and in item table, the specific item would be linked to 'item_id' column in the process table.
I think that the easiest solution (when creating new tables all the time is not an option) for this would be nested tables, where, in the process table, there could be another column, that would hold the item_table values and then those could have different columns based on the process itself.
So the big question is: Is there at least anything similar to nested tables or anything else in MySQL that would help me implement structure like this without creating new tables all the time, and if not, then maybe there are some tips or reviews about MariaDB? Maybe someone has already implemented nested tables with it (If that is possible at all)
One of the solutions would be to have one table for the 'item_table' and then have one column for all the different values for processes, that would be stored in JSON format for example, but this would make it a lot harder to read the table.
For example:
item_table:
ID | process_name | data
--------------------------------------
111| some_process | {values: {column1:val1,column2:val2,...}}
Do you use the values from the items-table for processing or something like that (do you run queries against them)?
This table/database structure looks.. ineffecient and unmaintainable imo.
This should all be done with just two tables. The processes table and the items table that contains the process_id (not the name) from the processes table.
If the column count for the items is always the same, just use "generic" names for the values like value_1, value_2 (or whatever suits best for the process) or a json/blob/varchar field with a JSON string for example. (depends if you need to run queries against this data)
id | process_id | data
EDIT:
Your edit and second solution should be the way to go.
"easy readability" has no priority above functionality and performance.
I have a situation where I have a table, for example:
| id | type |
------------------
| 0 | Complete |
| 1 | Zone |
Now, I always want my database to be populated with these values, but additionally users should be able to CRUD their own custom types beyond these. For example, a user might decide they want a "Partial Zone" type:
| id | type |
---------------------
| 0 | Complete |
| 1 | Zone |
| 2 | Partial Zone |
This is all fine. But I don't want anyone to be able to delete/modify the first and second rows.
This seems like it should be so simple, but is there a common strategy for handling this case that ensures that these rows go unaffected? Should I put a lock column on the table and only lock these two values when I initially populate the database on application setup? Is there something much more obvious and elegant that I am missing?
Unless I'm missing something, you should be able to just add a third column to your table for the user ID/owner of the record. For the Complete and Zone records, the owner could be e.g. user 0, which would correspond to an admin. In your deletion logic, just check the ID column and do not allow admin records to be deleted by anyone from the application.
If this won't work, you could also consider having two tables, one for system records which cannot be deleted, and another one for user created records. You would have to possibly always take a union of the two tables when you query.
I have two related tables, results and userID.
results looks like this:
+----+--------+--------+
| ID | userID | result |
+----+--------+--------+
| 1 | abc | 124 |
| 2 | abc | 792 |
| 3 | def | 534 |
+----+--------+--------+
userID looks like this:
+----+--------+---------+
| id | userID | name |
+----+--------+---------+
| 1 | abc | Angela |
| 2 | def | Gerard |
| 3 | zxy | Enrico |
+----+--------+---------+
In results, the userID field is a lookup field; it stores userID.id but the combo box has userID.userID as its choices.
When I try to enter data into results by setting the userID combo box and entering a value for result, I get this error message:
You cannot add or change a record because a related record
is required in table `userID`.
This is strange, because I'm specifically selecting a value that's provided in the userID combo box.
Oddly, there are about 100 rows of data already in results with the same value for userID.
I thought this might be a database corruption issue, so i created a blank database and imported all the tables into it. But I still got the same error. What's going on here?
Both tables include a text field named LanID. You are using that field in this relationship, which enforces referential integrity:
The problem you're facing is due to the Lookup field properties. This is the Row Source:
SELECT [LanID].ID, [LanID].LanID FROM LanID ORDER BY [LanID];
But the value which gets stored (the Bound Column property) is the first column from that SELECT statement, which is the Long Integer [LanID].ID. So that number will not satisfy the relationship, which requires results.LanID = [LanID].LanID.
You must change the relationship or change the Lookup properties so both reference the same field value.
But if it were me, I would just eliminate the Lookup on the grounds that simple operations (such as this) become unnecessarily confusing when Lookup fields are involved. Make results.LanID a plain numeric or text field. If you want some kind of user-friendly drop-down for data entry, build a form with a combo or list box.
For additional arguments against Lookup fields, see The Evils of Lookup Fields in Tables.
If you are using a parameter query, make sure you have them in the same order as the table you are modifying and the query you have created. You might have one parameter inserting the conflicting data. Parameters are used in the order they are created...not the name of the parameter. I had the same problem and all I had to do was switch the order they were in so they matched the query. This is an old thread, so I hope this helps someone who is just now having this problem.
I have 6 tables. These are simplified for this example.
user_items
ID | user_id | item_name | version
-------------------------------------
1 | 123 | test | 1
data
ID | name | version | info
----------------------------
1 | test | 1 | info
data_emails
ID | name | version | email_id
------------------------
1 | test | 1 | 1
2 | test | 1 | 2
emails
ID | email
-------------------
1 | email#address.com
2 | second#email.com
data_ips
ID | name | version | ip_id
----------------------------
1 | test | 1 | 1
2 | test | 1 | 2
ips
ID | ip
--------
1 | 1.2.3.4
2 | 2.3.4.5
What I am looking to achieve is the following.
The user (123) has the item with name 'test'. This is the basic information we need for a given entry.
There is data in our 'data' table and the current version is 1 as such the version in our user_items table is also 1. The two tables are linked together by the name and version. The setup is like this as a user could have an item for which we dont have data, likewise there could be an item for which we have data but no user owns..
For each item there are also 0 or more emails and ips associated. These can be the same for many items so rather than duplicate the actual email varchar over and over we have the data_emails and data_ips tables which link to the emails and ips table respectively based on the email_id/ip_id and the respective ID columns.
The emails and ips are associated with the data version again through the item name and version number.
My first query is is this a good/well optimized database setup?
My next query and my main question is joining this complex data structure.
What i had was:
PHP
- get all the user items
- loop through them and get the most recent data entry (if any)
- if there is one get the respective emails
- get the respective ips
Does that count as 3 queries or essentially infinite depending on the number of user items?
I was made to believe that the above was inefficient and as such I wanted to condense my setup into using one query to get the same data.
I have achieved that with the following code
SELECT user_items.name,GROUP_CONCAT( emails.email SEPARATOR ',' ) as emails, x.ip
FROM user_items
JOIN data AS data ON (data.name = user_items.name AND data.version = user_items.version)
LEFT JOIN data_emails AS data_emails ON (data_emails.name = user_items.name AND data_emails.version = user_items.version)
LEFT JOIN emails AS emails ON (data_emails.email_id = emails.ID)
LEFT JOIN
(SELECT name,version,GROUP_CONCAT( the_ips.ip SEPARATOR ',' ) as ip FROM data_ips
LEFT JOIN ips as the_ips ON data_ips.ip_id = the_ips.ID )
x ON (x.name = data.name AND x.version = user_items.version)
I have done loads of reading to get to this point and worked tirelessly to get here.
This works as I require - this question seeks to clarify what are the benefits of using this instead?
I have had to use a subquery (I believe?) to get the ips as previously it was multiplying results (I believe based on the complex joins). How this subquery works I suppose is my main confusion.
Summary of questions.
-Is my database setup well setup for my usage? Any improvements would be appreciated. And any useful resources to help me expand my knowledge would be great.
-How does the subquery in my sql actually work - what is the query doing?
-Am i correct to keep using left joins - I want to return the user item, and null values if applicable to the right.
-Am I essentially replacing a potentially infinite number of queries with 2? Does this make a REAL difference? Can the above be improved?
-Given that when i update a version of an item in my data table i know have to update the version in the user_items table, I now have a few more update queries to do. Is the tradeoff off of this setup in practice worthwhile?
Thanks to anyone who contributes to helping me get a better grasp of this !!
Given your data layout, and your objective, the query is correct. If you've only got a small amount of data it shouldn't be a performance problem - that will change quickly as the amount of data grows. However when you ave a large amount of data there are very few circumstances where you should ever see all your data in one go, implying that the results will be filtered in some way. Exactly how they are filtered has a huge impact on the structure of the query.
How does the subquery in my sql actually work
Currently it doesn't work properly - there is no GROUP BY
Is the tradeoff off of this setup in practice worthwhile?
No - it implies that your schema is too normalized.
Lookup table - unique row identity
The other lookup tables just do not make sense as from what I have seen giving a row an ID then putting that id in another table which also has a id then adding these id's to some more tables which may reference them and still creating a lookup tables with more id's (this is how all the examples I can find seem) What I have done is this :
product_item - table
------------------------------------------
id | title | supplier | price
1 | title11 | suuplier1 | price1
etc.
it then goes on to include more items (sure you get it)
product_feature - table
--------------------------
id | title | iskeyfeature
1 | feature1 | true
feature_desc - table
-----------------------------
id | title | desc
1 | desc1 | text description
product_lookup - table
item_id | feature_id | feature_desc
1 | 1 | 1
1 | 2 | 2
1 | 3 | 3
1 |64 | 15
(as these only need to be referenced in the lookup the id's can be multiples per item or multiple items per feature)
What I want to do without adding item_id to every feature row or description row is retrieve only the columns from the multiple tables where their id is referenced in the same row of the lookup table. I want to know if it is possible to select all the referenced columns from the lookup row if I only know the item_id eg. Item_id = 1 return all rows where item_id = 1 with the columns referenced in the same row. Every item can have multiple features and also every feature could be attached to multiple items , this will not matter if I can just get the pattern right in how to construct this query from a single known value.
Any assistance or just some direction will be greatly appreciated. I'm using phpmyadmin, and sure this will be easier with some php voodoo I am learning mysql from tutorials ect and would like to know how to do it with sql directly.
Having a NULL value in a column is not the major concern that would lead to this design - it's the problem with adding new attribute columns in the future, at which MySQL is disgracefully bad.
If you want to make a query that returns everything about an item in one row, you need to LEFT OUTER JOIN back to the product_lookup table for each feature_id. This is about every 10th mysql question on Stack Overflow, so you should be able to find tons of examples.