Transport table
id name
1 T1
2 T2
Pallets table
id name
1 P1
2 P2
Transport Pallet Capacity table
id transport_id pallet_id capacity
1 1 1 10
2 1 2 null
3 2 1 20
4 2 2 24
How to generate table like this:
id transport_id pallet_id_1_capacity pallet_id_2_capacity
1 1 10 null
2 2 20 24
Problem: pallets and transports can be added, so, neither quantity is known in advance.
For example, manager adds another pallet type and 'pallet_id_3_capacity' column should be generated (and can show null if no capacity data is yet available).
Another manager can fill 'transport pallet capacity' table later when notified.
Is there a way to build sql in mysql that will care about the above: specifically - dynamic number of pallets?
The SQL select-list must be fixed at the time you write the query. You can't make SQL that auto-expands its columns based on the data it finds.
But your request is common, it's called a pivot-table or a crosstab table.
The only solution is to do this in multiple steps:
Query to discover the distinct pallet ids.
Use application code to build a dynamic SQL query with as many columns as distinct pallet id values found in the first query.
Run the resulting dynamic SQL query.
This is true for all SQL databases, not just MySQL.
See MySQL pivot row into dynamic number of columns for a highly-voted solution for producing a pivot-table query in MySQL.
I am not voting your question as a duplicate of that question, because your query also involves transport_id, which will make the query solution a bit different. But reading about other pivot-table solutions should get you started.
Related
I have two tables, one user table and an items table. In the user table, there is the field "items". The "items" table only consists of a unique id and an item_name.
Now each user can have multiple items. I wanted to avoid creating a third table that would connect the items with the user but rather have a field in the user_table that stores the item ids connected to the user in a "csv" field.
So any given user would have a field "items" that could have a value like "32,3,98,56".
It maybe is worth mentioning that the maximum number of items per user is rather limited (<5).
The question: Is this approach generally a bad idea compared to having a third table that contains user->item pairs?
Wouldn't a third table create quite an overhead when you want to find all items of a user (I would have to iterate through all elements returned by MySQL individually).
You don't want to store the value in the comma separated form.
Consider the case when you decide to join this column with some other table.
Consider you have,
x items
1 1, 2, 3
1 1, 4
2 1
and you want to find distinct values for each x i.e.:
x items
1 1, 2, 3, 4
2 1
or may be want to check if it has 3 in it
or may be want to convert them into separate rows:
x items
1 1
1 2
1 3
1 1
1 4
2 1
It will be a HUGE PAIN.
Use atleast normalization 1st principle - have separate row for each value.
Now, say originally you had this as you table:
x item
1 1
1 2
1 3
1 1
1 4
2 1
You can easily convert it into csv values:
select x, group_concat(item order by item) items
from t
group by x
If you want to search if x = 1 has item 3. Easy.
select * from t where x = 1 and item = 3
which in earlier case would use horrible find_in_set:
select * from t where x = 1 and find_in_set(3, items);
If you think you can use like with CSV values to search, then first like %x% can't use indexes. Second, it will produce wrong results.
Say you want check if item ab is present and you do %ab% it will return rows with abc abcd abcde .... .
If you have many users and items, then I'd suggest create separate table users with an PK userid, another items with PK itemid and lastly a mapping table user_item having userid, itemid columns.
If you know you'll just need to store and retrieve these values and not do any operation on it such as join, search, distinct, conversion to separate rows etc. etc. - may be just may be, you can (I still wouldn't).
Storing complex data directly in a relational database is a nonstandard use of a relational database. Normally they are designed for normalized data.
There are extensions which vary according to the brand of software which may help. Or you can normalize your CSV file into properly designed table(s). It depends on lots of things. Talk to your enterprise data architect in this case.
Whether it's a bad idea depends on your business needs. I can't assess your business needs from way out here on the internet. Talk to your product manager in this case.
I would like a table or query in mysql of all permutations of two separate datasets, but with rules.
I have a table of jobs, and a table of drivers.
I'd like to produce table or query of all combinations of jobs to drivers. Each job must have a driver, but each driver doesn't necessarily need a job. Like this:
In this example I have 4 jobs and 3 drivers.
Job1 | Job2 | Job3 | Job4
1 1 1 1
1 1 1 2
1 1 1 3
1 1 2 1
1 1 3 1
This, I can't do, so if someone could help me that would be awesome. I believe that the number of rows in this example would be 4 to the power 3 (jobs to the power of drivers) which is 64 rows.
But the second part of this is what I call the "rules". Each job will have defined drivers that can do the job.
For example Job 1 can only be done by driver 1 or 3.
Job 2 can only be done by driver 1.
etc
I was thinking that if I did a create table, then running delete queries, but I am really at a loss. I would like to just create the query using the rules to start with in an attempt to speed it up.
This will eventually help me to make a plan for each job by showing all the ways that these jobs can be assigned.
Sorry for being vague but I'm hoping the community can help me out here.
Edit:
I think my maths may have been wrong. According to this: combination calculator where I input 3 to choose from (drivers) and 4 numbers chosen (jobs) order is important and repetition is allowed (not sure what that is) then it produces 81.
Although way unclear and smells like homework, her my two cents:
Setup one table with the jobs jobs:
job_id|job_name
Next, set up a table with drivers:
driver_id|driver_name
Now we need to maps:
First the "rules", drivers which are capable of jobs, job_capabilities:
driver_id|job_id
this will contain one row for each assignment job => driver, the combination (job_id, driver_id) should be unique - one driver is only one times capable of a particular job.
Second map contains the assignments itself, assigned_jobs:
driver_id|job_id
actually that has the same layout, but for a given time period (which is missing here) one driver actually can only work on one job, so driver_id and a date-time should be unique. Skipped for clearness now.
Now we can construct an SQL like
SELECT job_capabilities.driver_id, job.job_id from jobs, job_capabilities
where job.job_id = job_capabilities.job_id
AND job.job_id = 42;
We could just use that to insert into assigned_jobs with
INSERT INTO assigned_jobs .... (Select from above) ...
probably enhanced by a ON DUPLACET KEY UPDATE ... clause.
To now get the assigned jobs, we can alter that statement a bit:
SELECT assigned_jobs.driver_id, drivers.driver_name, job.job_id, job.job_name
from drivers, jobs, assigned_jobs
where job.job_id = assigned_jobs.job_id
AND drivers.driver_id = assigned_jobs.driver_id
AND job.job_id = 42;
This is not tested and probably not valid SQL, but a first approach I would use.
I've done some research and I know about building views but I feel like what I am trying to accomplish may not be possible without creating a hard coded table of information..Below is what I am trying to accomplish
Table 1
ID
1
2
3
4
5
Table 2
ID DATE
1 4/23/2014
1 5/10/2014
2 4/15/2014
5 12/13/2013
3 3/15/2014
What I am wanting to do is compose a query that will start with the data in table one as it contains all possible ID's, I want to join this data with the data from table 2 based on the ID field, but the key is I will be passing in a date argument that will be filtering the data based on month and year as well. What I need to happen is if an ID in table one does not have a record in Table 2 for a given month I need a null record to be created. The issue I am running into is not having a single null record no matter what for each ID in table one being put into the view which below I explain why I need this. Here is the view I am getting with just typical left join:
Table 3 (View or just a general query joining the above tables)
ID DATE
1 4/23/2014
1 5/10/2014
2 4/15/2014
5 12/13/2013
3 3/15/2014
4 null
Now its easy to see if I pulled for this data for the month/year of 4/2014 or null I would get
ID DATE
1 4/23/2014
2 4/15/2014
4 null
What I am looking to get (which I realize is impossible from the data in this view) is:
ID DATE
1 4/23/2014
2 4/15/2014
4 null
3 null
5 null
So is there a way to create these rows in a view dynamically so that when I pass in my date argument I will get the desired results? One thing I just thought of right this second would be to create a view with just the ID's from table 1 and a null date field and possibly union the two views together? Or could a Union be used in building the initial view to get the desired results? Once I have the data in the final view (or pulling it with the correct query in general) I will be able to accomplish my final task of building reports based on the data. Any help would be greatly appreciated!
I actually have a table with 30 columns. In one day this table can get around 3000 new records!
The columns datas look like :
IMG Name Phone etc..
http://www.site.com/images/image.jpg John Smith 123456789 etc..
http://www.site.com/images/image.jpg Smith John 987654321 etc..
I'm looking a way to optimize the size of the table but also the response time of the sql queries. I was thinking of doing something like :
Column1
http://www.site.com/images/image.jpg|John Smith|123456789|etc..
And then via php i would store each value into an array..
Would it be faster ?
Edit
So to take an example of the structure, let's say i have two tables :
package
package_content
Here is the structure of the table package :
id | user_id | package_name | date
Here is the structure of the table package_content :
id | package_id | content_name | content_description | content_price | content_color | etc.. > 30columns
The thing is for each package i can get up to 16rows of content. For example :
id | user_id | package_name | date
260 11 Package 260 2013-7-30 10:05:00
id | package_id | content_name | content_description | content_price | content_color | etc.. > 30columns
1 260 Content 1 Content 1 desc 58 white etc..
2 260 Content 2 Content 2 desc 75 black etc..
3 260 Content 3 Content 3 desc 32 blue etc..
etc...
Then with php i make like that
select * from package
while not EOF {
show package name, date etc..
select * from package_content where package_content.package_id = package.id and package.id = package_id
while not EOF{
show package_content name, desc, price, color etc...
}
}
Would it be faster? Definitely not. If you needed to search by Name or Phone or etc... you'd have to pull those values out of Column1 every time. You'd never be able to optimize those queries, ever.
If you want to make the table smaller it's best to look at splitting some columns off into another table. If you'd like to pursue that option, post the entire structure. But note that the number of columns doesn't affect speed that much. I mean it can, but it's way down on the list of things that will slow you down.
Finally, 3,000 rows per day is about 1 million rows per year. If the database is tolerably well designed, MySQL can handle this easily.
Addendum: partial table structures plus sample query and pseudocode added to question.
The pseudocode shows the package table being queried all at once, then matching package_content rows being queried one at a time. This is a very slow way to go about things; better to use a JOIN:
SELECT
package.id,
user_id,
package_name,
date,
package_content.*
FROM package
INNER JOIN package_content on package.id = package_content.id
WHERE whatever
ORDER BY whatever
That will speed things up right away.
If you're displaying on a web page, be sure to limit results with a WHERE clause - nobody will want to see 1,000 or 3,000 or 1,000,000 packages on a single web page :)
Finally, as I mentioned before, the number of columns isn't a huge worry for query optimization, but...
Having a really wide result row means more data has to go across the wire from MySQL to PHP, and
It isn't likely you'll be able to display 30+ columns of information on a web page without it looking terrible, especially if you're reading lots of rows.
With that in mind, you'll be better of picking specific package_content columns in your query instead of picking them all with a SELECT *.
Don't combine any columns, this is no use and might even be slower in the end.
You should use indexes on a column where you query at. I do have a website with about 30 columns where atm are around 600.000 results. If you use EXPLAIN before a query, you should see if it uses any indexes. If you got a JOIN with 2 values and a WHERE at the same table. You should make a combined index with the 3 columns, in order from JOIN -> WHERE. If you join on the same table, you should see this as a seperate index.
For example:
SELECT p.name, p.id, c.name, c2.name
FROM product p
JOIN category c ON p.cat_id=c.id
JOIN category c2 ON c.parent_id=c2.id AND name='Niels'
WHERE p.filterX='blaat'
You should have an combined index at category
parent_id,name
AND
id (probably the AI)
A index on product
cat_id
filterX
With this easy solution you can optimize queries from NOT DOABLE to 0.10 seconds, or even faster.
If you use MySQL 5.6 you should step over to INNODB because MySQL is better with optimizing JOINS and sub queries. Also MySQL will try to run them into MEMORY which will make it a lot faster aswel. Please keep in mind that backupping INNODB tables might need some extra attention.
You might also think about making MEMORY tables for super fast querieing (you do still need indexes).
You can also optimize by making integers size 4 (4 bytes, not 11 characters). And not always using VARCHAR 255.
I need to create a join query for hierarchical data across 2 tables. These tables can have unlimited amounts of data and their structures are as follows:
group_id group_name group_order
1 group 1 2
2 group 2 1
field_id field_name parent_group field_order
1 field 1 1 1
2 field 2 2 2
3 field 3 2 1
I am currently able to get the correct format of data using 2 select queries with the second query inside a loop created from the results of the first query on the groups table.
The structure of the data I require from the result is as follows:
-group 2
- field 3
- field 2
- group 1
- field 1
Is it possible to get these results from one mysql query? I have read through the mysql document on hierarchical data by I am confused about how to incorporate the join.
Thanks for looking
You shouldn't need to think about it in terms of hierarchical data, you should just be able to select your fields and join on your group information. Try something like:
SELECT *
FROM Fields AS F
INNER JOIN Groups AS G
ON G.group_id = F.parent_group
ORDER BY group_order, field_order
Then you will get each fields as a row with the applicable group, also in the correct group order. Your loop should be able to handle the display you need.
one method
something that may convince you change your db schema