SQL 'merging' columns from result sets into one result set - mysql

I've been pulling my hair about how to write a particular view within the constraints of MySQL.
The following tables and columns are of importance:
CREATE TABLE `invoices` (
`id` int(10) unsigned NOT NULL AUTO INCREMENT,
PRIMARY KEY (`id`)
)
-- Joins payments to invoices. The sum of all `invoice_currency_value`s is the balance paid towards an invoice.
CREATE TABLE `financial_transactions_invoices` (
`id` int(10) unsigned NOT NULL AUTO INCREMENT,
`invoice` int(10) unsigned NOT NULL,
`invoice_currency_value` decimal(8,2) unsigned NOT NULL,
PRIMARY KEY (`id`)
)
-- Lists items (services) available to purchase.
CREATE TABLE `items` (
`id` int(10) unsigned NOT NULL AUTO INCREMENT,
`value` decimal(8,2) unsigned NOT NULL
PRIMARY KEY (`id`)
)
-- Each instance represents that the `item` has been purchased.
CREATE TABLE `item_instances` (
`id` int(10) unsigned NOT NULL AUTO INCREMENT,
`invoice` int(10) unsigned NOT NULL,
`item` int(10) unsigned NOT NULL,
`invoice_currency_rate` decimal(11,5) unsigned NOT NULL,
PRIMARY KEY (`id`)
)
-- Any number of tax instances can exist for an item instance and indicate this tax has been applied to the associated item instance.
CREATE TABLE `tax_instances` (
`id` int(10) unsigned NOT NULL AUTO INCREMENT,
`item_instance` int(10) unsigned NOT NULL,
`value` decimal(8,2) unsigned NOT NULL,
PRIMARY KEY (`id`)
)
Now, I need a view that lists for each row,
the invoice number
the total value of the invoice
the total tax on the invoice
and the total value of payments made towards the invoice
However, I can't figure out how to get these three separate queries into the same result set of one row per invoice, e.g.
inv_no total_value total_tax payments
1 150 5 120
2 120 10 20
3 10 0 10
4 1000 150 1150
I have written the following query which produces the desired result, but due to the 'no subquery' rule in MySQL views, it is not acceptable.
SELECT `invoice_id`, SUM(`total_value`) AS `total_value`, SUM(`total_tax`) AS `total_tax`,
SUM(`paid_balance`) AS `paid_balance`
FROM
(SELECT `invoices`.`id` AS `invoice_id`, SUM(`items`.`value` * `item_instances`.`invoice_currency_rate`) AS `total_value`,
NULL AS `total_tax`, NULL AS `paid_balance`
FROM `items`
JOIN `item_instances` ON `items`.`id` = `item_instances`.`item`
JOIN `invoices` ON `item_instances`.`invoice` = `invoices`.`id`
GROUP BY `invoices`.`id`
UNION
SELECT `invoices`.`id`, NULL, SUM(`tax_instances`.`value`), NULL
FROM `tax_instances`
JOIN `item_instances` ON `tax_instances`.`item_instance` = `item_instances`.`id`
JOIN `invoices` ON `item_instances`.`invoice` = `invoices`.`id`
GROUP BY `invoices`.`id`
UNION
SELECT `invoices`.`id`, NULL, NULL, SUM(`financial_transactions_invoices`.`invoice_currency_value`)
FROM `financial_transactions_invoices`
JOIN `invoices` ON `financial_transactions_invoices`.`invoice` = `invoices`.`id`
GROUP BY `invoices`.`id`) AS `components`
GROUP by `invoice_id`;
Without tackling the problem in the way I have, I can't think of any other way I can do it within MySQL.
Any ideas? Appreciate any help.

You could create two views. One with the UNION Subquery, and one with the outer query.

Related

I'm trying to display total no. of orders by each vendor every year?

I'm trying to display total no. of orders by each vendor every year?
Table: Sales
Table: Products
Table: Vendors & Vendor_Info
select year(orderdate) as date_year,count(orderid) as count, OrderID
from products
NATURAL JOIN sales
NATURAL JOIN vendor_info
WHERE products.Vendor_Id=vendor_info.Vendor_Id
group by year(orderdate);
on doing this query i'm getting
query result
Where am i going wrong ? and what would the most efficient way to do this?
-- Table structure for table `products`
--
CREATE TABLE IF NOT EXISTS `products` (
`Product_id` varchar(255) NOT NULL,
`OrderId` int(11) NOT NULL,
`Manufacture_Date` date DEFAULT NULL,
`Raw_Material` varchar(255) DEFAULT NULL,
`Vendor_Id` int(11) DEFAULT NULL,
PRIMARY KEY (`Product_id`),
KEY `OrderId` (`OrderId`)
);
-- Table structure for table `sales`
--
CREATE TABLE IF NOT EXISTS `sales` (
`OrderID` int(11) NOT NULL AUTO_INCREMENT,
`OrderDate` date DEFAULT NULL,
`OrderPrice` int(11) DEFAULT NULL,
`OrderQuantity` int(11) DEFAULT NULL,
`CustomerName` varchar(255) DEFAULT NULL,
PRIMARY KEY (`OrderID`)
) AUTO_INCREMENT=10 ;
-- Table structure for table `Vendors`
--
CREATE TABLE IF NOT EXISTS `Vendors` (
`Raw_material` varchar(255) DEFAULT NULL,
`Vendors` varchar(255) DEFAULT NULL,
`Vendor_id` int(11) DEFAULT NULL,
KEY `Vendor_id` (`Vendor_id`)
)
-- Table structure for table `Vendor_info`
--
CREATE TABLE IF NOT EXISTS `Vendor_info` (
`Vendor_id` int(11) DEFAULT NULL,
`Vendor_name` varchar(255) DEFAULT NULL,
KEY `Vendor_id` (`Vendor_id`)
)
Some guesswork here, because you have not explained your requirement.
I guess you want to summarize numbers of orders by year. You can do that with a simpler query than your example, because all the data you need is in your sales table.
select year(orderdate) as date_year, count(orderid) as count
from sales
group by year(orderdate);
If this does not meet your need, please edit your question to give us more details.
If you want to get the number or orders by vendor and year yoy can use a query like this one:
select vendor_info.vendor_name,year(sales.orderdate),count(distinct products.orderId)
from vendor_info
join products on vendor_info.vendor_id=productos.vendor_id
join sales on products.orderid=sales.orderid
group by vendor_info.vendor_name,year(sales.orderdate)
Where you make a join from vendors to products and sales. Here you will get only the vendors who have sales.

How do I structure a Database with a Sales Table MySql

I have a sales table that I want to record all the sales done by the employee.
The problem i'm having is that I can only store one ProductId that comes from a Products table. What is wrong is that a sale has multiples products and with my current structure I can only store one ProductId. I know my approach is wrong but I just don't know how to properly fix it. The question I have is how do I store multiples products in the Sales Table.
This is my Sales Table columns.
CREATE TABLE `Sales` (
`SaleId` int(11) unsigned NOT NULL AUTO_INCREMENT,
`EmployeeId` int(11) unsigned NOT NULL,
`ProductId` int(11) unsigned NOT NULL,
PRIMARY KEY (`SaleId`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Your table and approach is ok.
You just have to insert one row in your sales table per product-id.
The unique key for the sales is the SaleId,
the foreign keys for Employee and Product are also there.
The only thing that's missing in your table is the sale quantity and amount.
It's the typical n:m-issue which is solved by the V-structure.
You need a second table, a Sales-Product table:
CREATE TABLE `SaleProduct` (
`SaleId` int(11) unsigned NOT NULL,
`ProductId` int(11) unsigned NOT NULL,
`Quantity` int(11) unsigned NOT NULL,
PRIMARY KEY (`SaleId`, `ProductId`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
and change your Sales Table:
CREATE TABLE `Sales` (
`SaleId` int(11) unsigned NOT NULL AUTO_INCREMENT,
`EmployeeId` int(11) unsigned NOT NULL,
PRIMARY KEY (`SaleId`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The second way I suggest is more "standard" but since I have no idea how do you query the table and the size of it, the first method might(?) be useful
concatenate all prod_id into a single string, get them using regexp
example: prod_id = 13_07_255_23_11
SELECT * FROM Sales WHERE ProductId REGEXP '(^|)23(|$)';
split a single sale into multiple entries, have sales_id and product_id be your combined unique/primary key

Mysql optimize slow query with explain

I'm working on MySQL 5.5.29-0ubuntu0.12.04.1.
I have the need to create a query that can sort results by date and by a score.
I read the documentation and the posts here on stackoverflow (specifically this) about how to optimize a query but I'm still struggling to do it well.
The key findings is that to avoid the use of a temporary table the ORDER BY or GROUP BY must contains only columns from the first table in the join queue, so that's why the use of the STRAIGHT_JOIN clause and the two slightly different queries.
To avoid confusion, I'm going to assign a number to various query configuration:
order by date with STRAIGHT_JOIN clause
order by score with STRAIGHT_JOIN clause
order by date without STRAIGHT_JOIN clause
order by score without STRAIGHT_JOIN clause
Following is query 1, takes about 2.5 seconds to complete:
SELECT STRAIGHT_JOIN item.id AS id
FROM item
INNER JOIN score ON item.id = score.item_id
LEFT JOIN url ON item.url_id = url.id
LEFT JOIN doc ON url.doc_id = doc.id
INNER JOIN feed ON feed.id = item.feed_id
INNER JOIN user_feed ON feed.id = user_feed.feed_id AND score.user_id = user_feed.user_id
LEFT JOIN star ON item.id = star.item_id AND score.user_id = star.user_id
JOIN unseen ON item.id = unseen.item_id AND score.user_id = unseen.user_id
WHERE score.user_id = 1 AND user_feed.id = 7
ORDER BY zen_time DESC
LIMIT 0, 10
Following is query 2 (first join tables are inverted and the ordering column is different), takes only about 0.01 seconds to complete:
SELECT STRAIGHT_JOIN item.id AS id
FROM score
INNER JOIN item ON item.id = score.item_id
LEFT JOIN url ON item.url_id = url.id
LEFT JOIN doc ON url.doc_id = doc.id
INNER JOIN feed ON feed.id = item.feed_id
INNER JOIN user_feed ON feed.id = user_feed.feed_id AND score.user_id = user_feed.user_id
LEFT JOIN star ON item.id = star.item_id AND score.user_id = star.user_id
JOIN unseen ON item.id = unseen.item_id AND score.user_id = unseen.user_id
WHERE score.user_id = 1 AND user_feed.id = 7
ORDER BY score DESC
LIMIT 0, 10
Following are the EXPLAIN results for the queries.
Explain for query 1:
Explain for query 2:
Explain for query 3:
Explain for query 4:
Profiler result for query 1:
Profiler result for query 2:
Profiler result for query 3:
Profiler result for query 4:
Following are tables definitions:
CREATE TABLE `doc` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`md5` char(32) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `Md5_index` (`md5`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `feed` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`url` text NOT NULL,
`title` text,
PRIMARY KEY (`id`),
FULLTEXT KEY `Title_url_index` (`title`,`url`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE `item` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`feed_id` bigint(20) unsigned NOT NULL,
`url_id` bigint(20) unsigned DEFAULT NULL,
`md5` char(32) NOT NULL,
PRIMARY KEY (`id`),
KEY `Md5_index` (`md5`),
KEY `Zen_time_index` (`zen_time`),
KEY `Feed_index` (`feed_id`),
KEY `Url_index` (`url_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `score` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) unsigned NOT NULL,
`item_id` bigint(20) unsigned NOT NULL,
`score` float DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `User_item_index` (`user_id`,`item_id`),
KEY Score_index (`score`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `star` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) unsigned NOT NULL,
`item_id` bigint(20) unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `User_item_index` (`user_id`,`item_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `unseen` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) unsigned NOT NULL,
`item_id` bigint(20) unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `User_item_index` (`user_id`,`item_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `url` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`doc_id` bigint(20) unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
KEY Doc_index (`doc_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `user` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`email` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
KEY `IDX_Email` (`email`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `user_feed` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) unsigned NOT NULL,
`feed_id` bigint(20) unsigned NOT NULL,
PRIMARY KEY (`id`),
KEY `User_feed_index` (`user_id`,`feed_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Here are the row counts for the tables involved in the query:
Score: 68657
Item: 197602
Url: 198354
Doc: 186113
Feed: 754
User_feed: 721
Star: 0
Unseen: 150762
Which approach should I take since my program needs to be able to order results both by zen_time and score in the fastest way possible?
Due to the different query speeds I decided to make an even more accurate analysis based on the various results I want to achieve.
The result sets I need are four:
Select all the items from a specific feed, order them by SCORE.score (intelligent order)
Select all the items from a specific feed, order them by ITEM.zen_time (time order)
Select all the items, order them by SCORE.score (intelligent order)
Select all the items, order them by ITEM.zen_time (time order)
The query so has to be adapted to those conditions, and its variable parts are:
STRAIGHT_JOIN yes/no
First JOIN table score/item
WHERE condition on specific feed yes/no
ORDER BY score/zen_time
All of the tests have been executed with the SELECT SQL_NO_CACHE instruction.
Following are the results:
Now it's clear what I have to do:
No STRAIGHT_JOIN, first JOIN table SCORE
No STRAIGHT_JOIN, first JOIN table SCORE
STRAIGHT_JOIN (I did beat MySQL engine here :D ), first JOIN table SCORE
STRAIGHT_JOIN (I did beat MySQL engine here :D ), first JOIN table ITEM

Sum total points for multiple tables in MySQL

I'd like to find the sum of a column in a single query given joins between multiple tables.
I have a table of Activities, and a table that maps Users performing an Activity, as well as a table mapping Teams to performed Activities. Both Users and Teams can perform the same activity multiple times.
Each activity is worth a set number of points, and I'd like to know the total number of points for a given user by totalling their activities with their team's activities.
I've tried various combinations of joins between the three tables, but cannot work out the correct query to total the points for a given user.
The following SQL will create a minimal version of this setup:
CREATE TABLE `activity` (
`id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`name` VARCHAR(255) NOT NULL DEFAULT '',
`points` INT(10) UNSIGNED NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=INNODB;
CREATE TABLE `team_action` (
`id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`activity_id` INT(11) UNSIGNED NOT NULL,
`date` DATETIME NOT NULL,
PRIMARY KEY (`id`),
CONSTRAINT `team_action_ibfk_1` FOREIGN KEY (`activity_id`) REFERENCES `activity` (`id`)
) ENGINE=INNODB;
CREATE TABLE `user_action` (
`id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`activity_id` INT(11) UNSIGNED NOT NULL,
`date` DATETIME NOT NULL,
PRIMARY KEY (`id`),
CONSTRAINT `user_action_ibfk_1` FOREIGN KEY (`activity_id`) REFERENCES `activity` (`id`)
) ENGINE=INNODB;
INSERT INTO `activity` (`id`, `name`, `points`)
VALUES (1,'Running',10), (2,'Swimming',20), (3,'Hiking',30), (4,'Cycling',40);
INSERT INTO `team_action` (`id`, `activity_id`, `date`)
VALUES (1,2,'2012-05-22 14:32:31'), (2,4,'2012-05-22 14:32:36');
INSERT INTO `user_action` (`id`, `activity_id`, `date`)
VALUES (1,1,'2012-05-22 14:32:08'), (2,1,'2012-05-22 14:32:18'), (3,3,'2012-05-22 14:32:23');
It is not clear from the table definitions how users are related to teams (i.e. for a user, how do you know which is "their" team?) But I think the key to summing the points will be to use SUM on the result of UNION ALL in a subquery.
Something along the lines of:
SELECT SUM(points) AS total
FROM
(SELECT points
FROM team_action JOIN activity ON(activity.id = team_action.activity_id)
WHERE team_action.id = my_team
UNION ALL
SELECT points
FROM user_action JOIN activity ON(activity.id = user_action.activity_id)
WHERE user_action.id = my_user) me_and_team

Select most recent from mysql table

I have two mysql tables:
Item containing items that one can buy:
CREATE TABLE `item` (
`itemid` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(50) DEFAULT NULL,
PRIMARY KEY (`itemid`)
) ENGINE=InnoDB;
Purchase containing all purchases:
CREATE TABLE `purchase` (
`purchaseid` int(11) NOT NULL AUTO_INCREMENT,
`date` date DEFAULT NULL,
`amount` int(11) DEFAULT NULL,
`itemid` int(11) DEFAULT NULL,
PRIMARY KEY (`purchaseid`)
) ENGINE=InnoDB;
I want to select the most 20 recent purchases based on date and purchaseid and join the item table to show the name of these purchases. If an item has been purchased more than once in the 20 recent purchases it should only show up once. No duplicates. I really can't figure this out.. Maybe you can? Thanks!
Wouldn't it be:
SELECT `name`
from `item` join `purchase` using(`itemid`)
group by `itemid` order by `date` desc limit 20
OR
SELECT DISTINCT `name`
from `item` join `purchase` using(`itemid`)
order by `date` desc limit 20
Using DISTINCT allows you to omit duplicates, as does GROUP BY (which also allows you to perform functions on the grouped data)