Edit2: Chose to separate the queries and collate/handle the information as a whole outside of the database's output. Taking these out in a .CSV format, and adding them into Excel where I'm going to be running the actual numbers.
Query 1 to pull out orders and desired info:
SELECT
shipstation_orders_v2.id AS SSO_id,
shipstation_orders_v2.order_number AS SSO_orderNumber,
shipstation_orders_v2.order_id AS SSO_orderID,
shipstation_orders_v2.storename AS SSO_storeName,
shipstation_orders_v2.order_date AS SSO_orderDate,
shipstation_orders_v2.order_total AS SSO_orderTotal,
shipstation_orders_v2.name AS SSO_name,
shipstation_orders_v2.company AS SSO_company
FROM shipstation_orders_v2
GROUP BY shipstation_orders_v2.id,
shipstation_orders_v2.order_number,
shipstation_orders_v2.order_id,
shipstation_orders_v2.storename,
shipstation_orders_v2.order_date,
shipstation_orders_v2.order_total,
shipstation_orders_v2.name,
shipstation_orders_v2.company
ORDER BY SSO_orderDate
Query 2 to pull out fulfillments and equivalent info:
SELECT DISTINCT
shipstation_orders_v2.id AS SSO_id,
shipstation_fulfillments.id AS SSF_id,
shipstation_fulfillments.order_number AS SSF_orderNumber,
shipstation_orders_v2.order_number AS SSO_orderNumber,
shipstation_orders_v2.order_id AS SSO_orderID,
shipstation_orders_v2.storename AS SSO_storeName,
shipstation_orders_v2.order_date AS SSO_orderDate,
shipstation_fulfillments.order_date AS SSF_orderDate,
shipstation_orders_v2.order_total AS SSO_orderTotal,
shipstation_fulfillments.amount_paid AS SSF_amountPaid,
shipstation_orders_v2.name AS SSO_name,
shipstation_orders_v2.company AS SSO_company,
shipstation_fulfillments.name AS SSF_name,
shipstation_fulfillments.company AS SSF_company
FROM shipstation_fulfillments
INNER JOIN shipstation_orders_v2
ON shipstation_fulfillments.order_number =
shipstation_orders_v2.order_number
WHERE shipstation_fulfillments.order_number =
shipstation_orders_v2.order_number
GROUP BY shipstation_orders_v2.id,
shipstation_fulfillments.id,
shipstation_fulfillments.order_number,
shipstation_orders_v2.order_number,
shipstation_orders_v2.order_id,
shipstation_orders_v2.storename,
shipstation_orders_v2.order_date,
shipstation_fulfillments.order_date,
shipstation_orders_v2.order_total,
shipstation_fulfillments.amount_paid,
shipstation_orders_v2.name,
shipstation_orders_v2.company,
shipstation_fulfillments.name,
shipstation_fulfillments.company
Edit: Question marked as answered. I figured out another way to do it that wasn't quite as harebrained. Props to DRapp for getting my brain moving.
Original Code is below Wall of Text
I'm a self-taught MySQL database user. I won't say administrator, since it's just me. I've put together a small database for work - about 60,000 rows and a maximum of 51 columns spread over three tables. I use this at work as a way to organize a fairly disparate sales data setup and make sense of it to identify trends, seasonality, all that good stuff. I work primarily with Shipstation data.
My problem is when I needed to introduce this third table. With two tables, obviously, it's just a simple JOIN. I got that working just fine. I'm having quite a bit of trouble setting up the JOINs correctly for this third table.
I'm attempting to JOIN the data from the two innermost queries to shipstation_orders_v2 and order_keys to the shipstation_fulfillments results I have in the third table.
For those of you who don't use Shipstation or aren't familiar with this element of it, fulfillments are in a different category than orders and don't use quite the same data. This is my dirty way of gluing them together so we have some decent, manipulable information on sales and shipping trends, etc.
I am making an internal query from shipstation_orders_v2 to order_keys as a way to SELECT DISTINCT the sum totals of split orders. I had problems with data duplication before I built up that subquery. With the (now) subquery and sub-subquery, the duping problem has been eliminated and with just those two tables it worked fine.
The issue is, when I'm making the SELECT from shipstation_fulfillments with a JOIN to the subquery and sub-subquery, I'm hitting a roadblock.
I've gotten several errors while working on this query. In order of occurrence and resolution:
Error 2013, lost connection to server during query (which told me I'm doing a full table read on three joined tables, since it isn't erroring out beforehand, but my rinkadink setup can't handle it). I got rid of that one.
Then, Error 1051 for an unidentified table name shipstation_fulfillments. To me I think it might be an issue for the query aliases. I am not sure.
Finally, good ole Error 1064, incorrect syntax on the first subquery after the
SELECT shipstation_fulfillments arguments.
Being self-taught, I'd virtually guarantee I'm merely missing an element of syntax somewhere that would appear fairly obvious to a well-practiced user of MySQL. Below is my current query setup.
If there needs to be any clarification, let me know.
SELECT
`shipstation_fulfillments`.`order_date` AS `orderDate`,
`shipstation_fulfillments`.`order_number` AS `orderNumber`,
(`shipstation_fulfillments`.`amount_paid` + `shipstation_fulfillments`.`tax_paid`) AS "Total Paid",
`shipstation_fulfillments`.`name` AS `name`,
`shipstation_fulfillments`.`company` AS `company`,
FROM
(
(SELECT
COUNT(`shipstation_orders_v2`.`order_key`) AS `orderCount`,
`shipstation_orders_v2`.`key_id` AS `key_id`,
`shipstation_orders_v2`.`order_number` AS `order_number`,
MAX(`shipstation_orders_v2`.`order_date`) AS `order_date`,
`shipstation_orders_v2`.`storename` AS `store`,
(`shipstation_orders_v2`.`order_total` - `shipstation_orders_v2`.`shippingPaid`) AS `orderPrice`,
`shipstation_orders_v2`.`shippingpaid` AS `shippingPaid`,
SUM(`shipstation_orders_v2`.`shippingpaid`) AS `SUM shippingPaid`,
`shipstation_orders_v2`.`order_total` AS `orderTotal`,
SUM(`shipstation_orders_v2`.`order_total`) AS `SUM Total Amount Paid`,
`shipstation_orders_v2`.`qtyshipped` AS `qtyShipped`,
SUM(`shipstation_orders_v2`.`qtyshipped`) AS `SUM qtyShipped`,
`shipstation_orders_v2`.`name` AS `name`,
`shipstation_orders_v2`.`company` AS `company`
FROM
(SELECT DISTINCT
`order_keys`.`key_id` AS `key_id`,
`order_keys`.`order_key` AS `order_key`,
`shipstation_orders_v2`.`order_number` AS `order_number`,
`shipstation_orders_v2`.`order_id` AS `order_id`,
`shipstation_orders_v2`.`order_date` AS `order_date`,
`shipstation_orders_v2`.`storename` AS `storename`,
`shipstation_orders_v2`.`order_total` AS `order_total`,
`shipstation_orders_v2`.`qtyshipped` AS `qtyshipped`,
`shipstation_orders_v2`.`shippingpaid` AS `shippingpaid`,
`shipstation_orders_v2`.`name` AS `name`,
`shipstation_orders_v2`.`company` AS `company`
FROM
(`shipstation_orders_v2`
JOIN `order_keys` ON ((`order_keys`.`order_key` = `shipstation_orders_v2`.`order_id`)))) `t`)
JOIN `shipstation_fulfillments`
ON (`shipstation_orders_v2`.`order_number` = `shipstation_fulfillments`.`order_number`)) `w`
As a couple notes... As for long table names, no problem, but you can use alias references to them such as I have done via example ...ShipStation_Fulfillments SSF... the "SSF" is now an alias for shorter typing yet still makes sense of origin.
When changing column names in query via "AS", you only need the as if your column name result will change from its original as you had in the beginning such as SSF.order_date AS orderDate where you remove the "_" from the final column name, but also in "Total Paid" (yet I HATE column names with embedded spaces, let the user interface handle labeling things, but thats just me).
When typing table.column (or alias.column), doing via CamelCasing helps readability vs camelcasing slightly harder to read where the brain naturally breaks into readable words for us.
Other issue based on query. Outer query portions can't recognize aliases from inner closed queryies, only the alias of the subselect as you had with the "t" and "w" aliases.
Next, when doing JOINs, my preference is to read them in the way the tables are within the query listing the first one on the left, and whatever is joined TO on the right.
If went from Table A Join to Table B, the ON clause would be ON A.KeyID = B.KeyID vs B.KeyID = A.KeyID especially if you are going several tables... A->B, B->C, C->D
Any query with aggregates (sum, avg, count, min, max, etc) must have a "GROUP BY" clause to identify when each record should break. In your example, I would assume break on the original sales order.
Although this query IS NOT WORKING, here is a cleaned-up version of your query showing implementations from above.
SELECT
SSF.order_date AS OrderDate,
SSF.order_number AS OrderNumber,
(SSF.amount_paid + SSF.tax_paid) AS `Total Paid`,
SSF.name,
SSF.company
FROM
( SELECT
SSOv2.key_id,
SSOv2.order_number,
SSOv2.storename AS store,
SSOv2.order_total - SSOv2.shippingPaid AS OrderPrice,
SSOv2.ShippingPaid,
SSOv2.order_total AS OrderTotal,
SSOv2.QtyShipped,
SSOv2.name,
SSOv2.company,
COUNT(SSOv2.order_key) AS orderCount,
MAX(SSOv2.order_date) AS order_date,
SUM(SSOv2.shippingpaid) AS `SUM shippingPaid`,
SUM(SSOv2.order_total) AS `SUM Total Amount Paid`,
SUM(SSOv2.qtyshipped) AS `SUM qtyShipped`
FROM
( SELECT DISTINCT
OK.key_id AS key_id,
OK.order_key AS order_key,
SSOv2.order_number AS order_number,
SSOv2.order_id AS order_id,
SSOv2.order_date AS order_date,
SSOv2.storename AS storename,
SSOv2.order_total AS order_total,
SSOv2.qtyshipped AS qtyshipped,
SSOv2.shippingpaid AS shippingpaid,
SSOv2.name AS name,
SSOv2.company AS company
FROM
shipstation_orders_v2 SSOv2
JOIN order_keys
ON SSOv2.order_id = OK.order_key
JOIN shipstation_fulfillments SSF
ON SSOv2.order_number = SSF.order_number ) t
) w
Next, without seeing actual data or listed structures critical to solve the query, I will ask you edit your existing post. Create a sample table listing table, columns and sample data so we can see the basis of what you are aggregating and trying to get out of the query. Especially show where there could be multiple rows per order and fulfillment respectively and a sample answer of what you EXPECT the results to show.
this seems like a very easy question and in fact I know how to help myself but I hope to find a quicker approach to this problem.
I have two tables with the same structure in an sql-dbms, whereas some lines of data occur in both tables. My way of finding these duplicates is making an inner-join with all columns in the "on-statement", like:
create table target as
select
a.column_1
a.column_2
a.column_3
...
a.column_20
from table source_1 as a
inner join source_2 as b
on a.column_1 = b.column_1 and
a.column_2 = b.column_2 and
...
a.column_20 = b.column_20
;
With a large amount of columns in the tables, this produces a long piece of code. Is there some kind of syntax or method to make this shorter?
Thanks for your help,
kind regards.
If your tables are of same structure then try using the following query,
select * from source_1
INTERSECT
select * from source_2
it will give you the common rows from both the tables
I have a set of table of data that looks like this:
[id],[testid],[date],[type],[score1],[score2],[score3],[score4]
I need it to look like this
[id],[testid][date],[type],[score],[scorecode] (score1)
[id],[testid][date],[type],[score],[scorecode] (score2)
[id],[testid][date],[type],[score],[scorecode] (score3)
[id],[testid][date],[type],[score],[scorecode] (score4)
The scorecode is dependent on which score(1-4), so the score1 row would need a scorecode of ACTMATH, score2 would need a different scorecode.
When I orginally built this, I used UNION ALL. However, I just want to make sure there's not a more efficient way of completing this.
You should break out the Scores into their own table
Which it has a ScoreCode and Score and a link back to the Id of the Test
And then you would do a SQL Join query and easily get your results in that format.
Tests
[testid],[date],[type]
Scores
[ScoreId],[testid],[score],[scorecode]
SELECT *
FROM Tests T
JOIN Scores S
ON T.Testid = S.TestId
I think there is no other way than using UNION.
You can check these other question: Mysql Convert Column to row (Pivot table)
I had an issue joining multiple tables to retrieve the data I needed. In order to accomplish the proper results I had to first create a view (shown below) called: vwinvgrossrev :
SELECT dbo.inv_item.inv_num, dbo.inv_item.co_line,
dbo.inv_hdr.co_num, dbo.inv_hdr.inv_date, dbo.inv_item.qty_invoiced,
dbo.inv_item.price
FROM dbo.inv_item INNER JOIN
dbo.inv_hdr ON dbo.inv_item.inv_num = dbo.inv_hdr.inv_num
and then I had to join the view on my final table in order to create a proper summation of the values that I wanted
select sum(vwinvgrossrev.qty_invoiced*vwinvgrossrev.price)
from vwinvgrossrev,coitem
WHERE vwinvgrossrev.co_num=coitem.co_num
AND coitem.Uf_Erne='Y'
AND vwinvgrossrev.co_line=coitem.co_line
AND DATEPART(mm,vwinvgrossrev.inv_date) = DATEPART(mm,Getdate())
AND YEAR(vwinvgrossrev.inv_date) = YEAR(Getdate())
My question is this. Is there anyway to do this in a single query. The problem is all 3 tables have a many to many relationship with one another and always returns the wrong value when joining all 3 tables.
If your current query (using the view) gives you what you are looking for then I believe that this query will give you the same thing in a single query:
SELECT sum(inv_item.qty_invoiced*inv_item.price)
FROM inv_item
JOIN inv_hdr ON dbo.inv_item.inv_num = dbo.inv_hdr.inv_num
JOIN coitem ON (inv_hdr.co_num=coitem.co_num
AND inv_item.co_line=coitem.co_line)
WHERE coitem.Uf_Erne='Y'
AND vwinvgrossrev.co_line=coitem.co_line
AND DATEPART(mm,inv_hdr.inv_date) = DATEPART(mm,Getdate())
AND YEAR(inv_hdr.inv_date) = YEAR(Getdate())
However, I'll be surprised if that is what you are looking for.
I am assuming that you don't actually have a many-to-many between all 3. I am assuming that inv_hdr has a unique inv_num and inv_item has many of those. Similarly I am assuming that co_num is unique within inv_hdr but has multiple occurrences within coitem. These are not many-to-many but rather 1-to-many relationships. Since SELECT multiplies rows I think you are going to be summing way more than you want.
Why do you want to include coitem at all if you are just interested in inv_item.qty_invoiced and inv_item.price? Is it just to limit on the Uf_Erne column? Is coitem always going to have either 1 or 0 records for that co_num and co_line with Uf_Erne='Y'?
In sum, I've reproduced your query in a single query without a view, but I think you may need to think through whether this is really what you want.
You need to refine your database schema to normalise out the many-to-many relationships. That is a given and is standard practice regardless of your current problem. The fact that it will simplify your current problem is the reason why. There are many tutorials/blog/etc out there to show you how to do this with join tables so I am not going into it here.
I have the following tables.
Articles table
a_id INT primary unique
name VARCHAR
Description VARCHAR
c_id INT
Category table
id INT
cat_name VARCHAR
For now I simply use
SELECT a_id,name,Description,cat_name FROM Articles LEFT JOIN Category ON Articles.a_id=Category.id WHERE c_id={$id}
This gives me all articles which belong to a certain category along with category name.
Each article is having only one category.
AND I use a sub category in a similar way(I have another table named sub_cat).But every article doesn't necessary have a sub category.It may belong to multiple categories instead.
I now think of tagging an article with more than one category just like the questions at stackoverflow are tagged(eg: with multiple tags like PHP,MYSQL,SQL etc).AND later I have to display(filter) all article with certain tags(eg: tagged with php,php +MySQL) and I also have to display the tags along with the article name,Description.
Can anyone help me redesign the database?(I am using php + MySQL at back-end)
Create a new table:
CREATE TABLE ArticleCategories(
A_ID INT,
C_ID INT,
Constraint PK_ArticleCategories Primary Key (Article_ID, Category_ID)
)
(this is the SQL server syntax, may be slightly different for MySQL)
This is called a "Junction Table" or a "Mapping Table" and it is how you express Many-to-Many relationships in SQL. So, whenever you want to add a Category to an Article, just INSERT a row into this table with the IDs of the Article and the Category.
For instance, you can initialize it like this:
INSERT Into ArticleCategories(A_ID,C_ID)
SELECT A_ID,C_ID From Articles
Now you can remove c_id from your Articles table.
To get back all of the Categories for a single Article, you would do use a query like this:
SELECT a_id,name,Description,cat_name
FROM Articles
LEFT JOIN ArticleCategories ON Articles.a_id=ArticleCategories.a_id
INNER JOIN Category ON ArticleCategories.c_id=Category.id
WHERE Articles.a_id={$a_id}
Alternatively, to return all articles that have a category LIKE a certain string:
SELECT a_id,name,Description
FROM Articles
WHERE EXISTS( Select *
From ArticleCategories
INNER JOIN Category ON ArticleCategories.c_id=Category.id
WHERE Articles.a_id=ArticleCategories.a_id
AND Category.cat_name LIKE '%'+{$match}+'%'
)
(You may have to adjust the last line, as I am not sure how string parameters are passed MySQL+PHP.)
Ok RBarryYoung you asked me about an reference/analyse you get one
This reference / analyse is based off the documention / source code analyse off the MySQL server
INSERT Into ArticleCategories(A_ID,C_ID)
SELECT A_ID,C_ID From Articles
On an large Articles table with many rows this copy will push one core off the CPU to 100% load and will create a disk based temporary table what will slow down the complete MySQL performance because the disk will be stress out with that copy.
If this is a one time process this is not that bad but do the math if you run this every time..
SELECT a_id,name,Description
FROM Articles
WHERE EXISTS( Select *
From ArticleCategories
INNER JOIN Category ON ArticleCategories.c_id=Category.id
WHERE Articles.a_id=ArticleCategories.a_id
AND Category.cat_name LIKE '%'+{$match}+'%'
)
Note dont take the Execution Times on sqlfriddle for real its an busy server and the times vary alot to make a good statement but look to what View Execution Plan has to say
see http://sqlfiddle.com/#!2/48817/21 for demo
Both querys always trigger an complete table scan on table Articles and two DEPENDENT SUBQUERYS thats not good if you have an large Articles table with many records.
This means the performance depends on the number of Articles rows even when you want only the articles that are in the category.
Select *
From ArticleCategories
INNER JOIN Category ON ArticleCategories.c_id=Category.id
WHERE Articles.a_id=ArticleCategories.a_id
AND Category.cat_name LIKE '%'+{$match}+'%'
This query is the inner subquery but when you try to run it, MySQL cant run because it depends on a value of the Articles table so this is correlated subquery. a subquery type that will be evaluated once for each row processed by the outer query. not good indeed
There are more ways off rewriting RBarryYoung query i will show one.
The INNER JOIN way is much more efficent even with the LIKE operator
Note ive made an habbit out off it that i start with the table with the lowest number off records and work my way up if you start with the table Articles the executing will be the same if the MySQL optimizer chooses the right plan..
SELECT
Articles.a_id
, Articles.name
, Articles.description
FROM
Category
INNER JOIN
ArticleCategories
ON
Category.id = ArticleCategories.c_id
INNER JOIN
Articles
ON
ArticleCategories.a_id = Articles.a_id
WHERE
cat_name LIKE '%php%';
;
see http://sqlfiddle.com/#!2/43451/23 for demo Note that this look worse because it looks like more rows needs to be checkt
Note if the Article table has low number off records RBarryYoung EXIST way and INNER JOIN way will perform more or less the same based on executing times and more proof the INNER JOIN way scales better when the record count become larger
http://sqlfiddle.com/#!2/c11f3/1 EXISTS oeps more Articles records needs to be checked now (even when they are not linked with the ArticleCategories table) so the query is less efficient now
http://sqlfiddle.com/#!2/7aa74/8 INNER JOIN same explain plan as the first demo
Extra notes about scaling it becomes even more worse when you also want to ORDER BY or GROUP BY the NOT EXIST way has an bigger chance it will create an disk based temporary table that will kill MySQL performance
Lets also analyse the LIKE '%php%' vs = 'php' for the EXIST way and INNER JOIN way
the EXIST way
http://sqlfiddle.com/#!2/48817/21 / http://sqlfiddle.com/#!2/c11f3/1 (more Articles) the explain tells me both patterns are more or less the same but 'php' should be little faster because off the const type vs ref in the TYPE column but LIKE %php% will use more CPU because an string compare algoritme needs to run.
the INNER JOIN way
http://sqlfiddle.com/#!2/43451/23 / http://sqlfiddle.com/#!2/7aa74/8 (more Articles) the explain tell me the LIKE '%php%' should be slower because 3 more rows need to be analysed but not shocking slower in this case (you can see the index is not really used on the best way).
RBarryYoung way works but doenst keep performance atleast not on a MySQL server
see http://sqlfiddle.com/#!2/b2bd9/1 or http://sqlfiddle.com/#!2/34ea7/1
for examples that will scale on large tables with lots of records this is what the topic starter needs