I have a problem in mysql. I need to fetch certain sets of records from a table that has the following data.
Let me brief you guys about the table. every primary item has a similar item and that similar item has a size.
but we have to show unique items from the above table having least size.
for example.. primary item is A and it's corresponding item in X and X is also primary and it's similar item in A but X also has a similar item P so A,X,P all are similar and in these the item with least size is P with size 4 so we have fetch P from the table with a query.
Furthermore the records are 1-M and all primary item as similar items and all similar items exits in the primary field in the database. And this table contain nearly 3 lacs record for fetching and looping one records at a time will not work..
Any help will be appreciate.
Thanks
Jawed Shamshedi
Given the schema:
CREATE TABLE similar_test ( id int(11) NOT NULL DEFAULT '0',
Stock varchar(20) NOT NULL, SimilarStock varchar(20) NOT NULL,
ItemSize decimal(18,2) DEFAULT '0.00' ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
please check this : I am considering for 1 level depth :
select * , s.ItemSize as minSize from similar_test as p
join similar_test a s on s.Stock= p.SimilarStock
where p.Sock='A'
order by s.itemSize
limit 1
If that is not addressing you questions as you said in comments the other options are :
a. create another table which will hold PRE- calculated values are desired
b. Write a stored procedure to go through the item list upto N-levels as N is unspecified
in this case (as I suspected)
c. Do the sorting and grouping in PHP/or any other language you are using
this might also help you hierarchical data
regards
Related
I'm fairly new to sql based databases, what I'm trying to implement is an inventory system where each user can have infinite amount of items with more than 20 item types.
What I'm trying right now is:
having an items table where I keep track of all the items the user has.
CREATE TABLE items (
id INT AUTO_INCREMENT PRIMARY KEY,
table_name VARCHAR(255) NOT NULL,
item_id INT NOT NULL,
user_id INT NOT NULL
)
and upon request to get all of the items, I SELECT items with user_id from items table and then SELECT for each table_name which ends up SELECT ing for each item type ( like 10-20 queries per user ) and if I want to delete an item I have to do it for two tables which is a pain and I really think there's a better way for this ( I request for each table only once but still )
any help is appreciated a lot!
I think I know what the problem is, but I need some help going in the correct direction.
I've got a tables 'products', and I've also got several temp_product tables for various suppliers.
The goal here is to update several fields in the products table from the appropriate temp_product table. My product suppliers give me a CSV file with all of their inventory data. Originally I was just looping through the CSV and updating line by line, but this takes forever, so now I load it into a temporary table using LOAD DATA LOCAL INFILE.
The problem I am having, is the UPDATE queries take forever to run, and most of the time MySQL just completely crashes. I am hoping I can show you my table structure, and somebody could help me out with what kind of key/index setup would work best?
I've tried 2 different update query variations, but neither one is working.
UPDATE product AS p, temp_product AS t
SET p.quantity = t.quantity,
p.ean = t.inventory,
p.cost = t.price,
p.date_modified = NOW()
WHERE p.sku = t.sku
-AND-
UPDATE temp_product AS t
INNER JOIN product AS p ON p.sku = t.sku
SET p.quantity = t.quantity,
p.ean = t.inventory,
p.cost = t.price,
p.date_modified = NOW()
Here is the structure to my tables in question:
temp_product
sku varchar(15) PRI
status varchar(2)
statusid int(11)
quantity int(11)
inventory varchar(15)
ETA varchar(25)
ETA_Note varchar(255)
price double(10,2)
product
product_id int(11) PRI
model varchar(64)
sku varchar(64)
upc varchar(50)
ean varchar(50)
mpn varchar(64)
location varchar(128)
quantity int(4)
price decimal(15,4)
cost decimal(15,4)
status tinyint(1)
date_added datetime
date_modified datetime
I have a feeling I could get this to work correctly if I had keys/indices set up correctly. The only thing I have set up now is the Primary Key, but those don't match up across all the tables. I'm pretty new to all this, so any help would be appreciated.
To make things even more complicated, I'm not sure if some of my suppliers use the same SKUs, so I would like to update the product table WHERE sku = sku and location = 'suppliername'.
Thanks for the help!
EDIT: Slimmed down the problem a little bit, originally had a product and supplier_product table to update, once I get the product table working I can probably take it from there.
First of all, could you run SHOW CREATE TABLE product; and SHOW CREATE TABLE temp_product; and paste the results? Also, how exactly large is your product table? (select count(1) from products can help)
Regarding the keys: you need at least to add sku key to your product table.
If sku is supposed to be a unique field, then you can do it with the following command:
ALTER TABLE product ADD UNIQUE KEY sku(sku);
If sku is NOT a unique field, then you can still add it as a key like that:
ALTER TABLE product ADD KEY sku(sku);
but in that case, this mean that for one record with a partcular sku from the temp_product table, you will update more than one record in your product table.
Regarding the table size: even if the table is large (say several million rows), but it's OK to run queries that take a lot of time (for example, if you are the only one using this database) then after you have added the key, either of the variants should in principle work and take less time than what it takes now. Otherwise, you would be better off with doing the update in batches (e.g. 100, 500 or 1000 records at a time) and preferably with some script that might even wait a little between the updates. This is especially recommended if your database is a master database that replicates to slaves.
I've created an insert only table for the purpose of speed and maintaining a history. It's structure is very generic, and is as follows:
`id` bigint(20) unsigned NOT NULL auto_increment,
`user_id` bigint(20) unsigned NOT NULL,
`property` varchar(32) NOT NULL,
`value` longblob NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
It's simply a key/value table with a user_id assigned to it. This approach has its advantages as not all users have the same properties, so fields aren't wasted in a table. Also, it allows for a rolling log of changes, since I can see every change to a particular property ever made by a user.
Now, since no deletes or updates ever occur in this table, I can assume that the greatest id will always be the newest entry.
However, I want to select multiple properties at once, for example 'address1', 'address2', 'city', 'state', and I want each to be the entry of it's type with the highest id.
So, if they have changed their 'state' property 8 times, and 'city' property 4 times, then I'd only want a SELECT to return the latest of each (1 state and 1 city).
I'm not sure this can even be done efficiently with this type of a table, so I'm open to different table approaches.
Please, let me know if I need to produce anymore information or clarify my question better.
===
I tried the following, but, there could be 3 rows of 'address1' changes after the last 'address2' change. Perhaps using a GROUP BY will work?
SELECT property, value FROM kvtable WHERE user_id = 1 AND (property = 'address1' OR property = 'address2') ORDER BY id
Assuming your ids are incremental integers and you have not manually specified them out of order, you can do this with a few MAX() aggregates in a subquery. The point of the subquery is to return the latest entry per property name, per user. That is joined against the whole table to pull in the associated property values. Essentially, the subquery discards all rows which don't have a max(id) per group.
SELECT kvtable.*
FROM
kvtable
JOIN (
SELECT
MAX(id) AS id,
user_id,
property
FROM kvtable
/* optionally limit by user_id */
WHERE user_id = <someuser>
GROUP BY user_id, property
) maxids ON kvtable.id = maxids.id
As part of a very slow refactoring process of an inherited system, I need to eliminate a couple of slow joins and subqueries. As I'm familiarising myself with the system, I'm slowly sanitising the database structure, to get rid of the held-together-by-duct-tape feeling, making incremental improvements, hoping nothing breaks in the meantime. Part of this involves combining data from two tables linked by a third into one.
Table structure is similar to this:
CREATE TABLE groups
(
group_id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,
-- various other fields that are appropriate to groups
...
-- these fields need to be filled
a ENUM(...) NOT NULL,
b INTEGER NOT NULL,
c VARCHAR(...) NOT NULL
);
CREATE TABLE items
(
-- key is determined by an external data source
item_id INTEGER NOT NULL PRIMARY KEY,
-- various other fields that are appropriate to items
...
-- these fields shouldn't be here, but in the groups table
a ENUM(...) NOT NULL,
b INTEGER NOT NULL,
c VARCHAR(...) NOT NULL
);
CREATE TABLE group_items
(
item_id INTEGER NOT NULL,
group_id INTEGER NOT NULL,
PRIMARY KEY (item_id,group_id)
);
An item may be in more than one group. Each record in the table "items" has values for columns a, b and c, which are actually not properties of the items, but of the groups of which the items are a part. (This is causing problems, as the values may be different if the item is in another group).
I can't remove the fields from the items table yet, as they are filled by an insane import process from an almost-as-insane data source. Until I get around to fixing the import process, I'm stuck with having the fields exist in the items table, but in the short term at least I can eliminate the slow lookups to get them.
Right now I have a loop in PHP that runs over each group, takes the values from the first item it encounters (which is fine -- all items in a group will have the same values for a, b and c) and places them into the group. This process is rather slow and laborious and unfortunately runs very frequently on an overloaded and underpowered server. Is there a smart way to copy these (and only these) values from the items table into the groups table and have MySQL do the heavy lifting, rather than relying on a PHP script?
Looks like I found my own answer. As the number of items in each group is relatively small, there may be some duplicate work being done but it's not a bottleneck and much faster than the PHP loop:
UPDATE
groups g
INNER JOIN group_items USING(group_id)
INNER JOIN items i USING(item_id)
SET
g.a = i.a,
g.b = i.b,
g.c = i.c;
Seems to do what I need.
I have the following table:
Id ParentId Weight
1 1 0
2 1 10
3 2 5
ParentId references Id of the same table. How can I query this table so that I join it on itself, adding up the cumulative weight of the third column?
For example, if I wanted to know the cumulative weight of Id 2, the result would return 15 (Id2 + Id3 = 15) as the parent of item 3 is 2. If I wanted to know the cumulative weight of item 3, it would return 5, as no records have a parent id of item 3.
Essentially, if the record I am querying has a child, I want to add the sequence of data's children and return one result.
Is this possible to do in one fell swoop to the database or would I have to loop through the entire record set to find matches?
Take a look on this article. If your table is not updated frequently, you can modify a little their GenericTree procedure that it generates all paths for all rows (and call it every time you insert record into the table or update ParentId column), store this data into a new table, and then you can perform all the tasks required using simple queries. Personally, I end up with the following table structure:
CREATE TABLE `tree_for_my_table` (
`rootID` INT(11) NOT NULL, // root node id
`parentID` INT(11) NOT NULL, // current parent id
`childID` INT(11) NOT NULL, // child id (direct child of the parent)
`level` INT(11) NOT NULL, // how far child is from root
PRIMARY KEY (`rootID`, `parentID`, `childID`),
UNIQUE INDEX `childID` (`childID`, `level`)
)
Populating data for that table doesn't take too long even for a quite large my_table.
Last I looked, mysql didn't have a built-in way of doing hierarchical queries, but you can always use a technique such as the adjacency list, discussed (among other techniques) in Managing Hierarchical Data in MySQL, which encodes the hierarchy in another table and lets you join against that to retrieve subtrees in your hierarchy.
You need to index your tree. See Managing Hierarchical Data in MySQL for some ways to do this.