I've got two tables where I'm trying to insert data from one to another, I've been able to find a few examples of how this can be accomplished on the web, the problem is these examples mostly rely on identical table structure between the two ... you see I'm trying to insert some data from one table into another table with quite a different structure.
I'm trying to insert data from a table called 'catalog_product_entity_media_gallery' into a table called 'catalog_product_entity_varchar'. Below is a simple description of their structure
The 'catalog_product_entity_varchar' looks as follows:
value_id | entity_type_id | attribute_id | store_id | entity_id | value
PK INT INT INT INT VARCHAR
And the 'catalog_product_entity_media_gallery' table looks as follows:
value_id | attribute_id | entity_id | value
PK INT INT VARCHAR
I need to insert the entity, and value columns from catalog_product_entity_media_gallery into catalog_product_entity_varchar. However as you can see the structure is quite different.
The query I'm trying to use is as follows
USE magento_db;
INSERT INTO catalog_product_entity_varchar(entity_type_id, attribute_id, store_id, entity_id, value)
SELECT
4,
74,
0,
catalog_product_entity_media_gallery.entity_id,
catalog_product_entity_media_gallery.value
FROM catalog_product_entity_media_gallery;
I only need the entity_id and value from media_gallery and the other values are always the same, I have tried to do this using the above but this is just hanging in MySQL (no errors)
I think it's due to the fact that I'm trying to select 4, 74 and 0 from catalog_product_entity_media_gallery but I'm not 100% sure (apologies, I'm a bit of a novice with MySQL)
Can anybody point me in the right direction? Is there any way way I can insert some data from the media table whilst inserting static values for some columns? (I hope this all makes sense)
The query syntax is ok.
However, there may be issues with the unique and foreign keys in catalog_product_entity_varchar table, which doesn't allow you to insert data. Also the query may be waiting for some other query to complete (if your query is just a part of bigger scenario), so it is an issue with locking. Most probable is the first case.
Currently, the question lacks important details:
The MySQL client / programming code you use to perform query. So we
are not able to see the case in full and to reproduce it correctly
The scenario you perform. I.e. whether you do it inside the Magento application in some
module during a web-request. Or whether there are other queries in your script,
some opened transactions, other people accessing the DB server, etc.
Based on most probable assumption that you just don't see the actual error with unique/foreign keys, you may try the following queries.
1) Unique index failure.
Try this:
USE magento_db;
INSERT INTO catalog_product_entity_varchar(entity_type_id, attribute_id, store_id, entity_id, value)
SELECT
4 as etid,
74 as aid,
0 as sid,
catalog_product_entity_media_gallery.entity_id as eid,
catalog_product_entity_media_gallery.value as val
FROM
catalog_product_entity_media_gallery
GROUP BY
eid, aid, sid;
There is a huge possibility, that you insert non-unique entries, because catalog_product_entity_media_gallery can hold multiple entries for the same product, while catalog_product_entity_varchar can not. If the query above successfully completes, then the issue is really with unique key. In such a case you must re-verify what you want to achieve, because the initial aim (not the query itself) is wrong.
2) Wrong foreign key (non-existing attribute 74)
Try this (replacing ATTRIBUTE_CODE and ATTRIBUTE_ENTITY_TYPE_ID with the values you need, e.g. 'firstname' and 6):
USE magento_db;
INSERT INTO catalog_product_entity_varchar(entity_type_id, attribute_id, store_id, entity_id, value)
SELECT
4 as etid,
eav_attribute.attribute_id as aid,
0 as sid,
gallery.entity_id as eid,
gallery.value as val
FROM
catalog_product_entity_media_gallery AS gallery
INNER JOIN
eav_attribute
ON
eav_attribute.attribute_code = '<ATTRIBUTE_CODE>'
AND eav_attribute.entity_type_id = <ATTRIBUTE_ENTITY_TYPE_ID>
GROUP BY
eid, aid, sid;
If it executes successfully AND
Some rows are added to the catalog_product_entity_varchar - then it seems, that 74 was chosen as a wrong id of the attribute you needed, thus foreign key in catalog_product_entity_varchar didn't allow you to insert the records.
No rows are added to the catalog_product_entity_varchar - then it seems, that you mistake in attribute id, attribute code and entity type. Recheck, what you put as ATTRIBUTE_CODE and ATTRIBUTE_ENTITY_TYPE_ID.
If both queries still hang - then you have issues with your MySQL client or server or execution scenario.
Note: your initial query may make sense in your specific case, but some issues are signalling that something may be wrong with your approach, because:
You're using direct numbers for ids. But ids are different for different installations and Magento versions. It is expected to use more stable values, like attribute code in second query, by which you should extract the actual attribute id.
You copy data from the storage catalog_product_entity_media_gallery, which can store multiple entries for the same product, to the storage catalog_product_entity_varchar, which is able to store only one entry for the product. It means, that you cannot copy all the data in such a way. Probably, your query doesn't reflect the goal you want to achieve.
The entity type id, inserted to the catalog_product_entity_varchar is not related to attribute id. While in Magento these are deeply connected things. Putting the wrong entity type id in a table will either make Magento behave incorrectly, or it won't notice your changes at all.
try this
INSERT INTO catalog_product_entity_varchar( entity_id, value)
VALUES (
SELECT entity_id, value
FROM catalog_product_entity_media_gallery
WHERE value_id = here the row id of value_id which have those values 4,74,0 )
Assuming the valued_id in the catalog_product_entity_varchar table is an autoincrement, could you not do the following?
USE magento_db;
INSERT INTO catalog_product_entity_varchar(entity_type_id, store_id, entity_id, value)
SELECT
4,
74,
catalog_product_entity_media_gallery.entity_id,
catalog_product_entity_media_gallery.value
FROM catalog_product_entity_media_gallery;
Note that there is no attribute_id column in your catalog_product_entity_varchar table.
Related
Is there a way of removing near identical values from a table in mysql? My table has records more than 10K out of which one of the company looks like this:
id name
123 Vianet Group Inc
5214 Vianet Group, Inc.
on using describe tablename I get this:
Field Type Null Key Default Extra
id int NO PRI auto_increment
name varchar(150) NO UNI
the names of the company are same however I would like to delete the second instance from table, thereby keeping just a single instance of the name in the table. This is just one instance and there are others like these.. Is there a swift way of removing identical values from the column? Please help.
You could try using soundex to find the "near identical" values -
SELECT *
FROM tablename t1
INNER JOIN tablename t2
ON t1.id < t2.id
AND SOUNDEX(t1.name) = SOUNDEX(t2.name)
You will need to test it with some of your example "near identical" values to see what it does and does not work for. As suggested by Akina you will probably need to go for some kind of normalisation process (stored function) or the Levenshtein distance function linked by Slava.
I am trying to display where a record has multiple categories though my query only appears to be showing the first instance. I need for the query to be displaying the domain multiple times for each category it appears in
The SQL statement I have is
SELECT domains.*, category.*
FROM domains,category
WHERE category.id IN (domains.category_id)
Which gives me the below results
You should not store numeric values in a string. Bad, bad idea. You should use a proper junction table and the right SQL constructs.
Sometimes, we are stuck with other people's bad design decisions. MySQL offers find_in_set() to help in this situation:
where find_in_set(category.id, domains.category_id) > 0
Use find_in_set().
SELECT domains.*, category.*
FROM domains,category
WHERE find_in_set (category.id ,domains.category_id)
But it is very bad db design to store fk as a csv.
As others have pointed out, you can use FIND_NI_SET() as a workaround to solve your problem.
My suggestion is that you refactor your code and database a bit. Storing values in CSV format stored in a single column is almost always a bad idea.
As Gordon Linoff correctly points out you'd be better off if you'd create an additional table to store the category_id values:
CREATE TABLE domain_categories (domain_id INT, category_id INT, PRIMARY KEY (domain_id, category_id));
That's assuming you want to enfore that each category is only stored once against each domain. If you don't, just drop the primary key.
You'd then insert your IDs into this new table:
INSERT INTO domain_categories (domain_id, category_id) VALUES (2,6),(2,8);
or
INSERT INTO domain_categories (domain_id, category_id) VALUES (4,3);
INSERT INTO domain_categories (domain_id, category_id) VALUES (4,11);
INSERT INTO domain_categories (domain_id, category_id) VALUES (20,3);
Now that you have properly stored the data you can easily query as needed:
SELECT domains.*,category,*
FROM domains
JOIN domain_category ON (domain_category.domain_id=domains.id)
JOIN category ON (category.id=domain_category.category_id);
Using MySQL quirks you can even show the category_id column in CSV format.
SELECT domains.*, GROUP_CONCAT(DISTINCT domain_category.category_id)
FROM domains
JOIN domain_category ON (domain_category.domain_id=domains.id)
GROUP BY domains.id;
I'm building a db to hold friendships between users of my app.
The server I use to communicate with the MySQL instance is written using Node.js (Express).
My table 'friendships' consists mainly of two INTs which correspond (foreign keys) to user ids.
I want to avoid bidirectional duplicates ( 1,2 vs. 2,1) so I need to write a query which does the following:
INSERT INTO friendships f (id_1, id_2) VALUES (?, ?) IF (SELECT * FROM friendships s WHERE s.id_1=? AND s.id_2=?) IS NULL ;
Obviously this one doesn't really work. And of course I would have the last two question marks have opposite values compared to the first ones, and a UNIQUE key on the ids (id_1, id_2).
The usual answer for these kind of questions is "just order your ids by size to avoid duplicates" and it's a good answer. But in my case, I want to keep record of who sent the friend request (and who approved), without using any extra variables (and extra queries).
Also, I don't want to use code for this, in order to avoid "concurrent" problems.
Thanks!
In MySQL, you can do this with a trigger that does the check. Some other databases have functional indexes, indexes on computed columns, or check constraints that help implement this functionality.
If you want to do the check in the insert, you can do:
INSERT INTO friendships(id_1, id_2)
select new1, new2
from (select ? as new1, ? as new2) t
where not exists (select 1
from friendships f
where f.id_1 = new2 and f.id_2 = new1
);
You should also have a unique index on id_1 and id_2:
create unique index idx_friendsships_id1_id2 on (id_1, id_2);
EDIT:
The basic query is:
INSERT INTO friendships(id_1, id_2)
select ?, ?
from dual
where not exists (select 1
from friendships f
where f.id_1 = ? and f.id_2 = ?
);
But you have to get the arguments in the right order, so the earlier method is less prone to error.
I have this Query:
INSERT INTO `items` (`id`,`image`)
VALUES(112,'asdf.jpg'),(113,'foobar.png')
ON DUPLICATE KEY UPDATE
`id`VALUES(`id`),
`image` = IF(image_uploaded = 0, VALUES(`image`),image);
The worse: its properly working but not as i want.
What i want: The images should only be updated if the field "image_uploaded" is set to 0.
Any ideas?
The Background: I have a DB-Table with data. Now each night a cronjob calls an API-Fn to get new Data from another DB and write it down to my table. This function is getting all items from the second DB so its currently just overwriting my existing data. Now my Application allows changes on data i got from the 2nd DB and overwrites the changes in my own Table. So the Problem is: I need the ability to edit Data via my App AND update Data via API without colliding. The User may change the "image" but all other things should be updated from the 2nd DB. The Image should only be overwritten if it wasn't uploaded manually.
Without playing around with ON DUPLICATE KEY... I'm not sure, if it can handle this situation. I'd work around it by using another (temporary) table.
It's the same structure as your target_table plus a column (in the example sid) to indicate if the entry exists in your target_table or not.
CREATE TEMPORARY TABLE tmp_whatever (
sid int,
id int, image varchar(50)
);
Now we insert the data you want to insert into your target_table into the newly created table first and check with coalesce() and left joining, if the entry already exists. I'm assuming here, that id is your primary key.
INSERT INTO tmp_whatever (sid, id, image)
SELECT
COALESCE(t.id, 0),
id, image
FROM (
SELECT 112 AS id,'asdf.jpg' AS image
UNION ALL
SELECT 113,'foobar.png'
) your_values v
LEFT JOIN your_target_table t ON v.id = t.id;
Then we update the target_table...
UPDATE your_target_table t INNER JOIN tmp_whatever w ON t.id = w.id AND w.sid <> 0
SET t.image = w.image
WHERE t.image_uploaded = 0;
And finally we insert the rows not already existing...
INSERT INTO your_target_table (id, image)
SELECT
id, image
FROM tmp_whatever
WHERE sid = 0;
While I was writing this down, it came to my mind, that I might have had wrong assumptions on what your problem is. This
The worse: its properly working but not as i want.
is definitely not the way to ask a question or describe a problem. I answered because I have a good day :) (and it's the reason why you get downvotes, btw)
Anyway, another cause of "not as i want" could be, that you're missing a unique index in your table. Though a primary key is a unique key, afaik/iirc ON DUPLICATE KEY relies on a unique index on another column to work correctly.
I'm using MySQL 4.1. Some tables have duplicates entries that go against the constraints.
When I try to group rows, MySQL doesn't recognise the rows as being similar.
Example:
Table A has a column "Name" with the Unique proprety.
The table contains one row with the name 'Hach?' and one row with the same name but a square at the end instead of the '?' (which I can't reproduce in this textfield)
A "Group by" on these 2 rows return 2 separate rows
This cause several problems including the fact that I can't export and reimport the database. On reimporting an error mentions that a Insert has failed because it violates a constraint.
In theory I could try to import, wait for the first error, fix the import script and the original DB, and repeat. In pratice, that would take forever.
Is there a way to list all the anomalies or force the database to recheck constraints (and list all the values/rows that go against them) ?
I can supply the .MYD file if it can be helpful.
To list all the anomalies:
SELECT name, count(*) FROM TableA GROUP BY name HAVING count(*) > 1;
There are a few ways to tackle deleting the dups and your path will depend heavily on the number of dups you have.
See this SO question for ways of removing those from your table.
Here is the solution I provided there:
-- Setup for example
create table people (fname varchar(10), lname varchar(10));
insert into people values ('Bob', 'Newhart');
insert into people values ('Bob', 'Newhart');
insert into people values ('Bill', 'Cosby');
insert into people values ('Jim', 'Gaffigan');
insert into people values ('Jim', 'Gaffigan');
insert into people values ('Adam', 'Sandler');
-- Show table with duplicates
select * from people;
-- Create table with one version of each duplicate record
create table dups as
select distinct fname, lname, count(*)
from people group by fname, lname
having count(*) > 1;
-- Delete all matching duplicate records
delete people from people inner join dups
on people.fname = dups.fname AND
people.lname = dups.lname;
-- Insert single record of each dup back into table
insert into people select fname, lname from dups;
-- Show Fixed table
select * from people;
Create a new table, select all rows and group by the unique key (in the example column name) and insert in the new table.
To find out what is that character, do the following query:
SELECT HEX(Name) FROM TableName WHERE Name LIKE 'Hach%'
You will se the ascii code of that 'square'.
If that character is 'x', you could update like this:(but if that column is Unique you will have some errors)
UPDATE TableName SET Name=TRIM(TRAILING 'x' FROM Name);
I'll assume this is a MySQL 4.1 random bug. Somes values are just changing on their own for no particular reason even if they violates some MySQL constraints. MySQL is simply ignoring those violations.
To solve my problem, I will write a prog that tries to resinsert every line of data in the same table (to be precise : another table with the same caracteristics) and log every instance of failures.
I will leave the incident open for a while in case someone gets the same problem and someone else finds a more practical solution.