Insert non-repeating data into table - mysql

I've got a file that I've succesfully updated to a table. Now, I'm required to do it again in a way that the data is not being repeated.
my target table:
mysql> show columns from MARCAS;
+----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+----------------+
| ID_MARCA | int(11) | NO | PRI | NULL | auto_increment |
| MARCA | varchar(50) | YES | | NULL | |
+----------+-------------+------+-----+---------+----------------+
So what I did was to create a temporary table and loaded data from the file into it.
code used:
CREATE TEMPORARY TABLE tempe LIKE marcas;
LOAD DATA LOCAL INFILE myfile.txt
INTO TABLE MARCAS
FIELDS TERMINATED BY '#' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(ID_MARCA, MARCA, #ignore1, #ignore2, #ignore3);
All fine here. The problem starts when I am trying to be smart and use 'ON DUPLICATE KEY'(like it was sugested here) to insert data from temporary table into table MARCAS as so data does not repeat itself. The code I used was:
INSERT into MARCAS select * from TEMPE on duplicate key update MARCA=values(MARCA);
The result:
mysql> select * from MARCAS;
+----------+---------+
| ID_MARCA | MARCA |
+----------+---------+
| 1 | PICASSA |
| 2 | PICASSA |
| 3 | C4 |
| 4 | C4 |
| 5 | C3 |
| 6 | C3 |
| 7 | C2 |
| 8 | C2 |
| 9 | MONDEO |
| 10 | MONDEO |
| 11 | S-MAX |
| 12 | S-MAX |
| 13 | CIVIC |
| 14 | CIVIC |
| 15 | ACCORD |
| 16 | ACCORD |
| 17 | CLS |
| 18 | 900 |
| 19 | LEON |
| 20 | LEON |
| 21 | IBIZA |
| 22 | IBIZA |
| 23 | 307 |
| 24 | 307 |
| 25 | 308 |
| 26 | 308 |
| 27 | 407 |
| 28 | 407 |
| 29 | 408 |
| 30 | 408 |
| 31 | MEGANE |
| 32 | MEGANE |
| 33 | PASSAT |
| 34 | PASSAT |
| 35 | GOLF |
| 36 | GOLF |
| 37 | TOUAREG |
+----------+---------+
Not impressed with myself. What am I doing wrong?

It's caused by the fact that you don't have a UNIQUE index on your MARCA column.
alter table MARCAS add unique(MARCA);
If you specify ON DUPLICATE KEY UPDATE, and a row is inserted that
would cause a duplicate value in a UNIQUE index or PRIMARY KEY, MySQL
performs an UPDATE of the old row.
source: http://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html
Please also note that it's also possible to do INSERT IGNORE or A REPLACE when you do a load data.

May you get some idea from this
INSERT INTO MARCAS(field1)
WHERE NOT EXISTS (
SELECT field1 FROM MARCAS WHERE name = 'field1'
) LIMIT 1;

Related

Complex INSERT or UPDATE MariaDB tables with data from other MariaDB tables using JOIN or UNION

I need to INSERT or UPDATE data in a table using data from other tables; I understand the basic
insert into table (a,b,c)
select h, i, j
from otherTable
where........
My challenge comes from the fact that the data is spread across multiple tables and in one of the tables the data is metadata stored in rows, not columns. Therefore I need to use JOIN and possible UNION to get what is needed.
Unfortunately after trying everything I read in both the Maria manual, on the Maria forum and on Stack overflow I can not get it to work.
Here is what I am attempting to do:
insert data into dbc_jot_groupmembers in the following fields using source data as shown:
jot_grpid = dbc_bp_groups_members.group_id
jot_bbmemid = dbc_bp_groups_members.user_id
jot_grpmemname = dbc_bp_xprofile_data.value where field_id=3
jot_grpmemnum = dbc_bp_xprofile_data.value where field_id=4
I need the final result to look like this:
select * from dbc_jot_groupmembers;
+--------------+-----------+----------------+---------------+---------------------+-------------+
| jot_grpmemid | jot_grpid | jot_grpmemname | jot_grpmemnum | jot_grpmemts | jot_bbmemid |
+--------------+-----------+----------------+---------------+---------------------+-------------+
| 1 | 17 | hutchdad | +17047047045 | 2021-06-15 14:56:19 | 14 |
| 2 | 24 | hutchdad | +17047047045 | 2021-06-15 19:49:58 | 14 |
| 3 | 25 | hutchdad | +17047047045 | 2021-06-15 19:49:58 | 14 |
| 4 | 17 | hutchmom | +17773274355 | 2021-06-15 19:49:58 | 15 |
| 5 | 24 | hutchmom | +17773274355 | 2021-06-15 19:49:58 | 15 |
| 6 | 16 | ledwards | +14567655645 | 2021-06-15 19:49:58 | 11 |
| 7 | 16 | medwards | +12223334545 | 2021-06-15 19:49:58 | 10 |
| 7 | 20 | medwards | +12223334545 | 2021-06-15 19:49:58 | 10 |
SAMPLE DATA FROM SOURCE TABELS AND TABLE DEFINITIONS:
MariaDB [devDisciplePlaceCom]> describe dbc_bp_groups_members;
+---------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| group_id | bigint(20) | NO | MUL | NULL | |
| user_id | bigint(20) | NO | MUL | NULL | |
| inviter_id | bigint(20) | NO | MUL | NULL | |
| is_admin | tinyint(1) | NO | MUL | 0 | |
| is_mod | tinyint(1) | NO | MUL | 0 | |
| user_title | varchar(100) | NO | | NULL | |
| date_modified | datetime | NO | | NULL | |
| comments | longtext | NO | | NULL | |
| is_confirmed | tinyint(1) | NO | MUL | 0 | |
| is_banned | tinyint(1) | NO | | 0 | |
| invite_sent | tinyint(1) | NO | | 0 | |
+---------------+--------------+------+-----+---------+----------------+
12 rows in set (0.002 sec)
describe dbc_bp_xprofile_data;
+--------------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+---------------------+------+-----+---------+----------------+
| id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| field_id | bigint(20) unsigned | NO | MUL | NULL | |
| user_id | bigint(20) unsigned | NO | MUL | NULL | |
| value | longtext | NO | | NULL | |
| last_updated | datetime | NO | | NULL | |
+--------------+---------------------+------+-----+---------+----------------+
5 rows in set (0.001 sec)
THIS IS THE LIST OF GROUPS AND WHAT USERS THEY ARE IN.
select group_id,user_id from dbc_bp_groups_members ;
+----------+---------+
| group_id | user_id |
+----------+---------+
| 16 | 13 |
| 16 | 12 |
| 16 | 11 |
| 16 | 10 |
| 17 | 14 |
| 17 | 15 |
| 17 | 16 |
| 17 | 17 |
| 17 | 18 |
| 17 | 19 |
| 20 | 10 |
| 24 | 14 |
| 24 | 16 |
| 24 | 15 |
| 24 | 17 |
| 24 | 19 |
| 25 | 19 |
| 25 | 14 |
| 1 | 14 |
| 11 | 14 |
+----------+---------+
20 rows in set (0.000 sec)
THIS IS THE TABLE CONTAINING THE USERS METADATA. IN MY CASE I NEED THE PHOEN NUMBER AND NAME WHICH ARE IN THE value FIELD WITH A field_id of 3 and 4.
select * from dbc_bp_xprofile_data where user_id > 9 and field_id > 2 AND field_id < 5;
+-----+----------+---------+---------------+---------------------+
| id | field_id | user_id | value | last_updated |
+-----+----------+---------+---------------+---------------------+
| 31 | 3 | 10 | medwards | 2021-06-24 03:11:59 |
| 34 | 3 | 11 | ledwards | 2021-06-24 03:11:24 |
| 37 | 3 | 12 | nedwards | 2021-04-24 14:47:18 |
| 40 | 3 | 13 | iedwards | 2021-04-24 14:47:52 |
| 43 | 3 | 14 | hutchdad | 2021-06-21 14:53:08 |
| 46 | 3 | 15 | hutchmom | 2021-06-24 03:10:58 |
| 49 | 3 | 16 | hutchdaughter | 2021-04-24 16:54:48 |
| 52 | 3 | 17 | hutchson1 | 2021-04-24 16:55:43 |
| 55 | 3 | 18 | hutchson2 | 2021-04-24 16:57:42 |
| 58 | 3 | 19 | hutchson3 | 2021-04-24 16:58:44 |
| 78 | 3 | 25 | demoadmin | 2021-06-08 02:01:39 |
| 158 | 4 | 14 | 7047047045 | 2021-06-21 14:53:08 |
| 190 | 3 | 58 | dupdup | 2021-06-23 19:46:19 |
| 191 | 4 | 15 | 7773274355 | 2021-06-24 03:10:58 |
| 193 | 4 | 11 | 4567655645 | 2021-06-24 03:11:24 |
| 195 | 4 | 10 | 2223334545 | 2021-06-24 03:11:59 |
+-----+----------+---------+---------------+---------------------+
16 rows in set (0.000 sec)
If this can not be done is a single INSERT then I can use an INSERT with subsequent UPDATE statements. I also understand that this is not best practice and violates 3nf and probably several other best practice principles. Unfortunately, I am at the mercy of the application and can not change the code, so the only way to get this to work is to put duplicate data in the database as described below:
It can be done with a single INSERT. However, there are some information need to be addressed as what I've posted in a the comment. In the meantime, here is an example query that you can use to do the operation that you want:
SELECT ROW_NUMBER() OVER (ORDER BY A.group_id, A.user_id) AS 'jot_grpmemid',
A.group_id AS 'jot_grpid',
MAX(CASE WHEN B.field_id=3 THEN B.value ELSE '' END) AS 'jot_grpmemname',
MAX(CASE WHEN B.field_id=4 THEN CONCAT('+',B.value) ELSE '' END) AS 'jot_grpmemnum',
A.user_id AS 'jot_bbmemid'
FROM
dbc_bp_groups_members A JOIN dbc_bp_xprofile_data B
ON A.user_id=B.user_id
GROUP BY A.group_id, A.user_id;
Like I said in the comment, I'm not sure how you get/generate jot_grpmemid because you have two 7 in the expected result so I assume it's a typo. I guess, at this point it's up to you to modify the query accordingly.
Demo fiddle

INNER join taking way too long

I asked the original question here in stack before. Apologies if this is not the best way to go about this.
The problem is I have a query that even with a INNER JOIN is taking at least 5 seconds to complete and I'm wondering if there is a faster way to do this. Here is the answer I was given:
` q = "SELECT DISTINCT e2.eventId FROM event_tags e1 INNER JOIN event_tags e2 " \
"ON BINARY e2.tagName=e1.tagName AND e2.eventId != e1.eventId " \
"WHERE e1.eventId = {} ORDER BY RAND() LIMIT {}".format(eventId, '10')`
my tags table looks like this
mysql> describe event_tags;
+---------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------------+------+-----+---------+----------------+
| tagId | int(10) unsigned | NO | PRI | NULL | auto_increment |
| tagName | text | NO | | NULL | |
| eventId | int(10) unsigned | NO | PRI | NULL | |
+---------+------------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)
and I have a bunch of tags in them and they will only continue to grow. When I do a count on the tags table I have 504,402 tagId, and same for tagname. How could I make the look up faster?
Here is some sample data of the event tags table
mysql> select * from event_tags limit 40;
+-------+-------------------------------------------+---------+
| tagId | tagName | eventId |
+-------+-------------------------------------------+---------+
| 261 | Justin Timberlake (Rescheduled from 11/9) | 38 |
| 264 | Rogers Arena | 38 |
| 267 | Pop | 38 |
| 271 | Rock | 38 |
| 285 | Justin Timberlake (Rescheduled from 11/8) | 41 |
| 288 | Rogers Arena | 41 |
| 291 | Pop | 41 |
| 294 | Rock | 41 |
| 595 | Yogesh Soman | 84 |
| 599 | Geetanjali Kulkarni | 84 |
| 602 | Bhagyashree Shankpal | 84 |
| 606 | Lalit Prabhakar | 84 |
| 611 | Sameer Sanjay Vidwans | 84 |
| 617 | Drama | 84 |
| 647 | Shrihari Abhyankar | 89 |
| 651 | Deepali Borkar | 89 |
| 654 | Akash Kamble | 89 |
| 657 | Sharavi Kulkarni | 89 |
| 660 | Sharav Wadhawekar | 89 |
| 667 | Nipun Dharmadhikari | 89 |
| 670 | Drama | 89 |
| 689 | Frank Grillo | 94 |
| 692 | Jamie Bell | 94 |
| 695 | Margaret Qualley | 94 |
| 700 | James Badge Dale | 94 |
| 704 | Tim Sutton | 94 |
| 710 | Drama | 94 |
| 734 | Bruce Dern | 101 |
| 739 | Anthony Michael Hall | 101 |
| 745 | Sean Astin | 101 |
| 749 | Aly Michalka | 101 |
| 754 | Victoria Smurfit | 101 |
| 759 | Carl Bessai | 101 |
| 762 | Drama | 101 |
| 783 | Sarah Clarke | 106 |
| 785 | Xander Berkeley | 106 |
| 787 | Kristen Gutoskie | 106 |
| 790 | Mackenzie Astin | 106 |
| 794 | Bobby Campo | 106 |
| 798 | Adam Cushman | 106 |
+-------+-------------------------------------------+---------+
40 rows in set (0.00 sec)
and here is the CREATE statement for the table:
CREATE TABLE IF NOT EXISTS event_tags(
tagId INT UNSIGNED NOT NULL AUTO_INCREMENT,
tagName VARCHAR(40) NOT NULL,
eventId INT UNSIGNED NOT NULL,
PRIMARY KEY(tagId, eventId)
);
Here is the EXPLAIN for the query:
mysql> EXPLAIN SELECT DISTINCT e2.eventId FROM event_tags e1 INNER JOIN event_tags e2 ON BINARY e2.tagName=e1.tagName AND e2.eventId != e1.eventId WHERE e1.eventId = 487 ORDER BY RAND() LIMIT 10
-> ;
+----+-------------+-------+------+---------------+------+---------+------+-------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+-------+----------------------------------------------+
| 1 | SIMPLE | e1 | ALL | NULL | NULL | NULL | NULL | 34275 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | e2 | ALL | NULL | NULL | NULL | NULL | 34275 | Using where; Using join buffer |
+----+-------------+-------+------+---------------+------+---------+------+-------+----------------------------------------------+
2 rows in set (0.03 sec)
UPDATE: i created an index on the table with:
CREATE INDEX tagsNdx ON event_tags (eventId, tagName(255));
Which looks now like this:
mysql> show index from event_tags;
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| event_tags | 0 | PRIMARY | 1 | tagId | A | 455408 | NULL | NULL | | BTREE | | |
| event_tags | 0 | PRIMARY | 2 | eventId | A | 455408 | NULL | NULL | | BTREE | | |
| event_tags | 1 | tagsNdx | 1 | eventId | A | 186 | NULL | NULL | | BTREE | | |
| event_tags | 1 | tagsNdx | 2 | tagName | A | 186 | 255 | NULL | | BTREE | | |
+------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
4 rows in set (0.00 sec)
But it's still slow.
Following is possible optimization:
Remove column "eventId" from primary key (this step is optional an can be further elaborated if you like).
Create index on column (eventId,tag_name).
Execute Command: ANALYZE TABLE event_tags

MySQL Simple query took to long to return result

My simple query took too long to return result.
Here is my query:
select count(*) from f_logs as a
where a.id = (
select max(id)
from f_logs
where f_id = a.f_id
group by f_id
) and log_status = 'In storage';
This is my sample data from my table:
+----+-------+------------+------------+-----------------+-------------+----------------+-------------+
| id | f_id | log_date | log_action | log_destination | log_remarks | log_status | log_account |
+----+-------+------------+------------+-----------------+-------------+----------------+-------------+
| 1 | 1-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 2 | 2-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 3 | 3-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 4 | 4-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 5 | 5-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 6 | 1-EC | 2017-05-03 | Released | Treasury | | Not in Storage | Person A |
| 7 | 7-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 8 | 8-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 9 | 2-EC | 2017-05-03 | Released | Registrar | | Not in Storage | Person A |
| 10 | 10-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 11 | 11-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 12 | 12-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 13 | 3-EC | 2017-05-03 | Released | Registrar | | Not in Storage | Person B |
| 14 | 14-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 15 | 15-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 16 | 16-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 17 | 17-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 18 | 18-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 19 | 19-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
| 20 | 1-EC | 2017-05-03 | Receive | Records Unit | | In storage | Records |
+----+-------+------------+------------+-----------------+-------------+----------------+-------------+
Using explain returns the following:
+----+--------------------+--------+------------+------+---------------+------+---------+------+-------+----------+----------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+--------+------------+------+---------------+------+---------+------+-------+----------+----------------------------------------------+
| 1 | PRIMARY | a | NULL | ALL | NULL | NULL | NULL | NULL | 46439 | 10.00 | Using where |
| 2 | DEPENDENT SUBQUERY | f_logs | NULL | ALL | NULL | NULL | NULL | NULL | 46439 | 10.00 | Using where; Using temporary; Using filesort |
+----+--------------------+--------+------------+------+---------------+------+---------+------+-------+----------+----------------------------------------------+
2 rows in set, 2 warnings (0.00 sec)
Note (Code 1276): Field or reference 'gsis_new.a.f_id' of SELECT #2 was resolved in SELECT #1
Note (Code 1003):
/* select#1 */
select count(0) AS count(*)
from gsis_new.f_logs a
where ((gsis_new.a.id = (
/* select#2 */
select max(gsis_new.f_logs.id)
from gsis_new.f_logs
where (gsis_new.f_logs.f_id = gsis_new.a.f_id)
group by gsis_new.f_logs.f_id)
) and (gsis_new.a.log_status = 'In storage'))
Here is what i want to happen:
I have a table named f_logs, it is used for logging of files.
I want to return the total count of records in the table.
From my sample table data. It should return '14' since 2-EC and 3-EC is marked as not in storage from row 9 and 13.
While 1-EC is already mark as In storage again in row 20.
Note: My mySql query is working fine when the table is not populated.
Is there any chance to optimize my query?
Additional Information:
Here is the index of my f_logs table
+--------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| f_logs | 0 | PRIMARY | 1 | id | A | 46439 | NULL | NULL | | BTREE | | |
| f_logs | 1 | acountIndex | 1 | log_account | A | 1 | NULL | NULL | YES | BTREE | | |
+--------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Your explain shows there might be no indices on the table.
ALTER TABLE f_logs ADD INDEX (id), ADD INDEX (f_id);

SET in LOAD DATA INFILE

OK, I thought I can figure it out by myself but I just don't seem to see it. The goal for me is to fill table MODELOS in following manner:
+--------+----------+---------+
| ID_MOD | ID_MARCA | MODELO |
+--------+----------+---------+
| 1 | 1 | C2 |
| 2 | 1 | C3 |
| 3 | 1 | C4 |
| 4 | 1 | PICASSA |
| 5 | 2 | MONDEO |
| 6 | 2 | S-MAX |
| 7 | 3 | ACCORD |
| 8 | 3 | CIVIC |
| 9 | 4 | CLS |
| 10 | 5 | 900 |
| 11 | 6 | IBIZA |
| 12 | 6 | LEON |
| 13 | 7 | 307 |
| 14 | 7 | 308 |
| 15 | 7 | 407 |
| 16 | 7 | 408 |
| 17 | 8 | MEGANE |
| 18 | 9 | GOLF |
| 19 | 9 | PASSAT |
| 20 | 9 | TOUAREG |
+--------+----------+---------+
I've got this table:
mysql> select * from MARCAS;
+----------+------------+
| ID_MARCA | MARCA |
+----------+------------+
| 1 | CITROEN |
| 2 | FORD |
| 3 | HONDA |
| 4 | MERCEDES |
| 7 | PEUGEOT |
| 8 | RENAULT |
| 5 | SAAB |
| 6 | SEAT |
| 9 | VOLKSWAGEN |
+----------+------------+
and following file to work with:
"MARCA"#"MODELO"#"MATRICULA"#PRECIO
"CITROEN"#"PICASSA"#"CPG-2044"#12000
"CITROEN"#"PICASSA"#"CPR-1762"#12500
"CITROEN"#"C4"#"FPP-1464"#13500
"CITROEN"#"C4"#"FDR-4563"#13000
"CITROEN"#"C3"#"BDF-8856"#8000
"CITROEN"#"C3"#"BPZ-7878"#7500
"CITROEN"#"C2"#"CDR-1515"#5000
"CITROEN"#"C2"#"BCC-3434"#4500
"FORD"#"MONDEO"#"BTG-3267"#8000
"FORD"#"MONDEO"#"BPP-6792"#8200
"FORD"#"S-MAX"#"FDR-1564"#20000
"FORD"#"S-MAX"#"FCE-9327"#21000
"HONDA"#"CIVIC"#"FCC-7764"#16000
"HONDA"#"CIVIC"#"FBC-4567"#14000
"HONDA"#"ACCORD"#"FFC-6768"#22000
"HONDA"#"ACCORD"#"FPB-2231"#23000
"MERCEDES"#"CLS"#"FDR-2265"#31000
"SAAB"#"900"#"FPG-1165"#21000
"SEAT"#"LEON"#"DVB-1119"#14500
"SEAT"#"LEON"#"DCR-5634"#13500
"SEAT"#"IBIZA"#"DPR-3434"#9500
"SEAT"#"IBIZA"#"DPP-8756"#10000
"PEUGEOT"#"307"#"DGX-4598"#5500
etc.
Now, what I've achieved is:
mysql> select * from MODELOS;
+--------+----------+---------+
| ID_MOD | ID_MARCA | MODELO |
+--------+----------+---------+
| 1 | NULL | MODELO |
| 2 | NULL | PICASSA |
| 3 | NULL | C4 |
| 4 | NULL | C3 |
| 5 | NULL | C2 |
| 6 | NULL | MONDEO |
| 7 | NULL | S-MAX |
| 8 | NULL | CIVIC |
| 9 | NULL | ACCORD |
| 10 | NULL | CLS |
| 11 | NULL | 900 |
| 12 | NULL | LEON |
| 13 | NULL | IBIZA |
| 14 | NULL | 307 |
| 15 | NULL | 308 |
| 16 | NULL | 407 |
| 17 | NULL | 408 |
| 18 | NULL | MEGANE |
| 19 | NULL | PASSAT |
| 20 | NULL | GOLF |
| 21 | NULL | TOUAREG |
+--------+----------+---------+
Using, these commands:
ALTER TABLE MODELOS ADD UNIQUE(MODELO);
LOAD DATA LOCAL INFILE myfile.txt
IGNORE INTO TABLE MODELOS
FIELDS TERMINATED BY '#' ENCLOSED BY '"'
LINES ENDED BY '\n'
IGNORE 1 LINES
(#ignore1, MODELO, #ignore2, #ignore3);
How can I get that to create desired output? I suspect that the easiest way to do it will be by using SET sentence in LOAD DATA INFILE, but I just don't know how?
You can put a query in the SET clause.
LOAD DATA LOCAL INFILE myfile.txt
IGNORE INTO TABLE MODELOS
FIELDS TERMINATED BY '#' ENCLOSED BY '"'
LINES ENDED BY '\n'
IGNORE 1 LINES
(#marca, modelo, #ignore2, #ignore3)
SET ID_MARCA = (SELECT ID_MARCA FROM MARCAS WHERE MARCA = #marca)
This uses the first column in the file to look up the corresponding ID in the MARCAS table.

Multiple table join with possible conditional matches

I have several tables that I combine in an application I'm creating in PHP that essentially creates a check list. I realize I could solve this problem using a conditional in PHP, but am curious if MySQL is capable of accomplishing this and if so, how? Specifically, I have four tables which are queried using the following statement:
SELECT
cl_status.status,
users.user_first,
cl_status.date AS status_date,
cl_status.id AS status_id,
cl_status.criteria_id,
cl_criteria.id AS cid,
cl_criteria.description AS description
FROM cl_criteria
LEFT JOIN cl_lists
ON cl_criteria.cl_id = cl_lists.id
RIGHT JOIN cl_status
ON cl_criteria.id = cl_status.criteria_id
LEFT JOIN users
ON cl_status.user_id = users.user_id
WHERE cl_lists.id = '1'
Table one - cl_lists:
+----+------------------+------------+------------+-------+
| id | title | date | comp_level | owner |
+----+------------------+------------+------------+-------+
| 1 | Newcomer's guide | 1452473606 | 1 | 1 |
+----+------------------+------------+------------+-------+
Table two - cl_assign:
+----+-------+-------+------------+
| id | cl_id | owner | date |
+----+-------+-------+------------+
| 1 | 1 | 1 | 1455843514 |
+----+-------+-------+------------+
Table three - cl_status:
+----+-------------+---------+-------------+------------+--------+----------+
| id | criteria_id | user_id | description | date | status | comments |
+----+-------------+---------+-------------+------------+--------+----------+
| 2 | 66 | 1 | | NULL | 1 | NULL |
| 15 | 65 | 1 | | 1455842197 | 5 | NULL |
| 16 | 67 | 1 | | 1455842201 | 5 | NULL |
| 17 | 68 | 1 | | 1455842203 | 5 | NULL |
| 18 | 69 | 1 | | 1455842217 | 0 | NULL |
| 19 | 70 | 1 | | 1455842222 | 5 | NULL |
| 20 | 72 | 1 | | 1455842237 | 1 | NULL |
| 21 | 71 | 1 | | 1455842234 | 0 | NULL |
| 22 | 73 | 1 | | 1455842246 | 5 | NULL |
| 23 | 76 | 1 | | 1455842249 | 5 | NULL |
| 24 | 77 | 1 | | 1455842268 | 5 | NULL |
| 25 | 78 | 152 | | 1455854420 | 3 | NULL |
| 26 | 81 | 1 | | 1455843660 | 5 | NULL |
+----+-------------+---------+-------------+------------+--------+----------+
Table four - users:
+---------+------------+
| user_id | user_first |
+---------+------------+
| 1 | Mark |
| 2 | Test |
+---------+------------+
Ideally, I'd like the join to look like this:
+--------+------------+-------------+-----------+-------------+------+-----------------------------+
| status | user_first | status_date | status_id | criteria_id | cid | description |
+--------+------------+-------------+-----------+-------------+------+-----------------------------+
| 5 | Mark | 1455842197 | 15 | 65 | 65 | Tour of facility |
| 5 | Mark | 1455842201 | 16 | 67 | 67 | Tax forms |
| 5 | Mark | 1455842203 | 17 | 68 | 68 | 2 forms of ID |
| 0 | Mark | 1455842217 | 18 | 69 | 69 | Benefits | |
| 5 | Mark | 1455842246 | 22 | 73 | 73 | Intro to policies |
| 5 | Mark | 1455842249 | 23 | 76 | 76 | Setup email account |
| NULL | NULL | NULL | NULL | 78 | 78 | Setup Computer account |
+--------+------------+-------------+-----------+-------------+------+-----------------------------+
However, it looks like this:
+--------+------------+-------------+-----------+-------------+------+-----------------------------+
| status | user_first | status_date | status_id | criteria_id | cid | description |
+--------+------------+-------------+-----------+-------------+------+-----------------------------+
| 5 | Mark | 1455842197 | 15 | 65 | 65 | Tour of facility |
| 5 | Mark | 1455842201 | 16 | 67 | 67 | Tax forms |
| 5 | Mark | 1455842203 | 17 | 68 | 68 | 2 forms of ID |
| 0 | Mark | 1455842217 | 18 | 69 | 69 | Benefits |
| 5 | Mark | 1455842246 | 22 | 73 | 73 | Intro to policies |
| 5 | Mark | 1455842249 | 23 | 76 | 76 | Setup email account |
| 3 | Temp | 1455854420 | 25 | 78 | 78 | Setup Computer account |
+--------+------------+-------------+-----------+-------------+------+-----------------------------+
Is there a way to apply the conditional before the join? Or another way to accomplish the result set that I want?
EDIT
This is a screenshot of what the application looks like:
The criteria table will include steps of every checklist I have. The list table is a list of the various checklists. The status table allows every user (such as Mark, or Test) to look at the same checklist and complete it as if it was a separate document. It also populates the date/time that the item was updated by that user.
I suspect that the RIGHT JOIN you have in your query is causing the records you want to appear to be filtered out. Remember that t1 RIGHT JOIN t2 is the same as t2 LEFT JOIN t1, meaning that t1 will lose any record which does not appear in t2, with t2 keeping all its records. Try this:
SELECT cl_status.status, users.user_first, cl_status.date AS status_date,
cl_status.id AS status_id, cl_status.criteria_id, cl_criteria.id AS cid,
cl_criteria.description AS description
FROM cl_criteria LEFT JOIN cl_lists
ON cl_criteria.cl_id = cl_lists.id
LEFT JOIN cl_status
ON cl_criteria.id = cl_status.criteria_id
LEFT JOIN users
ON cl_status.user_id = users.user_id
WHERE cl_lists.id = '1'