I imported a CSV file using LOAD DATA INFILE and have this column called Context.
The entire column appears empty - which it shouldn't be, because I already set the DEFAULT to be NULL. So I expected NULL to appear in the entire column.
I have tried if it is empty or NULL but the code below returns 0
SELECT COUNT(*) FROM table
WHERE Context IS NULL OR Context = '';
So now I do not know where the problem is.
FYI: This column is in TEXT datatype and I could change it to VARCHAR, but NOT INT - I don't know if it's related to the problem.
You can manipulate the empty values while importing the data using the SET col_name={expr | DEFAULT}, ... statement while importing the data combined with the function NULLIF(expr1,expr2).
So your query may looks like this:
LOAD DATA INFILE 'file.txt'
INTO table t1
(column1, #var1)
SET Context = NULLIF(#var1, '');
Or you can manipulate all records after the import
UPDATE `table` SET Context2 = NULL WHERE Context2 IS NULL;
LOAD DATA
NULLIF
Based on this thread, I modified my query to the following and was able to solve the problem.
UPDATE table SET Context = NULLIF(Context,' ');
UPDATE table SET Context = NULLIF(Context, '\t');
UPDATE table SET Context = NULLIF(Context, '\n');
UPDATE table SET Context = NULLIF(Context, '\r');
So the original problem could be caused by either \t , \n or \r.
Related
I have a very large database and for testing, I want to set a certain amount of data to NULL.
As an example, I have 57 columns across 3 tables, all of which need to be nullified. I can't delete the rows, I just need to know that if the row exists and there's no data in those fields, that everything still works.
To clarify, all the data in those fields has been moved to anther table, and the old data was not wiped in the migration. To test my reports I need to know that the reports are pulling from the new location, not the old, since as new data is added, it will only go to the new location. Our plan is to generate each report from the old database, migrate, and then generate them again and compare. But to ensure that they are pulling from the right place, we want to wipe the old data so it doesn't provide a false positive.
Is there a way for me to do this in bulk or should I resign myself to writing one comma separated SET statement after another?
You can create the statements using the data from the internal information_schema.COLUMNS table.
Assuming you have this table:
CREATE TABLE my_table (
keep1 INT,
keep2 INT,
set_null1 INT,
set_null2 INT,
set_null3 INT
);
and you want to set all columns to NULL except of keep1 and keep2. Execute the following script:
set #db_name = 'test';
set #table_name = 'my_table';
set #exclude_columns = 'keep1,keep2';
select concat(
'UPDATE `', #table_name, '` SET\n',
group_concat('`', COLUMN_NAME, '` = NULL' separator ',\n'),
';'
)
from information_schema.COLUMNS c
where c.TABLE_SCHEMA = #db_name
and c.TABLE_NAME = #table_name
and find_in_set(c.COLUMN_NAME, #exclude_columns) = 0;
This will generate the following statement:
UPDATE `my_table` SET
`set_null1` = NULL,
`set_null2` = NULL,
`set_null3` = NULL;
Copy the result and paste it into your UPDATE script. Do it for all 12 tables adjusting the variables #db_name, #table_name and #exclude_columns.
See demo on db-fiddle.
This is a very unusual task for an SQL database, so it's not surprising that it's a bit awkward.
As you know, to set multiple columns to NULL in an UPDATE statement, you'd have to set each column individually.
UPDATE mytable
SET col1 = NULL, col2 = NULL, ... col57 = NULL
WHERE id = ?;
That could be quite a bit of typing. Or it could be a task to write code to loop over the column names in your table, and concatenate the terms for UPDATE statement. Up to you.
An alternative that might be easier is to delete the row and then re-insert it with no values specified except the primary key.
DELETE FROM mytable WHERE id = ?;
INSERT INTO mytable SET id = ?;
By omitting the other columns, they'll be NULL or else take a DEFAULT value defined in your table. If you want those columns with defaults to be NULL too, you'll have to specify that.
INSERT INTO mytable SET id = ?, col23 = NULL;
I'm trying to import a XML file into a MySQL Table. In the XML file there is a timestamp in <CurrentTime> in the following format:
2016-01-26T09:52:19.3420655+01:00
This timstamp should go into the corresponding DATETIME CurrentTime column in my Table. So I did the following
LOAD XML INFILE 'xxx.xml'
INTO TABLE test.events
ROWS IDENTIFIED BY '<Event>'
SET CurrentTime = str_to_date(CurrentTime, '%Y-%m-%dT%H:%i:%s.%f');
But it quits with the error
Error Code: 1292. Incorrect datetime value: '2016-01-25T16:22:24.1840792+01:00' for column 'CurrentTime' at row 1
So it seems it doesn't convert the string at all. Why?
I think that error is thrown when the string value from the file is loaded directly to the column. The error is thrown before you get to the SET clause.
Here's an abbreviated example of how to use user-defined variables to pass the value of a field down to the SET, bypassing the assignment to the column.
Note that the columns _row and account_number are populated directly from the first two fields in the file. The later fields in the file are assigned to user-defined variables (identifiers beginning with #.
The SET clause evaluates the user-defined variables, and assigns the result of the expression to the actual column in the table.
In this example, the "dates" were formatted YYYYMMDD. I used the STR_TO_DATE() function to have that string converted to a proper DATE.
I abbreviated this sample somewhat, but it demonstrates the approach of reading field values into user-defined variables.
CREATE TABLE _import_water
(`_row` INT
,`account_number` VARCHAR(255)
,`total_due` DECIMAL(18,2)
,`end_date` DATE
,`start_date` DATE
,`ccf` DECIMAL(18,4)
)
LOAD DATA LOCAL INFILE '//server/share$/users/me/mydir/myfile.csv'
INTO TABLE _import_water
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(_row
,account_number
,#total_due
,#end_date
,#start_date
,#ccf
)
SET `total_due` = NULLIF(#total_due,'')
, `end_date` = STR_TO_DATE(#end_date,'%Y%m%d')
, `start_date` = STR_TO_DATE(#start_date,'%Y%m%d')
, `ccf` = NULLIF(#ccf,'')
Also, it doesn't look like there's any problem with your STR_TO_DATE, it seems to evaluate just fine.
testing...
SELECT STR_TO_DATE('2016-01-25T16:22:24.1840792+01:00','%Y-%m-%dT%H:%i:%s.%f') AS mydatetime
returns:
mydatetime
--------------------------
2016-01-25 16:22:24.184079
Alright, I have multiple MySQL statements that lead into an issue I'm having updating a particular table. First let me show you my code, then I'll explain what I'm trying to do:
/*STEP 1 - create a temporary table to temporarily store the loaded csv*/
CREATE TEMPORARY TABLE IF NOT EXISTS `temptable1` LIKE `first60dayactivity`;
/*STEP 2. load the csv into the previously created temporary table*/
LOAD DATA LOCAL INFILE '/Users/me/Downloads/some.csv'
IGNORE INTO TABLE `{temptable}`
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
SET CUSTID = 1030,
CREATED = NOW(),
isactive = 1;
/*STEP 3. update first60dayactivity table changing isactive for records that are not in the temptable*/
UPDATE `first60dayactivity` fa
INNER JOIN `temptable1` temp
ON temp.`mid` = fa.`mid`
AND temp.`primarypartnername` = fa.`primarypartnername`
AND temp.`market` = fa.`market`
AND temp.`agedays` = fa.`agedays`
AND temp.`opendate` = fa.`opendate`
AND temp.`CUSTID` = fa.`CUSTID`
SET fa.isactive = IF( temp.`mid` IS NULL, 0, 1 );
/*STEP 4. insert the temp table records into the real table*/
.....blah blah blah.....
Ok, first create a temporary table so that we have a table to hold the imported .csv data. Next, import the .csv data into the temporary table (all this works perfectly so far).
Here is where I run into an issue. I'm wanting to update the isactive column of each record of the first60dayactivity table to 0 if the record is NOT found in temptable1 (after my import). Ultimately, I'm gathering a .csv, the .csv has the new live data that should be considered "active" and I need to set the old data to inactive. So, the update does an INNER JOIN to match on several column to see if the record is found in the temptable1, if it isn't then set the activity to 0, if it is found in temptable1 then ensure the activity status is 1.
The problem here is that all records in first60dayactivity are retaining the 1 property to indicate it is active. Nothing is getting updated to 0 even though I have proof new records exist within temptable1... Can someone tell me what I'm doing wrong in my query?
Thanks in advance!
temp.mid can never be NULL because you use this column in your join condition and you use an INNER JOIN.
Your join (without the insert) should return the matching rows. Using a LEFT JOIN for the update should do what I suppose you want to do.
I've looked all over the internet for my answer, and perhaps I'm just doing things wrong. I have a column in my MySQL table that I need to replace all the NULL values with a text string in my SQL Query using phpMyAdmin. I don't want the output to come out that way, I want to actually replace the null values with the text string.
I've tried
UPDATE `tablename` SET fieldname = replace (fieldname, "", "textstring")
I've read up on
SELECT ISNULL(field,"replacetext)
But this only shows the output, but doesn't actually replace it in the table.
I can't figure this out, and I've wasted so much time trying to find an answer.
update tablename set fieldname = "textstring" where fieldname is null;
Have you tried
UPDATE `tablename` SET fieldname = '' where fieldname is null
I am using LOAD DATA INTO command to read data from a txt file into my Database. In my input, ı need to take '-' characters as null (not the string null). For example;the input
stack - overflow
have to be
1st column = stack
2nd column = null
3rd column = overflow.
How can I do that??
A simple approach would be to first use LOAD DATA without any conversions, then afterwards run some updates to correct the values that you want changing:
UPDATE yourtable
SET col1 = NULL
WHERE col1 = '-'
The LOAD DATA syntax also allows you to specify transformations to your data:
[SET col_name = expr,...]
The column list can contain either column names or user variables. With user variables, the SET clause enables you to perform transformations on their values before assigning the result to columns.
An example would be:
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, #var1)
SET column2 = CASE WHEN #var1 = '-' THEN NULL ELSE #var1 END;