I have a table contains lots of column. Many columns contain NULL value. But while I'm trying to create CSV file from it, NULL is being replaced by '/N'. I want to set all the columns value to empty string instead of NULL so that I don't have to face problem with '/N'
You can use FIELDS ESCAPED BY clause along with SELECT query and specify an empty String or whitespace (' ') to get empty values instead of \N, e.g.:
SELECT * INTO file
FIELDS ESCAPED BY ''
LINES TERMINATED BY '\n'
FROM test_table;
Here's MySQL's documentation about it, this is what it says:
If the FIELDS ESCAPED BY character is empty, no characters are escaped
and NULL is output as NULL, not \N.
You can update all the columns and set to empty string instead of NULL, If you don't want to continue with NULL.
If you want to continue with NULL in DB, then export into a temp DB table and apply above logic...
First you need to update column default value with below query.
ALTER TABLE `table_name` CHANGE `column_name` `column_name` varchar(length) default '';
UPDATE `table_name` SET `column_name`='' where `column_name`='/N';
May this will help you.
You can use resultset metadata. Iterate through all the columns and check for null values.
ResultSet rsMain = stmt.executeQuery(selectQuery);
ResultSetMetaData rsmd = rsMain.getMetaData();
while(rsMain.next()){
for (int i = 1; i <=rsmd.getColumnCount(); i++) {
if(rsMain.wasNull())
{
sb.append('');// this is the string buffer that writes to the csv
sb.append(',');
}
}
}
You can update the table like this:
UPDATE yourTable
SET col1 = IFNULL(col1, ''),
col2 = IFNULL(col2, ''),
col3 = IFNULL(col3, ''),
...
Once you've done this, you should probably change the schema so that nulls aren't allowed and the default value is '', so you don't have to go through this again.
Related
I created a SET sql table, so a row can contain multiple values: Foo_1 Foo_2 Foo_3.
The above query run but it will update the the row and set just one value.
QueryType valid values are 1 2 or 3
"UPDATE Foo_status
SET type = 'Foo_%s',
status = %d
WHERE name ='%s'
", queryType.c_str(),status,GetName()
When i run the query: Let's assume that QueryType is 1, it's working.. my type row will have Foo_1, but when i run again the query and Foo has QueryType 2, it will remove Foo_1 and add Foo_2
How i can change my query so when i run the query it will continue to update values ?
Example:
Run Query
-Add Foo_1
Run Query
-Add Foo_2
Table struct
CREATE TABLE `Foo_status` (
`name` text NOT NULL,
`type` set('Foo_1','Foo_2','Foo_3') CHARACTER SET latin2 NOT NULL DEFAULT '',
`status` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I tried to search for an solution but without succes, that's i asked here.
Thanks!
Multiple set elements are specified using a comma-separated string. To add a new value, you have to concatenate a string beginning with comma.
"UPDATE Foo_status
SET type = CONCAT(type, ',Foo_%s'),
status = %d
WHERE name ='%s'
", queryType.c_str(),status,GetName()
To remove a value from a SET, use REPLACE():
"UPDATE Foo_status
SET type = REPLACE(type, 'Foo_%s', ''),
status = %d
WHERE name ='%s'
", queryType.c_str(),status,GetName()
It would be easier if you normalized your schema and used a table with a separate row for each type, instead of using the SET datatype. This would also allow you to add an index that would make searching for specific set values more efficient.
You can remove a set wirh this. Here written for select.
You can it also use for UPDATE.
You only must change the string to remove
select t,
trim( BOTH ',' FROM
replace (concat (',',t,','),',Foo_2,',',')) as new_t
from F;
Sample: http://www.sqlfiddle.com/#!9/a8b041/2
I have a binary column in my table in mysql. I need to update all the rows for example.
Replace 2nd byte with 1. Doesn't matter what the value was.
SELECT HEX(data) FROM table;
Then
UPDATE table SET data[1] = 1; // the idea
But how do you do something like this in mysql?
Here how you can do it. First get the binary representation with hex function.
Then on the returned value use substring and concat function to replace the 2nd value with 1 and finally update.
Here is an example.
CREATE TABLE t (c BINARY(10));
INSERT INTO t SET c = 'b';
select (HEX(c)) from t;
62000000000000000000
SELECT concat(
substring(HEX(c),1,1),
'1',
substring(HEX(c),3,length(HEX(c)))
)
from t
61000000000000000000
Now putting all together in the update command will replace the value
update t set c = unhex(
concat(
substring(HEX(c),1,1),
'1',
substring(HEX(c),3,length(HEX(c)))
)
);
select (HEX(c)) from t;
61000000000000000000
Every month we get a CSV file that contains 300+ columns. The layout of the file can change whenever, and columns can be added/removed. I'm in need of a way to create a dynamic table that contains the exact number of columns needed. I don't care about column names, they can be Column1, Column2, etc.. and I plan to set all types to Varchar(500).
Ideally what I would I want to accomplish is a stored procedure that I can simply pass in the number of columns needed and it will loop to create the necessary Table Definition sql and then execute that sql.
Is accomplishing this even possible? I had started to write the following:
BEGIN
Declare loopvar int default 1;
Declare tsql VarChar(5000);
Declare table_definition VarChar(8000);
Declare tablename varchar(20);
Set table_definition = 'Column';
set tablename = 'npi_data';
Set loopvar = 1;
While loopVar < 362 DO
set tsql = table_definition + loopvar + 'varchar(500) DEFAULT NULL' ;
set loopvar = loopvar + 1;
end while;
set tsql = 'CREATE TABLE' + tablename + '(' + tsql + ') ENGINE=InnoDB DEFAULT CHARSET=utf8';
execute tsql;
END;
I'd like something close to that but all I really need is the ability to create a table with any given number of columns.
Thanks!
If you mean you want to be able to pass a list of column names, like 'A,B,C', and use it to generate a list of column definitions, like A varchar(500) DEFAULT NULL,B varchar(500) DEFAULT NULL,C varchar(500) DEFAULT NULL and, eventually, a CREATE TABLE statement with that definition list, you could use an approach like this:
...
SET columnnames = 'A,B,C';
SET sql = CONCAT(
REPLACE(columnnames, ',', ' varchar(500) DEFAULT NULL,'),
' varchar(500) DEFAULT NULL'
);
SET sql = CONCAT('CREATE TABLE ', tablename, ' (', sql, ')');
...
That is, every comma in the name list is expanded so as to complete the definition of the column preceding the comma. As the last item isn't followed by a comma, the type is simply concatenated one more time to the result of REPLACE.
Alternatively, to avoid the repetition of the varchar(500) DEFAULT NULL bit, you could do this:
...
SET columnnames = 'A,B,C';
SET sql = REPLACE(CONCAT(columnnames, ','), ',', ' varchar(500) DEFAULT NULL,');
SET sql = LEFT(sql, LENGTH(sql) - 1);
SET sql = CONCAT('CREATE TABLE ', tablename, ' (', sql, ')');
...
In this case, a comma is added after the last name, then all commas in the resulting string are replaced like in the previous examples. As a final touch to the list, the trailing comma is removed and finally the complete statement is built.
Either way, no loop is needed.
I do have a table with more than 100000 data elements, but there are almost 350 blank rows within. How do I delete this blank rows using phpmyadmin? Manually deleting is a tedious task.
The general answer is:
DELETE FROM table_name WHERE some_column = '';
or
DELETE FROM table_name WHERE some_column IS NULL;
See: http://dev.mysql.com/doc/refman/5.0/en/delete.html
More info when you post your tables!~
Also, be sure to do:
SELECT * FROM table_name WHERE some_column = '';
before you delete, so you can see which rows you are deleting! I think in phpMyAdmin you can even just do the select and then "select all" and delete, but I'm not sure. This would be pretty fast, and very safe.
I am doing the mysql operation in command prompt in windows. And the basic queries:
delete * from table_name where column=''
and
delete * from table_name where column='NULL'
doesn't work. I don't know whether it works in phpmyadmin sqlcommand builder. Anyway:
delete * from table_name where column is NULL
works fine.
I have a PHP script that automatically removes empty rows based on column data types.
That allows me to define "emptiness" differently for different column types.
e.g.
table
first_name (varchar) | last_name (varchar) | some_qty ( int ) | other_qty (decimal)
DELETE FROM `table` WHERE
(`first_name` IS NULL OR `first_name` = '')
AND
(`last_name` IS NULL OR `last_name` = '')
AND
(`some_qty` IS NULL OR `some_qty` = 0)
AND
(`other_qty` IS NULL OR `other_qty` = 0)
Since "0" values are meaningless in my system, I count them as empty. But I found out that if you do (first_name = 0) then you will always get true, because strings always == 0 in MySQL. So I tailor the definition of "empty" to the data type.
This procedure will delete any row for all columns that are null ignoring the primary column that may be set as an ID. I hope it helps you.
DELIMITER //
CREATE PROCEDURE DeleteRowsAllColNull(IN tbl VARCHAR(64))
BEGIN
SET #tbl = tbl;
SET SESSION group_concat_max_len = 1000000;
SELECT CONCAT('DELETE FROM `',#tbl,'` WHERE ',(REPLACE(group_concat(concat('`',COLUMN_NAME, '` is NULL')),',',' AND ')),';') FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = #tbl AND COLUMN_KEY NOT LIKE 'PRI' into #delete_all;
PREPARE delete_all FROM #delete_all;
EXECUTE delete_all;
DEALLOCATE PREPARE delete_all;
END //
DELIMITER ;
Execute the procedure like this.
CALL DeleteRowsAllColNull('your table');
I know this has already been answered and has got a tick, but I wrote a small function for doing this, and thought it might be useful to other people.
I call my function with an array so that I can use the same function for different tables.
$tableArray=array("Address", "Email", "Phone"); //This is the column names
$this->deleteBlankLines("tableName",$tableArray);
and here is the function which takes the array and builds the delete string
private function deleteBlankLines($tablename,$columnArray){
$Where="";
foreach($columnArray as $line):
$Where.="(`".$line."`=''||`".$line."` IS NULL) && ";
endforeach;
$Where = rtrim($Where, '&& ');
$query="DELETE FROM `{$tablename}` WHERE ".$Where;
$stmt = $this->db->prepare($query);
$stmt->execute();
}
You can use this function for multiple tables. You just need to send in a different table name and array and it will work.
My function will check for a whole row of empty columns or NULL columns at the same time. If you don't need it to check for NULL then you can remove that part.
We have a database that has a bunch of records with some bad data in one column, in which an embedded editor escaped some stuff that shouldn't have been escaped and it's breaking generated links.
I want to run a query to replace the bad characters in all the records, but can't figure out how to do it. I found the replace() function in MySQL, but how can I use it inside a query?
For example, what would be the correct syntax if I wanted to replace the string < with an actual less-than angle bracket (<) in all records that have < in the articleItem column? Can it be done in a single query (i.e. select and replace all in one swoop), or do I have to do multiple queries? Even if it's multiple queries, how do I use replace() to do the replace on the value of a field on more than one record?
At a very generic level
UPDATE MyTable
SET StringColumn = REPLACE (StringColumn, 'SearchForThis', 'ReplaceWithThis')
WHERE SomeOtherColumn LIKE '%PATTERN%'
In your case you say these were escaped but since you don't specify how they were escaped, let's say they were escaped to GREATERTHAN
UPDATE MyTable
SET StringColumn = REPLACE (StringColumn, 'GREATERTHAN', '>')
WHERE articleItem LIKE '%GREATERTHAN%'
Since your query is actually going to be working inside the string, your WHERE clause doing its pattern matching is unlikely to improve any performance - it is actually going to generate more work for the server. Unless you have another WHERE clause member that is going to make this query perform better, you can simply do an update like this:
UPDATE MyTable
SET StringColumn = REPLACE (StringColumn, 'GREATERTHAN', '>')
You can also nest multiple REPLACE calls
UPDATE MyTable
SET StringColumn = REPLACE (REPLACE (StringColumn, 'GREATERTHAN', '>'), 'LESSTHAN', '<')
You can also do this when you select the data (as opposed to when you save it).
So instead of :
SELECT MyURLString From MyTable
You could do
SELECT REPLACE (MyURLString, 'GREATERTHAN', '>') as MyURLString From MyTable
UPDATE some_table SET some_field = REPLACE(some_field, '<', '<')
Check this
UPDATE some_table SET some_field = REPLACE("Column Name/String", 'Search String', 'Replace String')
Eg with sample string:
UPDATE some_table SET some_field = REPLACE("this is test string", 'test', 'sample')
EG with Column/Field Name:
UPDATE some_table SET some_field = REPLACE(columnName, 'test', 'sample')
you can write a stored procedure like this:
CREATE PROCEDURE sanitize_TABLE()
BEGIN
#replace space with underscore
UPDATE Table SET FieldName = REPLACE(FieldName," ","_") WHERE FieldName is not NULL;
#delete dot
UPDATE Table SET FieldName = REPLACE(FieldName,".","") WHERE FieldName is not NULL;
#delete (
UPDATE Table SET FieldName = REPLACE(FieldName,"(","") WHERE FieldName is not NULL;
#delete )
UPDATE Table SET FieldName = REPLACE(FieldName,")","") WHERE FieldName is not NULL;
#raplace or delete any char you want
#..........................
END
In this way you have modularized control over table.
You can also generalize stored procedure making it, parametric with table to sanitoze input parameter
This will help you.
UPDATE play_school_data SET title= REPLACE(title, "'", "'") WHERE title = "Elmer's Parade";
Result:
title = Elmer's Parade