Mysql query returns a tremendously large file - mysql

I have the query to select some columns from two tables, which must contain some cells which are actually empty but have to name it in query such as
select name, REPLACE(format, ' – ', '/') as format, CAST(price AS UNSIGNED INTEGER) as price, CONCAT('http://example.com/',url_key) as url, level_id_value, delivery, sku from t1 where delivery <> 'slut' INTO OUTFILE 'test.txt' FIELDS TERMINATED BY '\t';
which perfectly works producing 2.5 MB size text file in lib folder but the other code as follows for another query where i have to add one column from another table along with some empty columns with some title, seems to work but produces a file size of around 11GB,
sELECT t1.sku, t.name, ' ' as Size, format, ' ' as subcategory, CAST(t1.price AS UNSIGNED INTEGER) as price, ' ' as dummy, CONCAT('http://example.com/',t1.url_key) as url, CONCAT('Från ',t1.level_id_value) as shipping_fee, t2.is_in_stock as stock, ' ' as text1, ' ' as Description from t2,t1 where t1.delivery <> 'slut' INTO OUTFILE 'test.txt' FIELDS TERMINATED BY '\t';
any help to debug this issue would be appreciable, don know much about mysql, so detailed explanation are highly welcome. Is there a way where i can make the code analogous to the first working code from two tables? I just need one column from t2 table, that is is_in_stock.

So the first one is a simple query SELECT INTO with a WHERE clause, this is going to return a small number of results.
The second one is showing an implicit inner join, but you arent returning the values based on a unique row, you just pull in everything from both tables, regardless of whether they are matching in both tables
SELECT t1.sku,
t.name,
' ' as Size,
format,
' ' as subcategory, CAST(t1.price AS UNSIGNED INTEGER) as price,
' ' as dummy, CONCAT('http://example.com/',t1.url_key) as url,
CONCAT('Från ',t1.level_id_value) as shipping_fee,
t2.is_in_stock as stock,
' ' as text1,
' ' as Description **`from t2,t1`**
where t1.delivery <> 'slut' INTO OUTFILE 'test.txt' FIELDS TERMINATED BY '\t';
This one is how I would do it, but you still need to isolate each record, so what makes each row unique, and do the inner join ON those fields between t1 and t2 like inner join t2 ON t1.name = t2.name and t1.id = t2.id but it should look something like this
SELECT t1.sku,
t1.name,
t1.format,
CAST(t1.price AS UNSIGNED INTEGER) as price,
CONCAT('http://example.com/',t1.url_key) as url,
CONCAT('Från ',t1.level_id_value) as shipping_fee,
'' as dummy,
'' as size,
'' as subcat,
'' as text1,
'' as Description,
t2.is_in_stock as stock,
from t1
INNER JOIN t2 WHERE t1.delivery <> 'slut'
INTO OUTFILE 'test.txt' FIELDS TERMINATED BY 't\';
Hope it helps!

Related

mysql query - how to merge tables in another table

I have a problem joining tables in the result column. i have a working query which combine different tables using UNION but when i'm extending another table i got an error saying 'The used SELECT statements have a different number of columns'
this is my query:
(SELECT
IDNumber,
CONCAT(LastName, ', ', FirstName, ' ', Middle) as Name,
CONCAT(EmDesignation, ', ', Department) as Information,
Image,
v.PlateNo as PlateNumber
FROM
tblemployee as e, tblvehicle as v
WHERE
v.RFIDNo LIKE '6424823943'
AND
e.RFIDNo LIKE '6424823943')
UNION
(SELECT
IDNumber,
CONCAT(LastName, ', ', FirstName, ' ', Middle) as Name,
CONCAT(Course, ', ', Year) as Information,
Image,
v.PlateNo as PlateNumber
FROM
tblstudents as s, tblvehicle as v
WHERE
v.RFIDNo LIKE '6424823943'
AND
s.RFIDNo LIKE '6424823943')
I have problem with this. Continuation query above
UNION
(SELECT
Barrier
FROM
tblinformation as inf
WHERE
inf.RFIDNo IN (6424823943)
ORDER BY
AttendanceNo DESC LIMIT 1)
The error message is correct. Add NULLs to your second query to match the column number, and it will work.
For example:
SELECT
Barrier,
NULL,
NULL,
NULL,
NULL
FROM
tblinformation as inf
...
The error message states what the problem is. Just improve number of columns in SELECT and it will work correctly.

ERROR :- from keyword not found where expected

SELECT A.CUSTNAME|| WHO IS HOLDING ||A.ACCNAME|| ACCOUNT ||T.TRATYPE|| THE AMOUNT ||T.AMT||
ON || T.t_DATE
FROM ACCOUNTHOLDER A,TRANSACTION T
ON(T.ACCNO=A.ACCNO)
WHERE CUSTNAME='JAMES BOND'
/
I have two tables accountholder and transaction table where accno is common. accountholder accno is primary key and other is foreign key. I need the output in sentence form...
You are probably looking for CONCAT() function, and your custom parts of the sentence have to be enclosed in single quotes and have the desired spaces at the beginning and at the end:
SELECT CONCAT (
A.CUSTNAME,
' WHO IS HOLDING ',
A.ACCNAME,
' ACCOUNT ',
T.TRATYPE,
' THE AMOUNT ',
T.AMT,
' ON ',
T.t_DATE
)
FROM ACCOUNTHOLDER A
INNER JOIN TRANSACTION T ON T.ACCNO = A.ACCNO
WHERE CUSTNAME = 'JAMES BOND'
And your JOIN wasn't right either You either use the explicit way and add INNER JOIN and ON, or you separate your tables with commas and put the condition in the WHERE clause.
You need to use proper join syntax. In addition to the on clause, you need a join statement. In addition, what look like string constants should be in single quotes. And, the proper way to concatenate strings in MySQL is to use the concat() function, not vertical bars:
SELECT concat(A.CUSTNAME, ' WHO IS HOLDING ', A.ACCNAME, ' ACCOUNT ',
T.TRATYPE, ' THE AMOUNT ', T.AMT, ' ON ', T.t_DATE)
FROM ACCOUNTHOLDER A join
TRANSACTION T
ON T.ACCNO=A.ACCNO
WHERE CUSTNAME = 'JAMES BOND';

Shouldn't I see multiple inserts right away?

I'm trying to insert in Table2 multiple rows with multiple columns from Table1, but while doing so, I'm editing some of the values going into Table2. Both are large tables (several million rows). Here's the query:
insert into table2 (board_id,title,content,domainurl)
select (id, replace(title, '_', ' '), description, CONCAT("http://somesite.com/", title))
from table1
When running this query, shouldn't the insert start happening right away (being that I'm not doing a join)? I've started the query several minutes ago (phpmyadmin still shows "loading") but I don't see any rows being added to table2...?
EDIT: The query just stopped with the message "#1241 - Operand should contain 1 column(s)". I have no idea what that means!
Try this:
INSERT INTO table2 (board_id,title,content,domainurl)
SELECT
id AS board_id,
REPLACE(title, '_', ' ') AS title,
description AS content,
CONCAT("http://somesite.com/", title) AS domainurl
FROM table1
As per here all I had to do is remove the parenthesis from the SELECT clause.
select id, replace(title, '_', ' '), description, CONCAT("http://somesite.com/", title)
worked perfectly and fast!

mysql UNION query not working

What am I doing wrong with this:
$sql = "SELECT * FROM content
WHERE threadName LIKE '%$filter%'
ORDER BY lastUpdated desc
UNION SELECT *
FROM content
WHERE threadName NOT LIKE '%$filter%'
ORDER BY lastUpdated desc";
The first statement before the UNION works well on its own, but this one above returns:
mysql_fetch_array() warning - supplied argument is not a valid MySQL result resource
Am I right in believing that UNION will not return duplicate entries, in which case the second SELECT statement doesn't need to have the NOT LIKE but will just return everything that wasn't listed in the first statement.
EDIT: This query should get you the rows matched by filter first, followed by those not matched:
SELECT *
FROM content
ORDER BY
CASE WHEN threadName LIKE '%$filter%' THEN 0 ELSE 1 END,
lastUpdated DESC
Note that you should never SELECT *, list the necessary columns instead.
While using UNION we must use column names instead of '*'.
Here, I m going to create .csv file at specific location on the system with emailid validation expression by using mysql query as bellow.
select 'col1','col2','col3','col4' from tableName
UNION
SELECT col1,col2,col3,col4 FROM tableName WHERE col4 NOT REGEXP '^[a-zA-Z0-9][a-zA-Z0-9.-]*[a-zA-Z0-9.-]#[a-zA-Z0-9][a-zA-Z0-9._-]*[a-zA-Z0-9]\.[a-zA-Z]{2,4}$'
INTO OUTFILE '/home/sachin/mysqloutput/data.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY ''''
LINES TERMINATED BY '\n';

MySQL returning an empty field: CONCAT(nonEmpty1,empty2,nonEmpty3) = NULL

I have PHP 5 code accessing a MyISAM table on MySQL 5 server. The query looks like this:
SELECT CONCAT(fName1,' ',mName2,' ',lName3) AS userName
FROM users
WHERE level > 10
When there's no mName filled in, I am expecting output like "fname lname" , but I'm getting "" (empty string) instead (the number of rows returned is correct). Where am I making a mistake?
PHP code:
<?php
$result = mysql_query($the_above_query);
while ($result_row = mysql_fetch_assoc($result)) {
// do stuff with the name
// except I'm getting empty strings in $result_row['userName']
}
Relevant part of table structure:
CREATE TABLE users {
/* -snip- */
`fName1` varchar(50) default NULL,
`mName2` varchar(50) default NULL,
`lName3` varchar(50) default NULL,
`level` int(11) default 0,
/* -snip- */
} ENGINE=MyISAM DEFAULT CHARSET=utf8;
(also, is this way (column concatenation in MySQL) a good idea, or should I fetch the columns to PHP and join them there?)
Turns out that I was getting back a NULL; PHP treats a returned NULL and empty string("") similarly, you'd have to compare with === to see the difference.
From google: http://bugs.mysql.com/bug.php?id=480
[23 May 2003 4:32] Alexander Keremidarski
Thank you for taking the time to write to us, but this is not
a bug. Please double-check the documentation available at
http://www.mysql.com/documentation/ and the instructions on
how to report a bug at http://bugs.mysql.com/how-to-report.php
This is doccumented behaviour of CONCAT() function.
From Manual chapter 6.3.2 String Functions
CONCAT(str1,str2,...)
Returns the string that results from concatenating the arguments. Returns NULL if any
argument is NULL
Use CONCAT_WS() instead or wrap NULLable paremeters with IFNULL() function.
Documentation and usage for CONCAT_WS: http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#function_concat-ws
From MYSQL docs
CONCAT() returns NULL if any argument
is NULL.
you want to use CONCAT_WS()
CONCAT_WS(separator,str1,str2,...)
But best bet is to just pull it back and use php cause if you need a different format or just one of those fields later you'll have to make another db call
In MySQL concatenating any string to a NULL value results in NULL.
You have to check for NULL before concatenate using IFNULL:
SELECT CONCAT(IFNULL(fName1,''),' ',IFNULL(mName2,''),' ',IFNULL(lName3,'')) AS userName
FROM users
WHERE level > 10
This was the solution I came up with which included Keeper and Ersatz answer. System would not allow me to vote you guys up though :(
CONCAT_WS(IFNULL(ts_usr_nameDetails.first_name,''),' ',IFNULL(ts_usr_lib_connectionNameDetails.first_name,'')) AS composerName
This allowed for some amazing results
This is an answer based on the solution above by #chocojosh and another question here: MySQL/SQL: Update with correlated subquery from the updated table itself
I had a similar problem, but I was trying to concat a bunch of group_concats and some were NULL which caused the whole row to be NULL. The goal was to place data from other tables into a single field in the main table to allow fulltext searches.
Here is the SQL that worked. Note the first parameter for concat_ws I made ' ', this allowed for spaces between the values for the fulltext field.
Hope this helps someone.
update
products target
INNER JOIN
(
select p.id,
CONCAT_WS(
' ',
(select GROUP_CONCAT(field SEPARATOR ' ') from table1 where productId = p.id),
p.title,' ',
(select GROUP_CONCAT(field, ' ', descriptions SEPARATOR ' ') from table2 where productId = p.id),
(select GROUP_CONCAT(field SEPARATOR ' ') from table3 where productId = p.id),
(select GROUP_CONCAT(field, ' ', catno SEPARATOR ' ') from table4 where productId = p.id),
(select GROUP_CONCAT(field SEPARATOR ' ') from table5 where productId = p.id),
(select GROUP_CONCAT(field SEPARATOR ' ') from table6 where productId = p.id),
(select GROUP_CONCAT(field SEPARATOR ' ') from table7 where productId = p.id)
) as ft
from products p
) as source
on target.id = source.id
set target.fulltextsearch = source.ft
you could also the COALESCE() function to return the first non-null value. Like so:
SELECT CONCAT(fName1,COALESCE(CONCAT(' ',mName2,' '),' '),lName3) AS userName
FROM users
WHERE level > 10
This would also only put one space if there was no middle name and a space before and after if there was a middle name.
Reference for this function can be found at: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_coalesce