Query multiple records from table using derived field - mysql

I'm working on a legacy database system which deals with property reports and I am attempting to update a query which shows multiple reports within a time period for the same address. I am using MySQL Workbench to run this query.
The table has a UPRN field for unique addresses, which is fine, however addresses can be manually added without a UPRN. The table also has address1, address2, address3, town, postcode fields. There is NO field which contains the whole address as one line.
We already have a query which uses the UPRN field to generate a list of multiple records within a timescale; however, this will obviously exclude multiple reports without a UPRN.
SELECT * FROM
`rep_base_report` `rep`
WHERE
`rep`.`UPRN` IN (SELECT
`rep2`.`UPRN` AS `UPRN`
FROM
`rep_base_report` `rep2`
WHERE
`rep2`.`STATUS` = 'LODGED'
AND `rep2`.`GENERATED_DATE` > (CURDATE() - INTERVAL 3 MONTH)
GROUP BY `rep2`.`UPRN`
HAVING (COUNT(0) > 1)
ORDER BY `rep2`.`GENERATED_DATE` DESC)
GROUP BY `rep`.`RepReference`
ORDER BY `rep`.`UPRN`
In my head, if I could CONCAT address1, address2 and postcode and use this as the basis for the check, it would give the appropriate output, but I've no idea how to accomplish this.
I've attempted the following but it's giving fewer values that I was expect (I've manually checked some sample data in Excel to see what I should be expecting).
SELECT *
FROM `rep_base_report` `rep`
WHERE
#property :=CONCAT(`rep`.`ADDRESS1`, ' ' ,`rep`.`ADDRESS2`, ' ' ,`rep`.`POSTCODE`) IN (
SELECT #property2 :=CONCAT(`rep2`.`ADDRESS1`, ' ' ,`rep2`.`ADDRESS2`, ' ' ,`rep2`.`POSTCODE`) AS `PROPERTY`
FROM `rep_base_report` `rep2`
WHERE `rep2`.`STATUS` = 'LODGED'
AND `rep2`.`GENERATED_DATE` > CURDATE() - INTERVAL 3 MONTH
GROUP BY #property2
HAVING COUNT(0) > 1
ORDER BY `rep2`.`GENERATED_DATE` DESC
)
GROUP BY `rep`.`RepReference`
ORDER BY `rep`.`UPRN`

SELECT *
FROM `rep_base_report` `rep`
WHERE
CONCAT(`rep`.`ADDRESS1`, ' ', `rep`.`ADDRESS2`, ' ', `rep`.`POSTCODE`) IN (
SELECT CONCAT(`rep2`.`ADDRESS1`, ' ', `rep2`.`ADDRESS2`, ' ', `rep2`.`POSTCODE`) AS `PROPERTY`
FROM `rep_base_report` `rep2`
WHERE `rep2`.`STATUS` = 'LODGED'
AND `rep2`.`GENERATED_DATE` > CURDATE() - INTERVAL 3 MONTH
GROUP BY CONCAT(`rep2`.`ADDRESS1`, ' ', `rep2`.`ADDRESS2`, ' ', `rep2`.`POSTCODE`)
HAVING COUNT(0) > 1
ORDER BY `rep2`.`GENERATED_DATE` DESC
)
GROUP BY `rep`.`RepReference`
ORDER BY `rep`.`UPRN`

Related

How to display a $ next to a number in MYSQL

this is what I am trying, but the '$' doesn't get displayed in the result set
SELECT
id,
CONCAT(first_name, ' ', last_name),
CONCAT('$' + '' + salary),
started_on
FROM employees
WHERE salary >= 100000 AND started_on >= '20180101'
ORDER BY salary DESC, id;
if you want to do it in SQL:
CONCAT('$', salary)
You really should do it in the presentation layer (your app).

Get time duration per each available day

I need to get users visits duration for each day in MySQL.
I have table like:
user_id,date,time_start, time_end
1, 2018-09-01, 09:00:00, 12:30:00
2, 2018-09-01, 13:00:00, 15:10:00
1, 2018-09-03, 09:30:00, 12:30:00
2, 2018-09-03, 13:00:00, 15:10:00
and need to get:
user_id,2018-09-01_duration,2018-09-03_duration
1,03:30:00,03:00:00
2,02:10:00,02:10:00
So columns need to be dynamic as some dates can be missed (2018-09-02).
Is it possible to do with one query without explicit joins per each day (as some days can be null)?
Update #1
Yes, I can generate columns in application side, But I still have terrible query like
SELECT user_id, d1.dt AS "2018-08-01_duration", d2.dt AS "2018-08-03_duration"...
FROM (SELECT
user_id,
time_format(TIMEDIFF(TIMEDIFF(time_out,time_in),time_norm),"%H:%i") AS dt
FROM visits
WHERE date = "2018-09-01") d1
LEFT JOIN(
SELECT
user_id,
time_format(TIMEDIFF(TIMEDIFF(time_out,time_in),time_norm),"%H:%i") AS dt
FROM visits
WHERE date = "2018-09-03") d3
ON users.id = d3.user_id...
Update #2
Yes, data like
select user_id, date, SEC_TO_TIME(SUM(TIME_TO_SEC(time_out) - TIME_TO_SEC(time_in))) as total
from visits
group by user_id, date;
is correct, but in this case data for users goes consistently. And I hope there's the way when I have rows with users and columns with dates (like in example above)
Try something like this:
select user_id, date, sum(time_end - time_start)
from table
group by user_id, date;
You will need to do some tweaking, as you didn't mention the RDBMS provider, but it should give you a clear idea on how to do it.
There's no dynamic way to use pivotting in MySQL but you might use the following for your case :
create table t(user_id int, time_start timestamp, time_end timestamp);
insert into t values(1,'2018-09-01 09:00:00', '2018-09-01 12:30:00');
insert into t values(2,'2018-09-01 13:00:00', '2018-09-01 15:10:00');
insert into t values(1,'2018-09-03 09:30:00', '2018-09-03 12:30:00');
insert into t values(2,'2018-09-03 13:00:00', '2018-09-03 15:10:00');
select min(q.user_id) as user_id,
min(CASE WHEN (q.date='2018-09-01') THEN q.time_diff END) as '2018-09-01_duration',
min(CASE WHEN (q.date='2018-09-03') THEN q.time_diff END) as '2018-09-03_duration'
from
(
select user_id, date(time_start) date,
concat(concat(lpad(hour(timediff(time_start, time_end)),2,'0'),':'),
concat(lpad(minute(timediff(time_start, time_end)),2,'0'),':'),
lpad(second(timediff(time_start, time_end)),2,'0')) as time_diff
from t
) q
group by user_id;
If you know the dates that you want in the result set, you don't need a dynamic query. You can just use conditional aggregation:
select user_id,
SEC_TO_TIME(SUM(CASE WHEN date = '2018-09-01' THEN TIME_TO_SEC(time_out) - TIME_TO_SEC(time_in))) as total_20180901,
SEC_TO_TIME(SUM(CASE WHEN date = '2018-09-02' THEN TIME_TO_SEC(time_out) - TIME_TO_SEC(time_in))) as total_20180902,
SEC_TO_TIME(SUM(CASE WHEN date = '2018-09-03' THEN TIME_TO_SEC(time_out) - TIME_TO_SEC(time_in))) as total_20180903
from visits
group by user_id;
You only need dynamic SQL if you don't know the dates you want in the result set. In that case, I would suggest following the same structure with the dates that you do want.
By the query you can solve your problem. the query is dynamic and you can improve it.
i use TSQL for the query, you can use the idea in MySQL.
declare
#columns as nvarchar(max),
#query as nvarchar(max)
select
#columns =
stuff
((
select
distinct
',' + quotename([date])
from
table_test
for xml path(''), type
).value('.', 'nvarchar(max)'), 1, 1, '')
--select #columns
set #query =
'with
cte_result
as
(
select
[user_id] ,
[date] ,
time_start ,
time_end ,
datediff(minute, time_start, time_end) as duration
from
table_test
)
select
[user_id], ' + #columns + '
from
(
select
[user_id] ,
[date] ,
duration
from
cte_result
)
sourceTable
pivot
(
sum(duration)
for [date] in (' + #columns + ')
)
pivotTable'
execute(#query)

Populate data as individual days of the months as columns

I have created a query that gives me a sum of sales for each department for the day.
What I want is to generate this data for the whole month using one master query.
This is the result I expect to see:
Department No Department Name 1-April 2-April 3-April
1 Infants 100 112 96
2 Kids 120 132 123
Total Total 220 144 219
Currently, I can generate this for one day only, and I have to keep changing the date value to get the data for the next day.
As stated earlier, I want to run a this for the whole month in one master query.
Please try the following...
SELECT CONCAT( 'SELECT department.id AS `Department No`,
department.departmentName AS `Department Name`',
GROUP_CONCAT( ', SUM( date_',
DATE_FORMAT( salesDate, '%d_%M' ),
'.sales ) AS `',
DATE_FORMAT( salesDate, '%e-%M' ),
'`',
SEPARATOR '' ),
' FROM department',
GROUP_CONCAT( ' LEFT JOIN departmentSales AS date_',
DATE_FORMAT( salesDate, '%d_%M' ),
' ON department.id = date_',
DATE_FORMAT( salesDate, '%d_%M' ),
'.department_id AND date_',
DATE_FORMAT( salesDate, '%d_%M' ),
'.salesDate = `',
DATE_FORMAT( salesDate, '%Y-%M-%d' ),
'`',
SEPARATOR '' ),
' GROUP BY department.id
ORDER BY department.id
UNION
SELECT `Totals`,
` `,
GROUP_CONCAT( ', SUM( date_',
DATE_FORMAT( salesDate, '%d_%M' ),
'.sales )',
SEPARATOR '' ),
' FROM department',
GROUP_CONCAT( ' LEFT JOIN departmentSales AS date_',
DATE_FORMAT( salesDate, '%d_%M' ),
' ON department.id = date_',
DATE_FORMAT( salesDate, '%d_%M' ),
'.department_id AND date_',
DATE_FORMAT( salesDate, '%d_%M' ),
'.salesDate = `',
DATE_FORMAT( salesDate, '%Y-%M-%d' ),
'`',
SEPARATOR '' ),
' ) INTO #salesQuery
FROM ( SELECT DISTINCT salesDate
FROM departmentSales
WHERE salesDate BETWEEN FROM_DAYS( TO_DAYS( targetDate ) -
DAYOFMONTH( targetDate ) +
1
)
AND LAST_DAY( targetDate )
) AS salesDateFinder;
PREPARE salesTotals FROM #salesQuery;
EXECUTE salesTotals;
This attempt is based on code found at mysql select dynamic row values as column names, another column as value. I am concerned about the group_concat_max_len variable being a problem. If it is then I'll step up my research into the CASE method eggyal mentioned. If it is not a problem, then I'll take things a bit easier.
Most of the lines here actually form just one SQL statement. It takes the list of salesDate values constructed by the subquery near the end, which are then used to help construct a rather long string containing the SQL statement to be run against the database.
The string is formed from a list of substrings specified as the arguments to the function CONCAT(), which you can read about at https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_concat.
The string starts by specifying that the fields id and departmentName from the table department should be the first selected for each record, followed by the SUM() of each date's sales for that department. Each date field is given a name like date_09_March and in the output a heading like 9-March.
The repeated segments of the string are achieved by using the GROUP_CONCAT function, which you can read about at https://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html#function_group-concat.
After the fields are specified the string specifies that the values should come from a temporary table formed from department LEFT JOINed with an instance of the departmentSales table for each date, with each instance being given an alias like date_09_March and joined on its department_id field to the id field of department.
Once it is formed, the temporary table's records are grouped by department.id so that the totals for each department may be calculated. The output is then sorted by department.id.
The statement used to produce this sorted output and its column headings is then joined to another (somewhat simpler) statement used to generate the totals row. The string containing the SQL statement formed by the concatenation of each substring is then assigned to the variable #salesQuery, prepared and executed.
If you have any questions or comments, then please feel free to post a Comment accordingly.
Further Reading
https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html (for information on the DATE functions used)
Thanks for all the answers
I managed to write a query based on the Pivot Function.
You can control the range of dates with something like this:
select (...) where v_date >= 1-April and v_date <= 30-April
(consider v_date the name of the column and 1-April in date format)

SQL Query to display unique records based on certain conditions

[I have gone through a large number of questions before posting this question.]
I have a table which contains 4 fields. It is ClientId, ClientName,ClientAddress, ClientCity.
Now, I have an autocomplete textbox control where I need to fetch & display client name.
The problem is that in our database we have same client from the same city from different address.
Now the requirement given to me is that the user should be able to see "ClientName" or "ClientName + ClientCity" or "ClientName+ClientCity+ClientAddress" to make it easy for user to select the client.
It means that I need to add a column to the query till it makes it unique.
I am sure there must be some simple solution to this which I am not getting for past 2 days now.
As shown in below sample data, If I show only "D" as a client name to the end user, he will be confused as in which client "D" he has to select. So we need to concatenate city and address to make it unique.
I am expecting an output as below.
You can use COUNT() OVER() for this:
;WITH CTE AS(
SELECT *,
ByName = COUNT(*) OVER(PARTITION BY ClientName),
ByCity = COUNT(*) OVER(PARTITION BY ClientName,ClientCity)
FROM Client
)
SELECT
CASE
WHEN ByName = 1 AND ByCity = 1 THEN ClientName
WHEN ByName > 1 AND ByCity = 1 THEN ClientName + ', ' + ClientCity
WHEN ByName > 1 AND ByCity > 1 THEN ClientName + ', ' + ClientCity + ', ' + ClientAddress
END
FROM CTE
ORDER BY ClientID
RESULT
Client
--------------------------------------------------------
A
B
C, New York
D, London, LSE Houghton Streen London WC2A 2AE
D, London, Hard Rock Cafe London 150 Old Park Lane
F
C, Paris
See SQL Fiddle.
If you are using SQL Server, you can try the string concatenation using "+" operator as follows
select
ClientName + ', ' + ClientCity + ', ' + ClientAddress as ClientData,
Concat(ClientName, ', ', ClientCity, ', ', ClientAddress) Client
from client
The second concatenation is built using SQL CONCAT() funtion will work on SQL Server 2012 and later
For the conditional data following SELECT statement can help,
select
-- ClientName + ', ' + ClientCity + ', ' + ClientAddress as ClientData,
-- Concat(ClientName, ', ', ClientCity, ', ', ClientAddress) ClientDtata2,
client =
case when count(*) over (partition by ClientName) = 1 then ClientName
else
case when count(*) over (partition by ClientName, ClientCity) = 1 then CONCAT(ClientName, ', ' , ClientCity)
else Concat(ClientName, ', ', ClientCity, ', ', ClientAddress)
end
end
from client
TRY THIS,
Declare #t table (clientid int identity(1,1),clientname varchar(50),clientCITY varchar(50)
,clientaddress varchar(200))
insert into #t values ('A','PARIS','DFSDFSDF'), ('C','NEW YORK','DFSDFSDF')
,('C','PARIS','WEQWEQWE'),('D','LONDON','QWE342'),('D','LONDON','21DXCXZC')
;With CTE as
(select *,ROW_NUMBER()over(partition by clientname order by clientid)rn
,ROW_NUMBER()over(partition by clientname,ClientCity order by clientid)rn1
from #T A
)
--select * from cte
select clientname+','+ clientCITY
from cte A WHERE
EXISTS(SELECT CLIENTID FROM CTE WHERE clientname=A.clientname AND RN>1 AND RN1<=1)
UNION ALL
select clientname+','+ clientaddress
from cte A WHERE
EXISTS(SELECT CLIENTID FROM CTE WHERE clientname=A.clientname AND RN1>1)
UNION ALL
select clientname
from cte A WHERE not
EXISTS (SELECT CLIENTID FROM CTE WHERE clientname=A.clientname AND RN>1 )

Mysql - Concat multiple INT fields together to make DATE field

I am working ona small project for a client, and the data in the database is causing me a bit of bother. I am completing a search form for paid installments from the database. I can get the data fine, its the date fields which are giving me bother.
There are separate fields for the date, month and year for each installments. Is there a way I can use CONCAT or DATE or maybe even use CAST on the combined fields to search for the data.
I have tried the following
SELECT DATE_FORMAT( CONCAT( received_date, received_month, received_year ) , '%d/%m/%Y' ) FROM policy_installments
AND
SELECT DATE_FORMAT(STR_TO_DATE(CONCAT('pit.received_date', '-', 'pit.received_month','-', 'pit.received_year'), '%d-%m-%Y') AS mydate, '%d-%m-%Y') AS modate FROM policy_installments pit ORDER BY mydate
AND
SELECT * FROM policy_installments WHERE CONCAT(CAST(received_date AS CHAR),"-",CAST(received_month AS CHAR),"-",CAST(received_year AS CHAR)) >= '1-1-2014' AND CONCAT(CAST(received_date AS CHAR),"-",CAST(received_month AS CHAR),"-",CAST(received_year AS CHAR)) <= '31-3-2014'
When I use the following I get some unexpected data:
SELECT CONCAT_WS( received_date, "-", received_month, "-", received_year ) FROM policy_installments
RESULT
2d313130312d3132303039
2d3237313032372d323732303039 etc etc
Each of the 3 fields are type INT(11).
However, none of these give me the correct data. Is there a way i can convert all 3 fields and use a date to pass into the query to enable me to correctly grab the correct data?
Any help would be much appreciated!
To convert a concatenated string to date, string should be in the same format as what you are using to convert.
For a date-string in dd/mm/yyyy format, should be converted to sql date using %d/%m%Y format.
Example:
SELECT STR_TO_DATE(
CONCAT( received_date, '/', received_month, '/', received_year ),
'%d/%m/%Y'
) as rcvd_date
FROM policy_installments
And to compare with other dates, you can use BETWEEN clause.
Example:
SELECT * FROM policy_installments
WHERE
STR_TO_DATE(
CONCAT( received_date, '-', received_month, '-', received_year ),
'%d-%m-%Y'
) BETWEEN STR_TO_DATE( '1-1-2014', '%d-%m-%Y' )
AND STR_TO_DATE( '31-3-2014', '%d-%m-%Y' )
Refer to:
MySQL: STR_TO_DATE(str,format): Convert a string to a date
Generally, you want to avoid performing the function on the column. Instead, perform the function on the literal, so that you can utilize indexes on the columns:
SELECT *
FROM policy_installments
WHERE received_year = YEAR('2014-03-31')
AND received_month = MONTH('2014-03-31')
AND received_date = DAY('2014-03-31')
Add a multi-column index on received_year, received_month, and received_date, you'll be doing pretty good.
Obviously, this method will only work for = comparisons. For <=, it becomes a bit more complicated:
SELECT *
FROM policy_installments
WHERE
received_year < YEAR('2014-03-31')
UNION
SELECT *
FROM policy_installments
WHERE
received_year = YEAR('2014-03-31')
AND received_month < MONTH('2014-03-31')
UNION
SELECT *
FROM policy_installments
WHERE
received_year = YEAR('2014-03-31')
AND received_month = MONTH('2014-03-31')
AND received_date <= DAY('2014-03-31')
But this longer query is still able to utilize the index.
To simplify things, consider creating an actual date column and keep it updated with the proper value.