Apache Drill query retrieving data older than 6 months - apache-drill

Hello im new in Apache Drill and i'm trying to do something like this :
SELECT * FROM `dfs`.`data`.`./bp2s/nt/raw/20220909/16_09_34_1665670176715.NTFF04_FFMA_20220909_Positions_20220909181421268120.txt.parquet` Where VALUATION_DATE "older than 6 months"
Any suggestion please ?

select ... where valuation_date < current_date - interval '6' month;
The inequality in the WHERE clause compares the valuation_date column to an expression that subtracts 6 months from today's date thereby including only those records that have a valuation_date ealier than 6 months ago.

Related

SQL query to select apps used at least once during 3 different days in the last 7 days

I have a table that contains 2 columns : 1 column "app_id" and 1 column "time".
I am trying to make a SQL request to know the number of "app_id" that have been used at least once in 3 different days, in the last 7 days.
Currently, I achieved selecting all the data in the last 7 days using :
SELECT app_id,time FROM connexions WHERE time BETWEEN NOW() - INTERVAL 167 HOUR AND NOW()
I am using 167 HOUR instead of 7 DAY because I have a 1 hour time difference between my server and the database (no worries about that, i'll fix it later!)
Thanks!
SELECT app_id
FROM connexions
WHERE time BETWEEN NOW() - INTERVAL 167 HOUR AND NOW()
GROUP BY app_id
HAVING COUNT( DISTINCT day(time) ) > 3
Be aware this only works because is a week. If you want something like 3 months you would need be more specific.

Get records where today is 6 months after date

I am trying to run a query and get all the records where today is 6 months after the start date.
Here is what I have but it doesn't seem to be working. What am I missing?
select * from users where DATE_ADD(start_date, INTERVAL 1 MONTH) = CURDATE()
Thanks!
This is too long for a comment. First, your code is checking for one month rather than 6 months.
The first possibility is that you don't have any records on that date.
Another is that the dates have times associated with them. If so, you can do:
where DATE_ADD(DATE(start_date), INTERVAL 6 MONTH) = CURDATE()

SQL query to retrieve latest 2 week records

I have a database with CreatedDate is store in Unix epoch time and some other info. I want a query to able to retrieve latest 2 week record base on the last record.
Below is part of the example
ID User Ranking CreatedDate
-------------------------------------------------------
1 B.Sisko 1 1461136714
2 B.Sisko 2 1461123378
3 B.Sisko 3 1461123378
4 B.Sisko 3 1461600137
5 K.Janeway 4 1461602181
6 K.Janeway 4 1461603096
7 J.Picard 4 1461603096
The last record CreatedDate is 25 Apr 2016, so I want the record from 12 Apr to 25 Apr.
I not sure how to compare to get latest data? any suggestion
The simplest method is probably to just subtract two weeks from today's date/time:
where CreatedDate >= UNIX_TIMESTAMP() - 7*24*60*60
Another approach is to convert the value to a date/time:
where from_unixtime(CreatedDate) >= date_sub(now(), interval 2 week)
The advantage of this approach is that it is easier to align to days. So, if you want two weeks starting at midnight:
where from_unixtime(CreatedDate) >= date_sub(curdate(), interval 2 week)
The disadvantage is that the function on the column prevents the use of indices on that column.
EDIT:
This is definitely not how your question was phrased. But in that case, you should use:
select t.*
from t cross join
(select from_unixtime(max(CreatedDate)) as maxcd from t) m
where from_unixtime(CreatedDate) >= date_sub(maxcd, interval 2 week);
It may seem odd, but you need to execute two queries one to find the Maximum Date and knock off 14 days -- and then use that as a condition to requery the table. I used ID_NUM since ID is a reserved word in Oracle and likely other RDBMS as well.
SELECT ID_NUM, USER, RANKING,
TO_DATE('19700101000000', 'YYYYMMDDHH24MISS')+((CreatedDate-18000)
/(60*60*24)) GOOD_DATE
FROM MY_TABLE
WHERE
GOOD_DATE >=
(SELECT MAX( TO_DATE('19700101000000', 'YYYYMMDDHH24MISS')+
((CreatedDate-18000) /(60*60*24))) -14
FROM MY_TABLE)

mysql request every week since last 6 months

I'm trying to request my database to get numbers of rows where a specific login appears from last 6 months ordered by weeks.
The output has to be something like this :
My request is like this :
SELECT count(*) FROM table1 where (`EndDate` BETWEEN DATE_SUB(CURDATE(), INTERVAL 6 MONTH) and CURDATE()) AND the_login LIKE 'LOGIN 1';
It shows every rows from last 6 month for now.
Could you complete my request or change it if needed please ?
You can use YEAR() and WEEK() to extract them from the date, and group by them .
Note that I didn't use only WEEK because its not enough, if you'll perform this before June, weeks can be misunderstood.
Also, don't use LIKE for exact match, use them only for partially comparison.
SELECT YEAR(t.endDate) as m_Year, WEEK(t.endDate) as m_Week, count(*)
FROM table1 t
where t.`EndDate` BETWEEN DATE_SUB(CURDATE(), INTERVAL 6 MONTH)
and CURDATE())
AND t.the_login = 'LOGIN 1'
GROUP BY m_Year,m_Week

Need to find duplicates with multiple criteria and then comparing dates between duplicates

I am writing annual membership registrations to a single db table. I need to keep track of when renewals have occurred in less than 11 months from their last renewal.
I look for the duplicate rows based on multiple criteria. I currently have this working with out the 11 month criteria, although it's slow. Here's what I currently use.
SELECT y_reg.* FROM y_reg WHERE (((y_reg.season) In (SELECT season FROM y_reg As Tmp
GROUP BY season, Father_Last_Name, Father_First_Name
HAVING Count(*)>1
AND Father_Last_Name = y_reg.Father_Last_Name
AND Father_First_Name = y_reg.Father_First_Name)))
ORDER BY y_reg.season, y_reg.Father_Last_Name, y_reg.Father_First_Name
I have a field Date which is the date of the renewal that I need to evaluate. I'd like to add something like "AND Date - Date < 335"
335 is the number of days and is about 1 month short of a year. But I just keep getting syntax error because I clearly don't know what I'm doing.
Date arithmetic works quite well in MySQL; you just need the knack.
You can say things like
AND later.Date >= earlier.Date
AND later.Date < earlier.Date + INTERVAL 11 MONTH
That particular pair of comparisons comes up true if the later date occurs in the time range between the earlier date and 11 months later.
In general you can say stuff like this to do date arithmetic.
datestamp + INTERVAL 1 HOUR
datestamp - INTERVAL 5 MINUTE
datestamp + 1 MONTH - 3 WEEK
datestamp - INTERVAL 3 QUARTER (calendar quarters)
LAST_DAY(datestamp) + INTERVAL 1 DAY - INTERVAL 1 MONTH
The last item is the first day of the month containing the datestamp. This whole date thing works quite well.
I think you should consider a so-called self-join query to get your duplicate-except-for-date results. Try something like this.
SELECT a.*
FROM y_reg a
JOIN y_reg b ON a.Father_Last_Name = b.Father_Last_Name
AND a.Father_First_Name = b.Father_First_Name
AND b.Date < a.Date - 11 MONTH
AND b.Date >= a.Date - 12 MONTH