MySQL - Get missing entries in a date range - mysql

I have a table with the following structure:
Entry ID | Date | Approved
Whenever a new entry is made, Entry ID auto increments and date is set to whenever the entry was made through the web application. These entries are not necessarily made every day, so there are gaps between dates.
I need to find all "missing" entries, meaning that there is no entry for that date. For instance, if there was an entry for 2015-06-01 and the next one didn't come until 2015-06-07, I need a query that returns the list of dates from 2015-06-02 to 2015-06-06 and an indication of their approved status from that field. I've been looking for a while but can't seem to find a method to get a list of entries that don't exist. Is there a method for this, or should I restructure?

Create a temp table with all possible dates and do
SELECT Date FROM temp_table WHERE Date NOT IN (SELECT Date FROM your_table);

Related

Breaking Up a Date Range by Individual Dates

I have a table with the following features: Invoice ID, billing_period_start, billing_period_end, and items_purchased during that period.
I'm looking to break out a date range by individual dates. A date range can be contained within one month, but it can also be spread across two months, unequally. This will effectively create many more records than are currently in the table. Once I have done that, I need to breakout the amount of purchased items equally among that dates of the daterange.
billing_period_start billing_period_end
-------------------- ------------------
2010-03-05 2010-03-07
2010-04-29 2010-05-05
2010-06-29 2006-08-12
billing_date
------------
2010-03-05
2010-03-06
2010-03-07
2010-04-29
2010-04-30
2010-05-01
...
2010-05-05
2010-06-29
2010-06-30
...
2010-08-12
Now that the date range is broken into individual dates, I need to take the items_purchase and divide it by the number of the days in the billing period for each date, so that I have the items_purchase_per_date.
select
invoice_line_id AS invoice_id
,items_purchased
,billing_period_start
,billing_period_end
,date_from_parts(YEAR(billing_period_start), MONTH(billing_period_start), 1) AS period1_month_start
,last_day(month_start, month) AS period1_month_end
,datediff(day, billing_period_start, billing_period_end) + 1 AS billing_period_length
from "INVOICE_DATA"
order by 1;
I'm running this on Snowflake, but can easily convert from mySQL, if someone knows that DBMS better.
The best way to handle this in a data warehouse is using a date dimension table. That is, a table that contains all the dates you need for analysis, plus any date attributes that are interesting as well, such as which week/month/quarter etc the date belongs to and so on.
Once you have table with unique rows for all relevant dates, you can more easily tackle date spine challenges like this.
For example, for your case you'd write (assuming dates is the name of your date dimension and calendar_date the name of the column containing the unique dates:
select
d.calendar_date,
i.*
from
dates d
join
invoice_data i
on d.calendar_date between i.billing_period_start and i.billing_period_end
Now you have one row per date between those start/end dates and you can do your daily billing allocation.

MySQL Date Handling Before a Given Date Table Join

I have two tables.
One contains information on a given reporting date and another has information on an incident date.
I’m trying to generate a list of the latest information before a given incident (i.e. if an incident occurred on 12/15/2015 and I have reports for 12/15/2014, 12/1/2015, and 1/12/2016, I want to pull the information for 12/1/2015 from that Data set. If on the second row there is another incident that occurred on 1/13/2016 and I have the same data as above I only want to return the information on 1/12/2016).
Dates are in a different column than ID.
Tables are in the following format
ID - DateV- ValueX
JAS 2017-12-15 00:00:00 3.45
I’ve tried running a query similar to below, but it only returns the first item ID-DateCombination not a list of all items. I would even be fine with just the latest reporting date by incident ID. That would give me enough to get the values I need.
Reporting Table - TableRep
Incident Table - TableInc
SELECT TableRep.ID, TableRep.DateV, TableRep.ValueX
FROM TableRep
INNER JOIN Table
ON TableInc.DateP > (
SELECT Max(TableRep.DateV) FROM TableRep;);
I’ve narrowed it down to only records before a given date in the table, but I’m struggling to lose the dates other than the latest ones.
Essentially, I would like to pull the id and Date from TableInc and the first date before the date indicated in TableInc from TableRep for each record in TableInc. Some IDs have multiple entries, so I can’t just pull them all and choose the max for each ID, that would only represent about 50% of the population.

Mysql Function multiple entries in one cell

So I'm working on a schedule system for my job a basically i wanted to know if there is a way where mysql can do something like:
|Monday |tuesday|wendsday|total
|Dan |5am-7am |7am-6pm|6am-11am|
11am-2pm| |2pm-7pm |
5pm-12am|
where i can enter multiple shifts on 1 day for each person in the same cell if needed instead of the name repeating several times like:
Dan|5-4|
Dan|6-8|
and if there is a function to calculate total time in one cell with multiple shifts
There is a way (representing the data as string), but you wouldn't want to do this - you will loose all calculations, searches etc.
You should not try to represent the data in the database exactly as how it looks on paper.
I would make a table like this:
ShiftID|Person|StartTime|EndTime
Making StartTime & EndTime columns of type DATETIME, you will store not only the HH:mm of a shift's start, but also the day. This is helpful when you have a shift which starts on one day and ends in the next, like starting on Monday 2017-05-15 23:00 and ending on Tuesday 2017-05-16 02:00.
You can extract the date only from this filed using MySQL DATE() function and select only those entires which start OR end on this day.
To calculate the shift's duration you can use MySQL function TIMESTAMPDIFF()
You can even use DAYOFWEEK() to get if it is Monday, Tuesday, etc.
About duplicating the person's name - I would make another table, which will match users with their data to IDs an use ID in the column Person, but for a starter and if your data is not big and if speed is not an issue and if typo errors (like Den instead of Dan) are not a problem ... you could use the name directly in this table.
After storing the data in a table like this you could represent it as you wish in HTML (or print).
You can create a third table with the following columns:
person_id int,
start_time datetime,
end_time datetime
Where person_id would be foreign key to Person table and start_time and end_time would be datetime columns. You can then store multiple records for a person in this table and use MySQL's date functions with GROUP BY to generate the report similar to the one in question.

group by day based on a particular column with its unique entries - ROR

I have two tables ImportedContactUser and ImportedContact and the former contains a column imported_contact_id.
Now, there are many entries for the same imported_contact_id in the first table. What i want to do is to group the unique imported_contact_id entries by day.
I am able to group the unique entries by day but the challenge comes when some entries which was imported yesterday are being imported today also by some other user, in that case same entries are being counted for yesterday and today when i group it by day.
So, what I'm trying to do is
first get the unique entries till date
then apply group by day on it
So that I will have only unique entries count on grouping by each day. But with this also I'm not having any luck.
Below are the images of my two tables:
First image is for 'ImportedContactUser' table
Second one is for 'ImportedContact' table
Current code for retrieving the data:
ImportedContactUser.where("platform_id = 6 and updated_at > ? and updated_at <= ?", DateTime.new(2016, 6, 1), Date.today.at_end_of_day).select("DISTINCT imported_contact_id").group("date_format(updated_at, '%Y-%m-%d')").count

Mysql performance selecting blog post views by date

Question about SQL performance when selecting a 'blog post' based on user views by date.
I want to record the user views of each post, and i ll select everyone of them using 'daily' and 'monthly' as parameters:
PS:
Most viewed posts of the day, or month.
To record the views, i created a table to insert, after every page load, the date of each view.
And them select them (count them) by DAY() and MONTH() when needed.
The problem here is, when the table or the amount of users requiring this information grows the select starts to be slower, due to the amount of rows(views) multiplied for the amount of posts.
One alternative that i thought was, create a table for daily records, and another table for monthly records, then on every page load the code checks if there is a row for the selected date, if the rows exist the script increment the views count on it, if it doesn't, the script insert the row with views count = 1;
Ps:
Daily Views
Post ID | Views | Date
1 | 898 | 2014-07-11
2 | 676 | 2014-07-11
1 | 333 | 2014-07-10
This way every post can have only one row per day.
Is there any better option? what do you think about my alternative? there is no need for my suggestion?
I think the best solution is:
Create a table with statistical data with fields:
id
date (store date m-d-y)
day
month
year
views (store number of visits)
page (store blog post)
One unique row per day, and update programmatically as needed.
Then you can make queries using day, month, year fields, even you can add weeknum field to make queries to obtain statistics grouped by weeks.
As addition you can add a second table to store the full date (m-d-y h:m:s) for each visit, you can add fields like browser, ip, etc... to this table.