I have a table with the following features: Invoice ID, billing_period_start, billing_period_end, and items_purchased during that period.
I'm looking to break out a date range by individual dates. A date range can be contained within one month, but it can also be spread across two months, unequally. This will effectively create many more records than are currently in the table. Once I have done that, I need to breakout the amount of purchased items equally among that dates of the daterange.
billing_period_start billing_period_end
-------------------- ------------------
2010-03-05 2010-03-07
2010-04-29 2010-05-05
2010-06-29 2006-08-12
billing_date
------------
2010-03-05
2010-03-06
2010-03-07
2010-04-29
2010-04-30
2010-05-01
...
2010-05-05
2010-06-29
2010-06-30
...
2010-08-12
Now that the date range is broken into individual dates, I need to take the items_purchase and divide it by the number of the days in the billing period for each date, so that I have the items_purchase_per_date.
select
invoice_line_id AS invoice_id
,items_purchased
,billing_period_start
,billing_period_end
,date_from_parts(YEAR(billing_period_start), MONTH(billing_period_start), 1) AS period1_month_start
,last_day(month_start, month) AS period1_month_end
,datediff(day, billing_period_start, billing_period_end) + 1 AS billing_period_length
from "INVOICE_DATA"
order by 1;
I'm running this on Snowflake, but can easily convert from mySQL, if someone knows that DBMS better.
The best way to handle this in a data warehouse is using a date dimension table. That is, a table that contains all the dates you need for analysis, plus any date attributes that are interesting as well, such as which week/month/quarter etc the date belongs to and so on.
Once you have table with unique rows for all relevant dates, you can more easily tackle date spine challenges like this.
For example, for your case you'd write (assuming dates is the name of your date dimension and calendar_date the name of the column containing the unique dates:
select
d.calendar_date,
i.*
from
dates d
join
invoice_data i
on d.calendar_date between i.billing_period_start and i.billing_period_end
Now you have one row per date between those start/end dates and you can do your daily billing allocation.
Related
I have checked around but can't seem to find a solution to this, which is why I'm asking for help here.
I am working on a project with Laravel 5.8. I have a mysql table storing customer's orders. The orders table have among other columns ID, order No and a created_at column storing the timestamp of the order.
Now I want to return a collection that contains all the dates there are orders and possibly the number of orders for those dates, returned just once like so;
7/10/2019 - 5
6/10/2019 - 6
5/10/2019 - 12
4/10/2019 - 9
I want only the date part of the timestamp of the days when there are orders. The days without orders should be skipped.
I have done this;
Order::selectRaw('Date(created_at) as date')->groupBy('date')->get();
But keep getting only today's date.
Question about SQL performance when selecting a 'blog post' based on user views by date.
I want to record the user views of each post, and i ll select everyone of them using 'daily' and 'monthly' as parameters:
PS:
Most viewed posts of the day, or month.
To record the views, i created a table to insert, after every page load, the date of each view.
And them select them (count them) by DAY() and MONTH() when needed.
The problem here is, when the table or the amount of users requiring this information grows the select starts to be slower, due to the amount of rows(views) multiplied for the amount of posts.
One alternative that i thought was, create a table for daily records, and another table for monthly records, then on every page load the code checks if there is a row for the selected date, if the rows exist the script increment the views count on it, if it doesn't, the script insert the row with views count = 1;
Ps:
Daily Views
Post ID | Views | Date
1 | 898 | 2014-07-11
2 | 676 | 2014-07-11
1 | 333 | 2014-07-10
This way every post can have only one row per day.
Is there any better option? what do you think about my alternative? there is no need for my suggestion?
I think the best solution is:
Create a table with statistical data with fields:
id
date (store date m-d-y)
day
month
year
views (store number of visits)
page (store blog post)
One unique row per day, and update programmatically as needed.
Then you can make queries using day, month, year fields, even you can add weeknum field to make queries to obtain statistics grouped by weeks.
As addition you can add a second table to store the full date (m-d-y h:m:s) for each visit, you can add fields like browser, ip, etc... to this table.
I'm storing some data in a table and I want to be able to display the total data points per day on the graph. So the first graph point might be 7 for Monday, Jan 1, 2013, and then 3 for Tuesday, Jan 2, 2013...etc.
I have full time/date stamps for each of my data points in my table of SQL type datetime.
My pseudo statement looks like this, but I'm concerned since I'm using the datetime data type:
SELECT
DATE(created_at) AS create_date
COUNT(id) AS total
FROM
data_table
GROUP BY
create_date
How can I get the total data points per day, regardless of the timestamp?
Try this
SELECT
DATE(created_at) AS create_date,
COUNT(id) AS total
FROM
data_table
GROUP BY
DATE(created_at)
Best would be to start a daily cron job that stores the number of the data points for every day. So you can every day count the number between let's say 24.00.00 to 23.59.59.
If you want to count them on the fly you might have slow requests on huge data amounts, since the grouping query cannot use table index.
But maybe you can add a new table column where you store just the date additionally to the timestamp.
I have a little problem with a specific functionality. I have a table of reservations for rooms.
Table: Reservations
room (BIGINT), date (DATE)
4, 2012-09-25
4, 2012-09-27
4, 2012-09-30
I need to obtain the dates when room with id = 4 is available between a date range. For example, if i want the dates that the room is available between 2012-09-25 and 2012-10-01 (inclusive) i have to obtain the next result:
date (DATE)
2012-09-26
2012-09-28
2012-09-29
2012-10-01
Any help will be appreciated. Thanks in advance.
This sort of query is easily handled if you add a table of dates to your database. Fill it with a large number of dates (and add to it when you need to), and just do a right outer join to it to find the missing dates.
If you can't or don't want to add an additional table, you can convert the dates to integer field (representing the number of days since Jan 1, 1970) with this:
SELECT room, UNIX_TIMESTAMP(`date`)/86400 FROM reservations
From there, you can use the techniques in this question or elsewhere to look for gaps in that sequence.
As #steven-cheng said, i had to use the combination of mysql and code because creating a temporary table is too messy. Using the code to get the dates that are available is faster.
I'm looking to make some bar graphs to count item sales by day, month, and year. The problem that I'm encountering is that my simple MySQL queries only return counts where there are values to count. It doesn't magically fill in dates where dates don't exist and item sales=0. This is causing me problems when trying to populate a table, for example, because all weeks in a given year aren't represented, only the weeks where items were sold are represented.
My tables and fields are as follows:
items table: account_id and item_id
// table keeping track of owners' items
items_purchased table: purchaser_account_id, item_id, purchase_date
// table keeping track of purchases by other users
calendar table: datefield
//table with all the dates incremented every day for many years
here's the 1st query I was referring to above:
SELECT COUNT(*) as item_sales, DATE(purchase_date) as date
FROM items_purchased join items on items_purchased.item_id=items.item_id
where items.account_id=125
GROUP BY DATE(purchase_date)
I've read that I should join a calendar table with the tables where the counting takes place. I've done that but now I can't get the first query to play nice this 2nd query because the join in the first query eliminates dates from the query result where item sales are 0.
here's the 2nd query which needs to be merged with the 1st query somehow to produce the results i'm looking for:
SELECT calendar.datefield AS date, IFNULL(SUM(purchaseyesno),0) AS item_sales
FROM items_purchased join items on items_purchased.item_id=items.item_id
RIGHT JOIN calendar ON (DATE(items_purchased.purchase_date) = calendar.datefield)
WHERE (calendar.datefield BETWEEN (SELECT MIN(DATE(purchase_date))
FROM items_purchased) AND (SELECT MAX(DATE(purchase_date)) FROM items_purchased))
GROUP BY date
// this lists the sales/day
// to make it per week, change the group by to this: GROUP BY week(date)
The failure of this 2nd query is that it doesn't count item_sales by account_id (the person trying to sell the item to the purchaser_account_id users). The 1st query does but it doesn't have all dates where the item sales=0. So yeah, frustrating.
Here's how I'd like the resulting data to look (NOTE: these are what account_id=125 has sold, other people many have different numbers during this time frame):
2012-01-01 1
2012-01-08 1
2012-01-15 0
2012-01-22 2
2012-01-29 0
Here's what the 1st query current looks like:
2012-01-01 1
2012-01-08 1
2012-01-22 2
If someone could provide some advice on this I would be hugely grateful.
I'm not quite sure about the problem you're getting as I don't know the actual tables and data they contain that generates those results (that would help a lot!). However, let's try something. Use this condition:
where (items.account_id = 125 or items.account_id is null) and (other-conditions)
Your first query is perfectly acceptable. The fact is you don't have data in the mysql table and therefore it can't group any data together. This is fine. You can account for this in your code so that if the date does not exist, then obviously there's no data to graph. You can better account for this by ordering the date value so you can loop through it accordingly and look for missed days.
Also, to avoid doing the DATE() function, you can change the GROUP BY to GROUP BY date (because you have in your fields selected DATE(pruchase_date) as date)