Get cumulative sum using field names set earlier in the query - jinja2

I have a for loop embedded in my query that takes some categorical variables, renames it with a "total" in front of it (also replaces spaces with underscores), and then sums a different field, AMOUNT, tied to the variable.
{% set columns = ("transfer out", "transfer in", "fee" %}
{% for column in columns %}
, round(
sum(case when table.variable = '{{ column }}' then table.amount else 0 end)
, 2) as total_{{ column|replace(" ", "_")|replace("-", "_") }}s
{% endfor %}
Say this creates:
column
amount
total_transfer_outs
-90
total_transfer_ins
100
total_fees
-10
In the same query, I want to be able to get the sum of the 3 new columns that were created.
sum(total_transfer_outs + total_transfer_ins + total_fees ) = (-90 +100 -10) = 0. Note: this list could expand to >3 variables in the future, so I assume I need to use some sort of loop and cumulative sum to make this scalable.

Turns out you can just loop over it using a plus sign and starting from zero. I was trying to solve this using cumulative sum and/or append functions. This works though...
,0
{% for column in columns %}
+ (total_{{ column|replace(" ", "_")|replace("-", "_") }}s)
{% endfor %}
as sum_of_everything

Related

DBT : Getting error ‘>’ not supported between instances of ‘Table’ and ‘int’

I have to compare the result of query with hardcoded integer value in if condition
{%- set query %}
select cast(1 as integer)
{% endset -%}
{%- set main = run_query(query) -%}
{% if execute %}
{% if main > 2 %}
Getting below error…
'>' not supported between instances of 'Table' and 'int'
The query select cast(1 as integer) returns table(single column/single row) and it is not directly comparable with constant integer:
{% if main > 2 %}
should rather be:
{% if (main.columns[0].values()[0]) > 2 %}

Why am I getting an "unexpected ')' " error when I try and run my dbt jinja model?

I have created a model in DBT to run against Snowflake. I am using some jinja to iterate through a list of table names, and build some sql for each table.
The following script should loop through each table, generate some SQL script and insert a UNION statement between each piece of script except for the last iteration.
I am getting the following error however I can not see any stray parenthesis
11:44:23 001003 (42000): SQL compilation error
11:44:23 syntax error line 14 at position 6 unexpected ')'.
My script is
{{
config(
query_tag = 'DBT: Staging_History'
)
}}
{% set table_names_query %}
select table_name
from information_schema.tables
where table_type = 'BASE TABLE' and TABLE_CATALOG = 'PROD_SOURCE' and TABLE_SCHEMA =
'DIS_STG'
{% endset %}
{% set results = run_query(table_names_query) %}
{% if execute %}
{% set results_list = results.columns[0].values() %}
{% else %}
{% set results_list = [] %}
{% endif %}
{% for table_name in results_list %}
SELECT * FROM TABLE(INFORMATION_SCHEMA.copy_history(table_name=>'PROD_SOURCE.DIS_STG.DBO_ORGANIZATION', start_time=>dateadd(DAY, -90, current_timestamp)))
{% if not loop.last %} UNION {% endif %}
{% endfor %}
If you are going to use the macro of CURRENT_TIMESTAMP (aka without the parens) it must be in upper case.
So ether change to
current_timestamp()
or
CURRENT_TIMESTAMP

using aggregate with values : group by : django

i want to return a product names in distinct , and also i've used aggregate to some calculation as well
this is my models.py
class CustomerInvoice(models.Model):
customer = models.CharField(max_length=50)
items = models.ManyToManyField(Product,through='ProductSelecte')
date = models.DateTimeField(auto_now_add=True)
class ProductSelecte(models.Model):
product = models.ForeignKey(Product, on_delete=models.CASCADE)
products= models.ForeignKey(CustomerInvoice,on_delete=models.CASCADE,related_name='product')
qnt= models.IntegerField(default=1)
price = models.IntegerField()
cash = models.IntegerField()
i want to make table to track customers activity
and this is my query
context['clients'] = ProductSelecte.objects.filter(
products__customer=self.object.name).values('product').annotate(quantity=Sum(F('qnt'))).order_by('product')
till here works fine but i also need to aggregate price using this
.aggregate(
total_price=Sum(F('price')))
it raise this error
'dict' object has no attribute 'order_by'
thanks for your helping
UPDATE
for example at in first invoice of john 4th june john bought two pen price:20$ with 3 book price : 30$ total price for this invoice :50$ , and for second invoice 5th july he buy 1 pen price:10$ with 2 copybook price:20$ total price:30 , i need a result like this
client : john , activities : 3 pen(total pens he bought till now),price pens : 30 (total pens price he bought till now) , 3 book(total books he bought till now) ; price:30(total books price he bought till now) ,2 copybook ;price:20
and then total price of both invoice : 30+50 = 80
You should make the annotate per client per product and then perform some "grouping" at the Django/Python level:
from django.db.models import F, Sum
from itertools import groupby
from operator import itemgetter
qs = ProductSelecte.objects.values(
customer=F('products__customer')
product=F('product')
).annotate(quantity=Sum('qnt')).order_by('customer', 'product')
data = {
k: list(vs)
for k, vs in groupby(qs, itemgetter('customer'))
}
Here data is a dictionary that maps the name of the customer to a list of dictionaries that contain the id of the product, the customer name and the quantity.
If you pass it to the template, you can for example render this with:
<ul>
{% for c, vs in data.items %}
<li>{{ c }}</li>
<ul>
{% for v in vs %}
<li>{{ v.product }}: {{ v.quantity }}</li>
{% endfor %}
</ul>
{% endfor %}
</ul>
EDIT: based on the updated request, you should make an annotate and aggregate request:
class ProfileClientDetailView(LoginRequiredMixin,DetailView):
# …
def get_object(self):
return get_object_or_404(Client,pk=self.kwargs['pk'])
def get_context_data(self,*args,**kwargs):
data = super().get_context_data(*args,**kwargs)
data['products'] = ProductSelecte.objects.filter(
products__customer=self.object.name
).values('product').annotate(
quantity=Sum('qnt'),
total_price=Sum('price')
).order_by('product')
data['total'] = ProductSelecte.objects.filter(
products__customer=self.object.name
).aggregate(
quantity=Sum('qnt'),
total_price=Sum('price')
)
return data
Then it can be rendered as:
<ul>
{% for pr in products %}
<li>{{ pr.product }}: {{ pr.total_price }} ({{ pr.quantity }} items)</li>
</ul>
TOTAL: {{ total.total_price }} ({{ total.quantity }} items)

Liquid filter by date (Jekyll)

I'm trying to filter posts by specific dates in Jekyll. For example, I want to filer all posts posts today, in the past 30 days and everything before that.
However I'm running into a couple of issues:
How can I get the current date, starting at 00:01AM
How do I get the current date, and subtract 30 days
How can I use where_exp to filter by this date
Currently I tried to do something like this, but this converts the date into a string, and can't be used in the where_exp:
{% capture thirty_days_ago %}{{'now' | date: '%s' | minus: 2592000 }}{% endcapture %}
{% assign last_30_days_posts = site.posts | where_exp:"post", "post.posted_on > thirty_days_ago" %}
Liquid error (line 22): comparison of Time with String failed in index.html
I could do a simple check when looping over the posts, but would prefer using a filter before this.
{% capture thirty_days_ago %}{{'now' | date: '%s' | minus: 2592000 }}{% endcapture %}
{% for post in site.posts %}
{% capture post_date %}{{ post.posted_on | date: '%s' | plus: 0 }}{% endcapture %}
{% if job_date > thirty_days_ago %}
{% include components/job.html job=job %}
{% endif %}
{% endfor %}
For anyone looking at a solution; I ended up writing a custom filter:
def last_month_filter(posts)
now = DateTime.now
today = DateTime.new(now.year, now.month, now.day, 0, 0, 0, now.zone)
target = today - 30
posts.select do |post|
postedOn = post.data['posted_on'].to_datetime
if postedOn < today && postedOn > target
post
end
end
end
Usage:
{% assign last_30_days = site.posts | last_month_filter | sort:"posted_on" | reverse %}

Get date difference using adx studio liquid template

Im trying to get the day difference between two dates using adx studio liquid templates, my current code is
{%assign expirydate = bh_expirydate | date:'MM/dd/yyyy' %}
{%assign datenow = now | date: 'MM/dd/yyyy' %}
{%assign diffdays = expirydate | minus: datenow %}
I know that this line of code will not work, but the logic is that. I just can seem to find appropriate example. Can someone shed some light on this one?
Not sure how you would do this without javascript, but here is my solution assuming bh_expirydate exists:
Days till expiry: <span id="expiryDays"></span>
<script>
Date.daysBetween = function( date1, date2 ) {
//Get 1 day in milliseconds
var one_day=1000*60*60*24;
// Convert both dates to milliseconds
var date1_ms = date1.getTime();
var date2_ms = date2.getTime();
// Calculate the difference in milliseconds
var difference_ms = date2_ms - date1_ms;
// Convert back to days and return
return Math.round(difference_ms/one_day);
}
var dt1 = new Date();
var dt2 = new Date('{{ bh_expirydate| date: "yyyy/MM/dd" }}');
$('#expiryDays').text(Date.daysBetween(dt1, dt2));
</script>
Calculate years since first date until now
With the first assign we are taking the current time and subtracting a date of birth.
This will result in a timespan. Convert it to a string and split to get the number of days in the first array element.
The second assign will take that first string element with the number of days and convert it into an integer.
The display will divide by 365 to give the years since dob.
{% if item.dob %}
{% assign words = now | minus: item.dob | string | split: '.' %}
{% assign days = words.first | integer %}
{{ days | divided_by: 365 }}
{% endif %}