How can I pass pure sql into a macro and then do the iteration in a dbt jinja2 macro - jinja2

I am building a data pipeline that ingests data from one single table with different groupby objects. The following pure sql example demonstrates how it looks like:
select
a1,b1,
r,s,t
from table
group by a1, b1
union all
select
a2,b2,
r,s,t
from table
group by a2, b2
union all
select
a3,b3,
r,s,t
from table
group by a3, b3
union all
...
The problem I am trying to solve is that this sql has to repeat 40 times of union all with only groupby object changed for each time. I want to make the SQL configurable and dynamic to load its group by objects, so I only need to code the SQL query once along with a list of 40 elements of the changed groupby objects.
Theoretically, using macro I can do
(1) iteratively create a single table 40 times with different groupby objects.
(2) combine 40 tables together.
For (1) one approach I can think of is to embed the sql into a macro something like this pseudo code
{% macro create_table(groupby = [[a1, b1], [a2, b2], [a3, b3]]) %}
{% for a, b in groupby %}
table =
select
a,b,
r,s,t
from table_a
group by a1, b1
{% end_macro %}
However, one constraint is that I don't want to code the sql within a macro file. It would be better to place the sql logic in a sql file for some reason. Then how to code this out and pass the sql into a macro? I saw a sql can call macro but never saw the reverse way.
Also, for (2), I can't find any straightforward approach .
I would appreciate it if anyone could help to have a real macro code implementation since I don't have too much advance experience with dbt and jinja macro and am having a hard time researching on it
Edit: the column names are not strictly a1, a2, ... a40. I used a1 for simplicity only. You can imagine the columns are real English words like State, Country, Year, and things like these. And it doesn't necessarily have to iterative the whole list of columns. That's why macro is necessarily required.

I'd not use a macro for this. As you mentioned, you can define the SQL on a model and then just iterate through the different field names with a range.
One possible approach would be the following:
{% set max_range = 40 %}
{% for range_ in range(max_range) %}
select
a{{ range_ + 1 }},
b{{ range_ + 1 }},
r,
s,
t
from {{ source('your_schema', 'your_table') }} -- This could also be a `ref` if selecting from an existing model
group by 1, 2
{% if not loop.last -%} union all {% endif -%}
{% endfor %}
Update: I based my answer on the fact that fields would follow an increasing integer value on the field name. That's not the case according to the OP, so I propose another solution using an array of arrays containing pairs of field names:
{% set field_pairs = [['country', 'city'], ['first_name', 'last_name'], ['address', 'postal_code']] %}
{% for row in field_pairs %}
select
{% for value in row -%}
{{ value }},
{% endfor %}
r,
s,
t
from {{ source('your_schema', 'your_table') }} -- This could also be a `ref` if selecting from an existing model
group by 1, 2
{% if not loop.last -%} union all {% endif -%}
{% endfor %}

Related

Combine different properties into one colum in #ask query in semantic mediawiki

Is it possible to combine two properties with the same datatype into one column in an #ask query in SMW?
Suppose I have different datasets with Identifiers that are named differently:
#Object1
[[Has isbn::9780552145985]]
#Object2
[[Has id=83897239]]
I would like to combine these two in one column in an #ask query, like this:
{{#ask:
[[Category:Besitz]]
|?Has id = ID
|?Has ISBN = ID
|format=broadtable
}}
In the result I would like to have only the one column ID instead of two columns that are called
ID.
Thank you in advance!
Use template format instead of broadtable with template like
{{!}}-
{{!}} {{{Has isbn|{{{Has id|}}}}}}
Or install Semantic Scribunto, use mw.smw.ask and row['Has id'] or row['Has isbn'].

How can I return 2 decimals if they end in 0

How can I return two decimals if the number ends in 0? What I am using now returns two decimals only if the last number is >0 (i.e., 60.58). Two decimals aren't returned if decimal ends in 0. (I.e., 50.80 returns as 50.8)
Here is what I am using ${{ |float|round(2) }}
I understand your query relates to showing values in jinja template. This expression in your jinja template show the zero at the end:
{{ "%.2f"|format(your_variable) }}
So, if the value of your_variable is 123.4000, the template will show as 123.40.

add new column to existing table and show it into a view blade (LARAVEL,mysql)

my goal is to display the total of the SUCCESS payments in my project in home blade.
if is it possible to do this without creating new column its ok.
step 1 :
i have a table named (payments) , i want to add a new column named (total_amount) ,
i already have the amount column so i need to put the SUM of this column to the new one (total_amount).
NOTE : i need just the success payments , so the SUM query need a condition where status = 'success'
enter image description here
step 2 :
after getting the total amount , i want to show it into home view blade .
i just want to show the total of payments in < h1 >
please i need step by step and what i should put in the controller and model... ,
i try to get the total without creating a new column but wasn't work so i think that i need to create his column first and get the value of this column, note that its just a one value not array.
You don't need to have a sum column. In general, you should avoid storing aggregates. Do something like this instead:
[Controller]
$sum = Payment::where('status', 'success')->sum('amount');
Then when you construct the view, make sure to compact sum with the other variables:
[Controller]
return view('yourtemplate', compact( 'all', 'your', 'other', 'vars', 'sum' ));
Then in the view itself, just echo out the sum wherever you want it:
[View] <h1>{{ $sum }}</h1>
EDIT: Thought of another way. If you already have the payments as a collection in that view, you can just use the collection helper:
[View] <h1>{{ $payments->where('status', 'success')->sum('amount') }}</h1>
If I understand your question, and you need the total amount of every successfully payment? If the question is yes, here is your query (assuming you have created your Payment model).
Payment::selectRaw('SUM("amount")->where('status', 'success')->first();

Link filters to queries on Superset

I have created a visualisation in Apache Superset based on a Saved Query. How can I update the query based on the values filtered within a Filter Box?
I have experimented with Jinja and managed to pass hardcoded variables to my query through the template parameters. Now I just need to connect Jinja to the Filter Box such that the values are obtained through the filter rather than hard coded.
I discovered that this is possible using the filter_values function that is added to the Jinja context via this file: https://github.com/apache/superset/blob/master/superset/jinja_context.py
The example in that file shows how one can build a templated query that pulls values from a filter box:
SELECT action, count(*) as times
FROM logs
WHERE action in ( {{ "'" + "','".join(filter_values('action_type')) + "'" }} )
GROUP BY action
So if you have a filter box for selecting values for action_type, those values will be returned by filter_values.
The column name used in the filter should be the same in another table also. Have you tried it? If column names are different then create a materialized view with the changed column name or rename the column in the table itself

Store field value in a table field Access 2010

I have a problem I have a simple calculation carried out on a form
=[subtotal]-[discount]+[delivery]
I then want to store the result in my order table under the field TotalPrice
I have hunted around google but can not find anything that helps me.
I know its not good to store field values in tables but it needs to be done.
Thanks in advance.
Just to be clear.
[subtotal] [discount] and [delivery] are all text boxes on a form. there is then a box called [Total] what appears in the textbox called [total] is what I want to then store the field in my order table.
It would be better to create a query, and then have whomever/whatever needs that calculated value use the query instead of the table directly.
HOWEVER, you could create a trigger on the form on something like the OnCurrent event, and then have that trigger execute the following.
me.txtCalcField = me.subtotal - me.discount + me.delivery
However, that would fire any time you changed records. It might be slightly better to check if it's set first
if isnull me.txtCalcField then
me.txtCalcField = me.subtotal - me.discount + me.delivery
endif
But then if its components values get reset, your value doesn't get updated. In which case, you could put triggers on those fields as well (AfterUpdate events perhaps).
OR
You could check the value's aren't different each time.
if isnull me.txtCalcField then
if me.txtCalcField <> me.subtotal - me.discount + me.delivery then
me.txtCalcField = me.subtotal - me.discount + me.delivery
endif
endif
(which probably won't work properly with Reals, but might with Currency)
Of course, that's all assuming the only way the components of txtCalcField will be updated is through the form - forever.
It's all a dog's breakfast - in which case, I direct you back to my first statement - use a query.