I am trying to model the following situation:
given some query, return multi-column result-set (e.g. run_query or db_utils.get_query_results_as_dict
iterate over in a case/statment
for exmaple:
{% set conditions = dbt_utils.get_query_results_as_dict("select comment, criteria from "
~ ref('the_model') %}
...
select case
{% for condition in conditions %}
when {{ condition["criteria"] }}
then {{ condition["comment"] }}
{% endfor %}
Have not been able to get this to work, any guidance appreciated.
Some ideas I tried:
get_column_values x2 and zipping them into a new list of tuples. zip not recognised
get the count(*) from the_model then trying to iterate over the range - ran into issues with types
various for conditions {% for k, v in conditions.items() %}
Was able to self resolve with the following:
{% set conditions = dbt_utils.get_query_results_as_dict("select criteria, comment from " ~ ref('reference_data') ~ " order by sequence desc") %}
with main as (
select * from {{ ref('my_other_model') }}
),
-- [NEEDS_REVIEW] there's probably a cleaner way to do this iteration - however it's interpolated result. Could do with the zip function.
comments as (
select
*,
case
{# {{- log(conditions, info=True) -}} #}
{%- for comment in conditions.COMMENT -%}
when {{ conditions.CRITERIA[loop.index0] }}
then '{{ comment }}'
{% endfor %}
end as comment
from main
)
select * from comments
The gotchas:
this was on snowflake, so the keys returned by the function will be up-cased as that is how I loaded the data.
Using the loop.index0 to get the current iteration of the loop and index into the other collection of tuples (in this case CRITERIA).
i added a SEQUENCE key to my reference data just to ensure consistent rendering by using that to order. The criteria do overlap a-little bit so this was important.
Related
In my Jinja template, model.DataType value can be user defined or built in. My requirenement is if model.DataType start with the three letters ARR, then do a specific operation.
Example of values:
ARRstruct124
ARR_int123
ARR123123
CCHAR
UUINT
etc.
{% set evenDataType = model.eventDataType %}
{%if evenDataType | regex_match('^ARR', ignorecase=False) %}
// do the operation
{%else%}
// do the operation
{% endif %}
With this template, I am getting the error
{%if evenDataType | regex_match('^ARR', ignorecase=False) %}
jinja2.exceptions.TemplateAssertionError: no filter named 'regex_match'
There is indeed no regex_match filter in the Jinja builtin filters. You might have found some examples using it, but this is an additional filter provided by Ansible, so it won't work outside of Ansible.
This said, your requirement does not need a regex to be fulfilled, you can use the startswith() method of a Python string.
So, you template should be:
{% set evenDataType = model.eventDataType %}
{% if evenDataType.startswith('ARR') %}
`evenDataType` starts with 'ARR'
{% else %}
`evenDataType` does not starts with 'ARR'
{% endif %}
I'm new to DBT and Jinja and wondering if it is possible to dynamically define a list using a SQL query. Instead of manually declaring the items of the list like:
{% set myOrders = [123,234, 345, 456, 567] %}
Define the list with a SQL query, something like this:
{% set myOrders = SELECT DISTINCT OrderNum FROM OrdersTable ORDER BY OrderNum %}
Is this possible?
Thanks!
Yes! Not quite as you've written it, but this is supported.
First, a note that this is inherently difficult because DBT typically runs in two phases:
templates are compiled to make actual SQL queries (i.e. all the Jinja gets executed)
the compiled SQL queries are executed
But there is a construction, {% if execute %}, that allows you to defer compilation to the execution stage.
Straightforwardly adapting the example in the docs for your use case:
{% set my_orders_query %}
SELECT DISTINCT OrderNum
FROM {{ ref('OrdersTable') }}
ORDER BY OrderNum
{% endset %}
{% set rows = run_query(my_orders_query) %}
{% if execute %}
{# Return the first column #}
{% set myOrders = rows.columns[0].values() %}
{% else %}
{% set myOrders = [] %}
{% endif %}
I am creating a state which will trigger some module if both of the grains are not matching. I have tried several options, but no luck.
Based on comparison, False will trigger my module and that module will change GRAIN_B value to match with GRAIN_A. So during every highstate my module will not get triggered, unless there is a change in GRAIN_A.
Any suggestions please.
I have tried several jinja expressions.
{% if grains['GRAIN_A'] not in grains.get('GRAIN_B','None') %}
{% set GRAIN_B = grains.get('GRAIN_B','None') %}
{% if grains['GRAIN_A'] != {{ GRAIN_B }} %}```
```{% if grains['GRAIN_A'] not in grains.get('GRAIN_B','None') %}
MY_MODULE:
module.run:
- func: MYMODULE.FUNCTION_A
{% endif %}```
Issue fixed, there is a \n character in my GRAIN_A output, which makes the evaluation condition fails.
This condition works already.
{% if grains['GRAIN_A'] not in grains.get('GRAIN_B','None') %}
MY_MODULE:
module.run:
- func: MYMODULE.FUNCTION_A
{% endif %}
I have a collection in Jekyll which I want to sort. Sorting by title is easy of course.
<ul>
{% for note in site.note | sort: "title" %}
<li>{{note.path | git_mod }}: {{ note. title }}</li>
{% endfor %}
</ul>
I want to sort by date. But since collections don't have a date, I have a custom Liquid filter which takes the path of the item, and gets its last modified time in Git. You can see that in the code above, where I pass the path to git_mod. I can verify that this works, because when I print out the list, I get the correct last modified times, and it is a full date. (In practice, I also pass it to date_as_string.)
But I can't sort by that value because Liquid doesn't know about it, since it is a value already in each item in the site.note collection. How can I sort by that value? I was thinking something like this, but it doesn't work:
<ul>
{% for note in site.note | sort: path | date_mod %}
<li>{{note.path | git_mod }}: {{ note. title }}</li>
{% endfor %}
</ul>
I've also tried variants like: {% for note in site.note | sort: (note.path | git_mod) %}
None of these throw an error, but none of them work either.
This is a case where you can use Jekyll hooks.
You can create a _plugins/git_mod.rb
Jekyll::Hooks.register :documents, :pre_render do |document, payload|
# as posts are also a collection only search Note collection
isNote = document.collection.label == 'note'
# compute anything here
git_mod = ...
# inject your value in dacument's data
document.data['git_mod'] = git_mod
end
You then will be able to sort by git_mod key
{% assign sortedNotes = site.note | sort: 'git_mod' %}
{% for note in sortedNotes %}
....
Note that you cannot sort in a for loop. You first need to sort in an assign, then loop.
I have a query result on my views and i drop to my template, on my template i want to display values of my table, so i use
{{myTable.0}} and {{myTable.1}}
and i have the result of my query, but i want to display this values with a variable like {{myTable.x}} where x =0 to use in a loop for.
But it doesn't work, so if you have any solution.
Seems like myTable is a list an you want to iterate through this list. If so then you can use the {% for %} template tag:
{% for val in myTable %}
{{ val }}
{% endfor %}