I'm writing a dag model and my goal is to make it as generic as possible. For this, I use the jinja models coupled with the airflow connection to store the customer credentials. it works fine.
In order to go further, during my tasks, I want to be able to retrieve data in an airflow connection with the jinja models which I will then use to format another jinja model that I then need to retrieve.
For instance:
--context_param MYSQL_MONITORING_FLOW_HOST={{ conn.{{ conn.NEWCLIENTTEST_DATASOURCE.extra_dejson.MYSQL_BDD_SERVER_ID }}.host }}
{{ conn.NEWCLIENTTEST_DATASOURCE.extra_dejson.MYSQL_BDD_SERVER_ID }} tested alone returns the correct value (MYSQL_GENERIC_ID) and then {{ conn.MYSQL_GENERIC_ID.host }} also returns the correct value.
I would like it to do a double interpretation of the jinja template, is that possible?
Thank you very much for your help
cdt
Matthew
Related
I’m running my DAG with configuration JSON, which includes parameters which includes -, e.g. market-name
When I’m trying to read it using the following Jinja template:
path_prefix = f"market={{{{ params.market-name }}}}/configuration"
I’m getting the following error:
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'market'
It seems to me that Jinja doesn’t identify the full param name market-name but get the first part before the - which is market .
My questions:
Does Jinja supports having - in the middle of a param (e.g. market-name) or should I avoid it from the beginning and use market_name instead?
If Jinja doesn’t support having a - in the middle of a param
Should I escape the market-name ?
How should I escape it?
Should I use {{ params.market-name }} instead of {{{{ params.market-name }}}}?
This is not Airflow bug. I reported it in this issue and this is a problem coming from Jinja/Python.
I could have not find any official documentation in Jinja that explains it but several reports mention that Jinja uses python interpreter's identifiers (see this answer and this answer). It might be best to report this issue to Jinja add documentation about it.
You can just avoid using -.
I am currently working with the DatabricksSubmitRunOperator in Airflow. When I tried to integrate the jinja template into the operator, by using it in the "json"-parameter of the operator, I was facing an error. The problem is, that jinja returns a string but Databricks operator needs a dict type.
I already looked up the source code of the operator. "json" seems to be a template_field, which should be fine.
Is there a way to make jinja return a dict type instead of a string in this case? Or maybe another workaround?
Thanks in advance!
There is the new feature in Airflow 2.1/0 that can help with that https://airflow.apache.org/docs/apache-airflow/stable/concepts/operators.html#rendering-fields-as-native-python-objects - it uses JINJA capability of "safe evaluation" of the string and returning and object that the string represents directly in your code.
I have the following Django view method, being used to send information over to some Vue.js frontend code in details.html. I am basically wondering how do I send over the data to the vue template. How do I do JSON dumps? I think that is what I am messing up here.
def works(request):
heading_info = Heading.objects.order_by('-name')[:1]
welcome_info = Welcome.objects.order_by('-title')[:1]
skills_info = Skill.objects.order_by('position')[:5]
projects_info = Project.objects.order_by('name')[:10]
items = []
items.append({'heading_info':heading_info, 'welcome_info':welcome_info, 'path':path, 'path2':path2, 'skills_info':skills_info,'projects_info':projects_info})
# context = {}
# context["items_json"] = json.dumps(items)
context = {'heading_info':heading_info, 'welcome_info':welcome_info, 'path':path, 'path2':path2, 'skills_info':skills_info,'projects_info':projects_info}
return render(request, 'home/details.html', context)
And here is my html where I am trying to access this data.
<script type='text/javascript'>
var data = {{ projects_info|safe }};
</script>
<div id="app">
[[projects_info_vue]]
{% comment %} #Passing array as vue data. This gets rendered as QuerySet. How do I access these value in Vue. {% endcomment %}
<div class="row">
{% for project in projects_info %}
{% comment %} #Here is the array being rednered in Django. This array gets rendered as a QuerySet in Vue. {% endcomment %}
<div class="col">
{{project.name}}
</div>
{% endfor %}
</div>
</div>
<script>
var app = new Vue({
delimiters: ["[[", "]]"],
el: '#app',
data: {
projects_info_vue: '{{projects_info}}',
},
});
</script>
You're absolutely correct that at some point you will need to convert your Python object into a literal that is understandable by Javascript/Vue, and yes JSON is appropriate since your data is more complex than a single value (string, number, etc).
To convert your QuerySet into a json string, a good approach is to first convert it from a QuerySet object into a list of simple dictionaries using values() and then encode this into json. Example:
import json
j = json.dumps(list(Project.objects.order_by('name')[:10].values('name','id')))
You will need to adjust the parameters to values to include the fields you are interested in, or omit them entirely to include all fields.
Alternatively, you can use a seralizer to directly encode the QuerySet.
I believe the rest of your code should work without much tweaking once you've gotten the JSON on the template output. You can render the json directly into the component data as you are doing or alternatively as a component property.
I will note however that the typical approach to retrieving "complex" data from Django in a JSON format is via Django Rest Framework. Using your approach for simple properties is quite fine, but when you start pulling larger datasets, you'll realize some performance gains by having your Vue components make async requests to fetch the data. This comes at the cost of additional complexity, though (managing ajax requests/responses and state in your Vue code).
PS. I know it's not your question, but I wanted to point out that I believe there are shortcomings to the general approach of using Vue via script tags on your Django template, most notably missing out on the ease and convenience of Single File Components (SFCs), hot-reloading, and the optimizations of webpack. Unless your project is tiny, I recommend spending a bit of time setting up such an integration; in my experience the up-front effort is well worth it. Also, the setup isn't so tough, as I've written some articles describing the approach and built a cookiecutter to boostrap Vue+Django projects.
Good hacking!
I am working on a Symfony project and need to loop through data to populate saved fields in a form (specifically the number of bed types in a property). I save the data in the database table as a json string, as follows:
{"2":"5","3":"0","4":"0","5":"0","6":"0","7":"0"}
This JSON follows the structure "BEDID": NUMBER OF BEDS. I implemented the solution regarding decoding json in twig as stated here https://stackoverflow.com/a/14504988/5194337 but I am having trouble actually being able to access each specific value in the decoded json. I use this in the twig template to decode the json (based on the fact my data is stored in a variable called specifics and website.id references one of multiple websites owned by the user:
{% set beds = specifics[website.id].0.standardBedTypes|json_decode %}
So, once I do this, I try and access the value of each number of beds as follows:
{{ beds[standard_bed.id] }}
standard_bed being the value in the for loop. But, when I load the page I get the following error:
Impossible to access a key "2" on an object of class "stdClass" that
does not implement ArrayAccess interface.
I guess this means that the decoded value of the json is technically not an array, but I cannot think of any other method of referencing each value, so help with this is appreciated.
From the documentation here, you can pass it as an option. See the options here.
You want to pass JSON_OBJECT_AS_ARRAY
Decodes JSON objects as PHP array.
So basically you want to do:
{% set beds = specifics[website.id].0.standardBedTypes|json_decode(constant('JSON_OBJECT_AS_ARRAY')) %}
If you want to access to objects properties, you can do it using the attribute function.
I have a $scope variable scopeVar which contains a JSON Object. The JSON object has some complicated key names such as "onStatus[1]".
Is it possible to resolve such key names inside the view template so that I can use them like {{scopeVar.onStatus[1]}} or ng-bind="scopeVar.onStatus[1]" ?.
PS- I assume that using JSON keys in such a fashion is possible after reading this answer.However I am still skeptical regarding using symbols like '[' etc in key names as they may also be used to represent array elements.
If onStatus[1] is actually a property name and not the second element of the onStatus array you should use bracket notation to access the property:
{{ scopeVar['onStatus[1]'] }}
or as the expression in ngBind:
ng-bind="scopeVar['onStatus[1]']"
Use it like this in the view it works for me.
{{ scopeVar['onStatus[1]'] }}
Basically in the interpolation anything you put is treated as plain JS code so anything that works in the console of your browser will also work in the between the curly braces.