Hi I have a salt state file who has a ".sls" extension and it is using jinja template
There I am creating a list in jinja template. Problem is when I run that file then list converts into JSON which convert double quotation around each element into single quotation which causes error.
{%- set opentsdb_hosts = [] -%}
{%- for host in salt['pnda.opentsdb_hosts']() -%}
{%- do opentsdb_hosts.append("http://%s:%s" % (host, opentsdb_port)) -%}
{%- endfor -%}
As you can see, I am creating a list whose name is opentsdb_hosts. Each element inside opentsdb_hosts should have a double quotation around. But it is doing single quotation.
Please help.
You must be outputting opentsdb_hosts directly using {{ opentsdb_hosts }}, which outputs it in a python-friendly format, one that usually prefers single quotes over double quotes (although both quotation marks work in python as valid string literals).
To output opentsdb_hosts as json, simply use the json renderer provided by salt instead, i.e. {{ opentsdb_hosts | tojson }}.
You may use this regex_replace("'", '"') to replace the single quote with double quotes
Related
I've been using Jinja and DBT for a month now, and despite reading a lot about it, I didn't quite figure out how to create a list from another, using a simple for loop like I would in Python.
Just a toy example:
{%- set not_wanted_columns = ['apple', 'banana'] -%}
{%- set all_columns = ['kiwi', 'peach', 'apple', 'banana', 'apricot', 'pineapple'] -%}
What I want is a list as so:
{% set filtered_columns = ['kiwi', 'peach', 'apricot', 'pineapple'] %}
Naturally, I don't want to manually write this result because the full list might be dynamic or too long. I'm not even sure if Jinja does actually support this, although I do think this is a common problem.
As you have probably read from the documentation:
Please note that assignments in loops will be cleared at the end of the iteration and cannot outlive the loop scope. Older versions of Jinja had a bug where in some circumstances it appeared that assignments would work. This is not supported.
Source: https://jinja.palletsprojects.com/en/3.1.x/templates/#for
And I guess when you are speaking about
using a simple for loop like I would in Python
What you mean here is using a list comprehension.
So, as showed in the documentation, Jinja is using filter to achieve this:
Example usage:
{{ numbers|select("odd") }}
{{ numbers|select("divisibleby", 3) }}
Similar to a generator comprehension such as:
(n for n in numbers if test_odd(n))
(n for n in numbers if test_divisibleby(n, 3))
Source: https://jinja.palletsprojects.com/en/3.1.x/templates/#jinja-filters.select
There are actual four of those filter acting as generator comprehension:
reject
rejectattr
select
selectattr
So, in your case, a reject filter would totally do the trick:
{%- set filtered_columns = all_columns
| reject('in', not_wanted_columns)
| list
-%}
But, if you really want, you could also achieve it in a for:
{%- for column in all_columns if column not in not_wanted_columns -%}
{% do filtered_columns.append(column) %}
{%- endfor -%}
The do statement being a way to use list.append() without oddities being printed out: https://jinja.palletsprojects.com/en/3.1.x/templates/#expression-statement.
I am facing trouble in sending data from an azure web api having a legitimate backslash(\). Data field is user id which is of following pattern:
Domain\UserId
I want to store it in the database as it is. But Dot liquid doesn't process it.
I tried using escape, escape_once and replace
{{ body.requestor | escape_once }}
{{ body.requestor | escape }}
{{ body.requestor | replace "\", "\\"}}
but none of them worked. I cant ask caller of my web api to pass the user id with two backslashes - \\. I have to make a change in my web api to accept the user id's as they are.
Any inputs/pointers are appreciated.
Am I too late to the party? But here is the answer. First - Replace is case-sensitive. Then you need to use colon ":" fro parameters. And third, I have no explanation but I suspect that it takes item to find with escape and item to replace with, without escape. Here is the program
string templateString =
#"Nothing: '{{ k3 }}'
Replace with dash: '{{ k3|Replace:""\\"", ""-"" }}'
Replace with double slash: '{{ k3|Replace:""\\"", ""\\"" }}'";
Template.NamingConvention = new CSharpNamingConvention();
var t = Template.Parse(templateString);
string output = t.Render(Hash.FromDictionary(new Dictionary<string, object>() {{ "k3", "Domain\\user" } }));
Console.WriteLine(output);
Output:
Nothing: 'Domain\user'
Replace with dash: 'Domain-user'
Replace with double slash: 'Domain\\user'
This may be a bug in dot Liquid implementation of escape standard filter, or arguably a point where the Shopify specification is too vague hence implementations will differ. dot Liquid is using .NET RegEx replace causing "\" for pattern matching to be interpreted as the beginning of the pattern and an escape per https://learn.microsoft.com/en-us/dotnet/standard/base-types/character-escapes-in-regular-expressions
So you need {{ body.requestor | replace "\\", "\\"}}
(!)
The pattern to search for interprets the escape (so it's a single \ been matched) while the replacement string is not interpreting the escape (so it's the actual double \\ string).
This question appear when I'm looking for the same question, only for the ruby gem liquid implementation. David Burg's answer give me hint, and this is what works:
{{ "yes \ no" | replace "\", "\\\" }}
will replace the single backlash with double backlash
You nearly got it, the correct solution is:
{{ body.requestor | replace "\", "\\\\"}}
One backslash for the first argument and four for the 2nd.
The first argument is literal.
The 2nd argument gets parsed and because we want two backslashes we need to escape each one of them with another backslash. Making a total of four backslashes.
Is it possible to convert a folder of csv files to json as part of a Jekyll workflow? I currently use a python script to do this but would like to do it entirely within Jekyll
You can call your python script as part of the build process, but you'll have to make a teeny tiny plugin to do it. This also assumes you're not using github pages, because they don't like plugins.
make a _plugins directory in your site root.
Inside that directory, create csv_to_json.rb
In that ruby file, call your python script with the following code:
# The following line tells jekyll to run everything between 'do' and 'end'
# when it finishes writing the site to disk:
Jekyll::Hooks.register :site, :post_write do |_site|
# Backticks are one way to call shell commands from ruby:
`python your_script_here.py` # replace with the correct filename
end
This is untested code. Relevant documentation here and here
There are quite a few ways to do this, but I think that's the simplest for your case.
You can :
1 - Store your csv in _data/foldername (eg : _data/members) see : Jekyll's data files
2 - Put all your datas in a new array with concat filter
{% comment %} ### Create an empty array{% endcomment %}
{% assign all-members = "" | split: "" %}
{% for part in site.data.members %}
{% assign all-members = all-members | concat: part[1] %}
{% endfor %}
3 - Output datas unsing the jsonify filter : {{ all-members | jsonify }}
An all in one, a members.json file can look like :
---
layout: null
---
{% assign all-members = "" | split: "" %}
{%- for part in site.data.members %}
{% assign all-members = all-members | concat: part[1] %}
{% endfor -%}
{{ all-members | jsonify }}
Thank you to both of you. I have gone the plugin route as this is the easiest and I may need to develop another plugin at some point so I may as well learn how to make one.
When iterating through an array of files in the _data folder, what is the default criteria for sorting the files?
At first I was expecting it to be sorted alphabetically, but after some testing I realized it was not. Still, I couldn't figure out what was the criteria being used to sort the files.
{%- for file in site.data.folder -%}
{{ file | inspect }}
<br />
<br />
{%- endfor -%}
From what I understood file is an array containing the filename as the first element and the data as the second element, so I'm not sure using sort with any property name would work. When I tried I had the error message:
Liquid Exception: no implicit conversion of String into Integer
When using sort with no arguments, I could return the files sorted by filename alphabetic order:
{%- assign files = site.data.folder | sort -%}
{%- for file in files -%}
{{ file | inspect }}
<br />
<br />
{%- endfor -%}
So my questions are:
What is the default sorting criteria for _data files?
Is sorting in relation to an object property possible? (I'm thinking the issue with that one is having an array and not the pure objects when you access site.data.folder)
Example:
After creating the default Jekyll page, I created the _data/folder directory, where I'd include 5 random .json files:
_data/folder/a.json
_data/folder/b.json
_data/folder/c.json
_data/folder/d.json
_data/folder/e.json
Each of them have the following content:
_data/folder/a.json:
{"name":"Mike"}
_data/folder/b.json:
{"id":"4343"}
_data/folder/c.json:
[{"age":"29"},{"job":"journalist"}]
_data/folder/d.json:
{"name":"John"}
_data/folder/e.json
{"haircolor":"green"}
With those files in place, I created a page named page.html on the root directory with:
---
---
<pre>{{ site.data.folder | inspect }}</pre>
<br />
<br />
{%- for file in site.data.folder -%}
<pre>{{ file | inspect }}</pre>
<br />
{%- endfor -%}
And the output of that page was:
{"e"=>{"haircolor"=>"green"}, "c"=>[{"age"=>"29"}, {"job"=>"journalist"}], "d"=>{"name"=>"John"}, "a"=>{"name"=>"Mike"}, "b"=>{"id"=>"4343"}}
["e", {"haircolor"=>"green"}]
["c", [{"age"=>"29"}, {"job"=>"journalist"}]]
["d", {"name"=>"John"}]
["a", {"name"=>"Mike"}]
["b", {"id"=>"4343"}]
The files were not ordered alphabetically, but instead in some apparently random order. I can get them in alphabetical order by using:
---
---
<pre>{{ site.data.folder | sort | inspect }}</pre>
<br />
<br />
{%- assign folder = site.data.folder | sort -%}
{%- for file in folder -%}
<pre>{{ file | inspect }}</pre>
<br />
{%- endfor -%}
Output:
[["a", {"name"=>"Mike"}], ["b", {"id"=>"4343"}], ["c", [{"age"=>"29"}, {"job"=>"journalist"}]], ["d", {"name"=>"John"}], ["e", {"haircolor"=>"green"}]]
["a", {"name"=>"Mike"}]
["b", {"id"=>"4343"}]
["c", [{"age"=>"29"}, {"job"=>"journalist"}]]
["d", {"name"=>"John"}]
["e", {"haircolor"=>"green"}]
But it's still unclear what is the ordering criteria on the call without sort.
Going from #ashmaroli's assumption that this was not a Jekyll's issue, I started making a little bit of research about file ordering and ran into the following resources:
File ordering behavior while using Dir on Ruby
Indeterministic File order using Dir
The link describes a counter intuitive behavior when loading multiple dependencies. If the order the files are loaded matter the shortcut below could result in they being loaded in a different order than the expected.
Dir[File.join(File.dirname(__FILE__), 'example/*.rb')].each{ |f| require f }
This is apparently due to the underlying glob system call according to the answer in the link.
Python glob ordering
How is Pythons glob.glob ordered?
In the SO question above, the user is asking why the returned glob file order in Python is different than the order on the output of ls -l. Even though the question is about Python and not Ruby, the underlying call to the OS is likely the same. The OS is not required to deliver the files in any order, so they should be sorted after the call.
The first answer states that if you run ls -U you get the unordered list of files, which matches the order I have here when I make a list of _data objects on Jekyll without sorting. So this is most likely the cause of the weird ordering: it's OS dependent.
Since Jekyll orders the _post files, I think it wouldn't be a major issue to order _data files by default as well, to avoid any confusion. But as it was stated before in the question itself, it can be easily done with the sort filter.
I have a html file in a non-english langauge ( മലയാളം.html ). If I try to include this html file using the below code, I am getting the below shown error.
{% include blog_comments/മലയാളം.html %}
By the way I am using Jekyll. But I believe the error is from Liquid
Regenerating: 1 file(s) changed at 2016-05-05 09:49:14 Liquid
Exception: Invalid syntax for include tag. File contains invalid
characters or sequences: blog_comments/മലയാളം.html Valid syntax: {%
include file.ext param='value' param2='value' %} in
_includes/post_comments.html, included in _layouts/post.html
Does anyone know how to get the file മലയാളം.html included ?
If you read jekyll/tags/include.rb code, you can see that file names are validated against a regular expression, which main part is :
/^[a-zA-Z0-9_\/\.-]+$/
That means that includes names cannot contain something else than european alphabet letters (lower or upper case), numbers or /, _ , - and .
You will have to rename it to blog_comments/Malayalam.html ;-)
Note: You can have a page or a collection item (including posts) with a name containing മലയാളം