multi-line yaml input to mustache template outputs as JSON - json

Question

I have the following shell snippet
inputs="ingress_test_inputs.yaml"
auth_annotations=" # type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'"
echo "---" >$inputs
echo "namespace: qa" >> $inputs
echo "auth_annotations: ${auth_annotations}" >> $inputs
echo "----- Ingress inputs (${inputs}) -----"
cat $inputs
echo 'apiversion: extenstions/v1beta
kind: Ingress
metadata:
name: aname
annotations:
kubernetes.io/ingress.class: "nginx-internal"
nginx.ingress.kubernetes.io/server-snippet: |
add_header Content-Security-Policy "frame-ancestors 'self'";
{{{auth_annotations}}}
spec:
rules:
- host: bla-bla-bla.{{namespace}}.example.com' >ingress.mustache
echo "----- Raw Ingress (ingress.mustache): -----"
cat ingress.mustache
mustache $inputs ingress.mustache > ingress-1.0.yaml
echo "----- Will apply the following ingress: -----"
cat ingress-1.0.yaml
However, when I run this the output for auth_annotations seems to be converted into JSON format (with => in between elements and comma at the end) like this (see the line before spec: ) ...
----- Ingress inputs (ingress_test_inputs.yaml) -----
---
namespace: qa
auth_annotations: # type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
----- Raw Ingress (ingress.mustache): -----
apiversion: extenstions/v1beta
kind: Ingress
metadata:
name: aname
annotations:
kubernetes.io/ingress.class: "nginx-internal"
nginx.ingress.kubernetes.io/server-snippet: |
add_header Content-Security-Policy "frame-ancestors self";
{{{auth_annotations}}}
----- Will apply the following ingress: -----
apiversion: extenstions/v1beta
kind: Ingress
metadata:
name: aname
annotations:
kubernetes.io/ingress.class: "nginx-internal"
nginx.ingress.kubernetes.io/server-snippet: |
add_header Content-Security-Policy "frame-ancestors self";
{"nginx.ingress.kubernetes.io/auth-type"=>"basic", "nginx.ingress.kubernetes.io/auth-secret"=>"basic-auth", "nginx.ingress.kubernetes.io/auth-realm"=>"Authentication Required - foo"}
I would have expected my original YAML to be pasted into those lines intact. It even strips off the comments (which I don't really care about) however, this is not the behaviour I was expecting. Why does mustache treat multi-line input differently to single line input?
I have tried searching for a similar question, but have not been able to come up with an answer.
EDIT: Added a single line variable, for comparison of inputs.

Answer 1

There are several problems here:
Mustache has no idea whatsoever that ingress.mustache is a YAML file. It will not parse it as YAML and in fact assumes it is HTML (because it will escape HTML special chars with HTML entities).
The problem that arises from this is that mustache is not aware of current indentation etc when it is prompted to insert your value, so it is unable to paste your YAML while keeping the relative structure intact by adding to the original indentation in your $inputs.
Mustache uses YAML for input, but not for output. What you see is mustache's representation of the auth_annotations values. Mustache has parsed the YAML into an internal structure and has no idea that you want it to be rendered as YAML.
The problem is not single-line vs multi-line, but simple content (scalar) vs complex content (mapping).
To be able to properly indent your variables, you have to walk over the structure with mustache and insert them piece via a section. However, mustache is not able to walk over mapping values, only over sequences. So what you need to do is to make your input a sequence that is iterable via mustache:
auth_annotations=" # type of authentication
- {key: nginx.ingress.kubernetes.io/auth-type, value: basic}
# name of the secret that contains the user/password definitions
- {key: nginx.ingress.kubernetes.io/auth-secret, value: basic-auth}
# message to display with an appropriate context why the authentication is required
- {key: nginx.ingress.kubernetes.io/auth-realm, value: 'Authentication Required - foo'}"
Then, you can insert it by modifying your template, using a mustache section to iterate over your list:
echo 'apiversion: extenstions/v1beta
kind: Ingress
metadata:
name: aname
annotations:
kubernetes.io/ingress.class: "nginx-internal"
nginx.ingress.kubernetes.io/server-snippet: |
add_header Content-Security-Policy "frame-ancestors 'self'";
{{#auth_annotations}}
{{key}}: {{value}}
{{/auth_annotations}}
spec:
rules:
- host: bla-bla-bla.{{namespace}}.example.com' >ingress.mustache

Answer 2

Here's a stripped down version of the script, which demonstrates the problem.
inputs="ingress_test_inputs.yaml"
auth_annotations="
foo: bar baz
sam: jam man"
echo "namespace: qa" > $inputs
echo "auth_annotations: ${auth_annotations}" >> $inputs
echo "----- Ingress inputs (${inputs}) -----"
cat $inputs
echo '---
metadata:
annotations:
nginx/thing:
another_thing:
{{{auth_annotations}}}
spec:
rules:
- host: bla-bla-bla.{{{ namespace }}}.example.com' >ingress.mustache
echo "----- Raw Ingress (ingress.mustache): -----"
cat ingress.mustache
mustache $inputs ingress.mustache > ingress-1.0.yaml
echo "----- Will apply the following ingress: -----"
cat ingress-1.0.yaml
This gives the same problem as the original question that I posted.
----- Ingress inputs (ingress_test_inputs.yaml) -----
namespace: qa
auth_annotations:
foo: bar baz
sam: jam man
----- Raw Ingress (ingress.mustache): -----
---
metadata:
annotations:
nginx/thing:
another_thing:
{{{auth_annotations}}}
spec:
rules:
- host: bla-bla-bla.{{{ namespace }}}.example.com
----- Will apply the following ingress: -----
---
metadata:
annotations:
nginx/thing:
another_thing:
{"foo"=>"bar baz", "sam"=>"jam man"}
spec:
rules:
- host: bla-bla-bla.qa.example.com
However, if I replace "mustache" with "mo" which is the bash implementation, it works :-
----- Ingress inputs (ingress_test_inputs.yaml) -----
namespace: qa
auth_annotations:
foo: bar baz
sam: jam man
----- Raw Ingress (ingress.mustache): -----
---
metadata:
annotations:
nginx/thing:
another_thing:
{{{auth_annotations}}}
spec:
rules:
- host: bla-bla-bla.{{{ namespace }}}.example.com
----- Will apply the following ingress: -----
namespace: qa
auth_annotations:
foo: bar baz
sam: jam man
---
metadata:
annotations:
nginx/thing:
another_thing:
spec:
rules:
- host: bla-bla-bla..example.com
To my mind this suggests a bug in the Ruby gem, as it doesn't handle YAML correctly when it's part of the output.