yq - inserting JSON as a raw string - json

I am writing a GitHub Action that does some CD and it uses yq to insert environment variables into a yaml file for deployment.
I'm trying to read a JSON from a GH secret that will eventually be read from env and loaded into python, where said string will be evaluated as a dictionary.
Running this in a terminal, for example:
yq -i '.value="{\"web\": \"test\"}"' test.yaml
Gives me:
value: '{"web": "test"}'
But in a Github Action, where I am doing this:
env:
JSON="{\"web\": \"test\"}"
...
- name: test
run : |
yq -i '
.value=strenv(JSON)
' deployment.yaml
Gives me:
Error: Bad expression, please check expression syntax
Doing other variations of that string, e.g. '{\"web\": \"test\"}', '\"{\"web\": \"test\"}\"' etc also gives me the same error.
I've tried searching on the yq repository and consulted the documentation but can't seem to find what I am looking for.
To summarise, my problem is that I want to read a JSON string as a string when it is evaluated by yq.

One of yq users has recently contributed to yq's github action docs regarding using env variables in github actions - it may help here:
- name: Get an entry with a variable that might contain dots or spaces
id: get_username
uses: mikefarah/yq#master
with:
cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
- name: Reuse a variable obtained in another step
run: echo ${{ steps.get_username.outputs.result }}
See https://mikefarah.gitbook.io/yq/usage/github-action for more info.
Disclaimer: I wrote yq

Related

How to safely write json output to a file in a github workflow?

I have a github workflow that obtains information about all Github Releases in my repo and then processes the JSON response using jq. The issue is the shell code I have doesn't handle single-quotes in the JSON data (which it does have sometimes). How can I pipe out steps.release.outputs.data safely to a file without the quotations and other characters being interpreted by the shell?
- name: Get Release Info
uses: octokit/request-action#v2.x
id: release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
route: GET /repos/{org_repo}/releases/tags/{tag}
- name: Get Information from Release
run: |
echo '${{ steps.release.outputs.data }}' > release.json
jq -r '.assets[].browser_download_url' release.json > assets.txt
jq -r '.body' release.json > changelog.txt
The part that fails above is the line with echo in the second step. Because the steps.release.outputs.data content has single-quotes in it, that breaks it.
Note that, I only write the JSON data to a file (release.json in the example above) in an attempt to bypass shell processing special characters in the data. If there's a better way to do this without writing to a file I'd prefer that. The part that makes this challenging is that the response JSON gets placed into the final shell script as a literal string instead of as a bash variable.
You can use an environment variable:
- name: Get Information from Release
env:
DATA: ${{ steps.release.outputs.data }}
run: |
jq -r '.assets[].browser_download_url' <<<"$DATA" > assets.txt
jq -r '.body' <<<"$DATA" > changelog.txt

How to fix warning format JSON deprecation in cucumber 4.1.0 gem?

I've recently updated to the latest Ruby cucumber gem and now getting the following warning:
WARNING: --format=json is deprecated and will be removed after version 5.0.0.
Please use --format=message and stand-alone json-formatter.
json-formatter homepage: https://github.com/cucumber/cucumber/tree/master/json-formatter#cucumber-json-formatter.
I'm using json output later for my reporting. In my cucumber.yml I have the following default profile:
default:
-r features
--expand -f pretty --color
-f json -o reports/cucumber.json
According to the reference https://github.com/cucumber/cucumber/tree/master/json-formatter#cucumber-json-formatter they say to use something like
cucumber --format protobuf:cucumber-messages.bin
cat cucumber-messages.bin | cucumber-json-formatter > cucumber-results.json
and
Trying it out
../fake-cucumber/javascript/bin/fake-cucumber \
--results=random \
../gherkin/testdata/good/*.feature \ |
go/dist/json-formatter-darwin-amd64
But it's not really clear how to do that.
I guess you need to change your cucumber profile to produce protobuf output instead of json, and then add a step to post-process the file into the json you want?
I'd assumed that the 'Trying it out' above was your output from actually trying it out, rather than just a straight cut and paste from the formatter's Github page...

GitHub Actions:secret env is empty

I encountered issue that env defined by secret env is empty. I want to use secret env in run syntax. I defined env of secret env like here.
- name: Deploy
env:
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
run: |
date_time=`date +%Y%m%d%H%M%S`
IMAGE=gcr.io/$GCP_PROJECT_ID/web-api-server:$date_time
but $GCP_PROJECT_ID is empty.
invalid argument "gcr.io//web-api-server:20200718163842" for "-t, --tag" flag: invalid reference format
See 'docker build --help'
of course. I confirmed that $GCP_PROJECT_ID is defined at a secret.
The reason is here.
Organization secrets can only be used by public repositories on your plan.
If you would like to use organization secrets in a private repository, you will need to upgrade your plan.

Github actions echo command not creating file

I'm in the process of moving a working CircleCI workflow over to Github Actions.
I'm running:
runs-on: ubuntu-latest
container:
image: google/cloud-sdk:latest
I run the following command:
echo ${{ secrets.GCLOUD_API_KEYFILE }} > ./gcloud-api-key.json
Before running this command, gcloud-api-key.json has not yet been created. This command works in CircleCI but in Github Actions I get the error:
/__w/_temp/asd987as89d7cf.sh: 2: /__w/_temp/asd987as89d7cf.sh: type:: not found
Does anyone know what this error means?
The reason was because my secret key was more then 1 line long. Once I made it one line it worked.
In order to use secrets which contain more than just one line (like secret jsons) one has to save the base64 encoded secret in Github which makes it one line.
On linux the encoding is done via:
cat mysecret.json | base64
Then in the action you need to decode it using
echo ${{ secrets.YOUR_SECRET }} | base64 -d > secret.json

Proper syntax for adding Rundeck Ansible "Extra Vars"

Im using Rundeck 3.0.7 with Ansible 2.7 and cant figure out the correct syntax to pass variables to my Ansible playbook. If I run it from the command line it works fine.
ansible-playbook test-playbook.yml -i hosts -e "FirstName=John LastName=Doe OfficePhone=365"
However when I add those vars to the "Extra Variables" section of the Rundeck Job I add the following and it doesnt work.
-e "FirstName=John LastName=Doe OfficePhone=365"
Does anyone know the proper syntax?
In your workflow define your playbook extra arguments using options like:
-e "test1=${option.test1} test2=${option.test2}"
That way you get the values of options to those variable names in the arguments to ansible-playbook.
You can also use the "Extra Variables" section with the yaml syntax e.g.
FirstName: John
LastName: Doe
OfficePhone: 365
This can also be done with variables passed from Rundeck to Ansible:
Extra Variables:
FirstName: ${option.firstname}
LastName: ${option.lastname}
OfficePhone: ${option.phone}
Then of course you need the Rundeck options defined.
Source