I want to add the contents of a shell script into the body of pkg_preinst_${PN} or pkg_postinst_${PN} function (BitBake recipe of a software package).
For example, let's consider this "PREINST" shell script:
$ cat PREINST
#! /bin/sh
chmod +x /usr/bin/mybin
Executing a simple 'cat' command inside pkg_preinst function doesn't work:
pkg_preinst_${PN}() {
cat ${S}/path/to/PREINST
}
In this way, the contents for the .spec file for the generated rpm package are not the expected:
%pre
cat /Full/Path/To/Variable/S/path/to/PREINST
As you can see, %pre section doesn't include real contents of PREINST file, just includes the 'cat' command.
Is it possible to include the contents of PREINST file into the generated .spec file in some way?
Thank you in advance!
Finally I solved this issue by prepending this code to the do_package task:
do_package_prepend() {
PREINST_path = "${S}/${MYMODULE}/PREINST"
POSTINST_path = "${S}/${MYMODULE}/POSTINST"
PREINST = open(PREINST_path, "r")
POSTINST = open(POSTINST_path, "r")
d.setVar("pkg_preinst", PREINST.read())
d.setVar("pkg_postinst", POSTINST.read())
}
It modifies "pkg_preinst" and "pkg_postinst" keys in 'd' global dictionary with the content of each PREINST and POSTINST file as value. Now it works! :)
Related
I have a Snakemake pipeline where I get my input/output paths for my file folders from a json file and use the expand function to obtain the paths.
import json
with open('config.json', 'r') as f:
config = json.load(f)
wildcard = ["1234", "5678"]
rule them_all:
input:
expand('config["data_input"]/data_{wc}.tab', wc = wildcard)
output:
expand('config["data_output"]/output_{wc}.rda', wc = wildcard)
shell:
"Rscript ./my_script.R"
My config.json is
{
"data_input": "/very/long/path",
"data_output": "/slightly/different/long/path"
}
While trying to make a dry run, though, I get the following error:
$ snakemake -np
Building DAG of jobs...
MissingInputException in line 12 of /path/to/Snakefile:
Missing input files for rule them_all:
config["data_input"]/data_1234.tab
config["data_input"]/data_5678.tab
The files are there and their path is /very/long/path/data_1234.tab.
This is probably a low-hanging fruit, but what am I doing wrong in the syntax for the expansion? Or is it the way I call the json file?
expand() does not interpret access to dictionaries for its first argument while expanding the path with quotation marks, so this operation with expand() has to be done in a wildcard.
The correct syntax, in this case, would be e.g.
expand('{input_folder}/data_{wc}.tab', wc = wildcard, input_folder = config["data_input"])
I have 100,000's of lined json files that I need to split out based on whether or not, they contain a certain value for an attribute and then I need to convert them into valid json that can be read in by another platform.
I'm using a batch file to do this and I've managed to convert them into valid json using the following:
for /r %%f in (*.json*) do jq -s -c "." "%%f" >> "C:\Users\me\my-folder\%%~nxf.json"
I just can't figure out how to only copy the files that contain a certain value. So logic should be:
Look at all the files in the folders and sub solders
If the file contains an attribute "event" with a value of "abcd123"
then: convert the file into valid json and persist it with the same filename over to location "C:\Users\me\my-folder\"
else: ignore it
Example of files it should select:
{"name":"bob","event":"abcd123"}
and
{"name":"ann","event":"abcd123"},{"name":"bob","event":"8745LLL"}
Example of files it should NOT select:
{"name":"ann","event":"778PPP"}
and
{"name":"ann","event":"778PPP"},{"name":"bob","event":"8745LLL"}
Would love help to figure out the filtering part.
Since there are probably more file names than will fit on the command line, this response will assume a shell loop through the file names will be necessary, as the question itself envisions. Since I'm currently working with a bash shell, I'll present a bash solution, which hopefully can readily be translated to other shells.
The complication in the question is that the input file might contain one or more valid JSON values, or one or more comma-separated JSON values.
The key to a simple solution using jq is jq's -e command-line option, since this sets the return code to 0 if and only if
(a) the program ran normally; and (b) the last result was a truthy value.
For clarity, let's encapsulate the relevant selection criterion in two bash functions:
# If the input is a valid stream of JSON objects
function try {
jq -e -n 'any( inputs | objects; select( .event == "abcd123") | true)' 2> /dev/null > /dev/null
}
# If the input is a valid JSON array whose elements are to be checked
function try_array {
jq -e 'any( .[] | objects; select( .event == "abcd123") | true)' 2> /dev/null > /dev/null
}
Now a comprehensive solution can be constructed along the following lines:
find . -type f -maxdepth 1 -name '*.json' | while read -r f
do
< "$f" try
if [ $? = 0 ] ; then
echo copy $f
elif [ $? = 5 ] ; then
(echo '['; cat "$f"; echo ']') | try_array
if [ $? = 0 ] ; then
echo copy $f
fi
fi
done
Have you considered using findstr?
%SystemRoot%\System32\findstr.exe /SRM "\"event\":\"abcd123\"" "C:\Users\me\my-folder\*.json"
Please open a Command Prompt window, type findstr /?, press the ENTER key, and read its usage information. (You may want to consider the /I option too, for instance).
You could then use that within another for loop to propagate those files into a variable for your copy command.
batch-file example:
#For /F "EOL=? Delims=" %%G In (
'%SystemRoot%\System32\findstr.exe /SRM "\"event\":\"abcd123\"" "C:\Users\me\my-folder\*.json"'
) Do #Copy /Y "%%G" "S:\omewhere Else"
cmd example:
For /F "EOL=? Delims=" %G In ('%SystemRoot%\System32\findstr.exe /SRM "\"event\":\"abcd123\"" "C:\Users\me\my-folder\*.json"') Do #Copy /Y "%G" "S:\omewhere Else"
I'm a beginner with shell scripting and i have some issues while a jenkins job parametrized. I want to write all parameters of jenkins job pipeline build with parameters into a JSON file using ${params}!
In my case i have 4 parameters(apis:multi-select,name:string,version:single-select and status:Boolean), there is the script Jenkinsfile
pipeline {
agent any
stages {
stage('Build') {
steps {
script{
sh "./test.sh ${params}"
}
}
}
}
}
Content of test.sh
#!/bin/bash
echo $# > file.json
The output in jenkins
+ ./test.sh [apis:dev,qa,prod, name:CC, version:g3, status:true]
Result in file.json
SyntaxError: JSON.parse: unexpected character at line 1 column 2 of the JSON data
My question is how format the output to obtain a clean result in file.json ? please i need help.
Add this to the top of your script:
import groovy.json.JsonOutput
Then use this line instead of sh "./test.sh ${params}":
writeFile file: 'params.json', text: JsonOutput.toJson(params)
This uses a Groovy library and a native Jenkins method for writing files, which means you don't need to use the sh method.
I'm using the gulp-run plugin to get the git hash using the following code:
run('git log -1 --pretty=format:%H').exec();
I can add a pipe to save the output to a file, like so:
run('git log -1 --pretty=format:%H').exec().pipe(gulp.dest('some/dir'));
From there, I can read in the contents of the file using readFile() to get the git hash.
Is there a way for me to skip this step, and get the output of gulp-run into a variable directly?
Two comments:
I would recommend
git rev-parse HEAD
Instead of git log -1 --pretty=format:%H
And you can easily do this with https://github.com/mgutz/execSync, gulp-run is not necessary.
var sh = require('execSync');
var revision = sh.run('git rev-parse HEAD');
console.log(revision); // => 93004330f14fd502e1568a0c2c1a645eae269e1b
I have 2 environments variables :
echo $FRONT1_PORT_8080_TCP_ADDR # 172.17.1.80
echo $FRONT2_PORT_8081_TCP_ADDR # 172.17.1.77
I want to inject them in a my default.vcl like :
backend front1 {
.host = $FRONT1_PORT_8080_TCP_ADDR;
}
But I got an syntax error on the $ char.
I've also tried with user variables but I can't define them outside vcl_recv.
How can I retrieve my 2 values in the VCL ?
I've managed to parse my vcl
backend front1 {
.host = ${FRONT1_PORT_8080_TCP_ADDR};
}
With a script:
envs=`printenv`
for env in $envs
do
IFS== read name value <<< "$env"
sed -i "s|\${${name}}|${value}|g" /etc/varnish/default.vcl
done
Now you can use the VMOD Varnish Standard Module (std) to get environment variables in the VCL, for example:
set req.backend_hint = app.backend(std.getenv("VARNISH_BACKEND_HOSTNAME"));
See documentation: https://varnish-cache.org/docs/trunk/reference/vmod_std.html#std-getenv
Note: it doesn't work for backend configuration, but could work elsewhere. Apparently backends are expecting constant strings and if you try, you'll get Expected CSTR got 'std.fileread'.
You can use the fileread function of the std module, and create a file for each of your environment variables.
before running varnishd, you can run:
mkdir -p /env; \
env | while read envline; do \
k=${envline%%=*}; \
v=${envline#*=}; \
echo -n "$v" >"/env/$k"; \
done
And then, within your varnish configuration:
import std;
...
backend front1 {
.host = std.fileread("/env/FRONT1_PORT_8080_TCP_ADDR");
.port = std.fileread("/env/FRONT1_PORT_8080_TCP_PORT");
}
I haven't tested it yet. Also, I don't know if giving a string to the port configuration of the backend would work. In that case, converting to an integer should work:
.port = std.integer(std.fileread("/env/FRONT1_PORT_8080_TCP_PORT"), 0);
You can use use echo to eval strings.
Usually you can do something like:
VAR=test # Define variables
echo "my $VAR string" # Eval string
But, If you have the text in a file, you can use "eval" to have the same behaviour:
VAR=test # Define variables
eval echo $(cat file.vcl) # Eval string from the given file
Sounds like a job for envsubst.
Just use standard env var syntax in your config $MY_VAR and ...
envsubst < myconfig.tmpl > myconfig.vcl
You can install with apt get install gettext in Ubuntu.