I have 2 environments variables :
echo $FRONT1_PORT_8080_TCP_ADDR # 172.17.1.80
echo $FRONT2_PORT_8081_TCP_ADDR # 172.17.1.77
I want to inject them in a my default.vcl like :
backend front1 {
.host = $FRONT1_PORT_8080_TCP_ADDR;
}
But I got an syntax error on the $ char.
I've also tried with user variables but I can't define them outside vcl_recv.
How can I retrieve my 2 values in the VCL ?
I've managed to parse my vcl
backend front1 {
.host = ${FRONT1_PORT_8080_TCP_ADDR};
}
With a script:
envs=`printenv`
for env in $envs
do
IFS== read name value <<< "$env"
sed -i "s|\${${name}}|${value}|g" /etc/varnish/default.vcl
done
Now you can use the VMOD Varnish Standard Module (std) to get environment variables in the VCL, for example:
set req.backend_hint = app.backend(std.getenv("VARNISH_BACKEND_HOSTNAME"));
See documentation: https://varnish-cache.org/docs/trunk/reference/vmod_std.html#std-getenv
Note: it doesn't work for backend configuration, but could work elsewhere. Apparently backends are expecting constant strings and if you try, you'll get Expected CSTR got 'std.fileread'.
You can use the fileread function of the std module, and create a file for each of your environment variables.
before running varnishd, you can run:
mkdir -p /env; \
env | while read envline; do \
k=${envline%%=*}; \
v=${envline#*=}; \
echo -n "$v" >"/env/$k"; \
done
And then, within your varnish configuration:
import std;
...
backend front1 {
.host = std.fileread("/env/FRONT1_PORT_8080_TCP_ADDR");
.port = std.fileread("/env/FRONT1_PORT_8080_TCP_PORT");
}
I haven't tested it yet. Also, I don't know if giving a string to the port configuration of the backend would work. In that case, converting to an integer should work:
.port = std.integer(std.fileread("/env/FRONT1_PORT_8080_TCP_PORT"), 0);
You can use use echo to eval strings.
Usually you can do something like:
VAR=test # Define variables
echo "my $VAR string" # Eval string
But, If you have the text in a file, you can use "eval" to have the same behaviour:
VAR=test # Define variables
eval echo $(cat file.vcl) # Eval string from the given file
Sounds like a job for envsubst.
Just use standard env var syntax in your config $MY_VAR and ...
envsubst < myconfig.tmpl > myconfig.vcl
You can install with apt get install gettext in Ubuntu.
Related
I have a json object I am reading with jq and trying to write some properties with local variables.
I am setting local variable in my shell script like so:
LOCATION_NAME="stag5"
DOMAIN_LOCATION="example.io"
I am then building out the following variable:
echo "Build New ID"
DOMAIN_NAME_BUILT="$LOCATION_NAME.$DOMAIN_LOCATION.s3-website.us-east-2.amazonaws.com"
I am trying to read my distconfig.json file and set the properties with the above variables.
tmp=$(mktemp)
jq '.Origins.Items[0].DomainName = "$DOMAIN_NAME_BUILT"' distconfig.json > "$tmp" && mv "$tmp" distconfig.json
The command is working, but it is passing in the variable as a string to my new json file.
So when I view the property in the new created json file it is being saved as "$DOMAIN_NAME_BUILT" instead of stag5.example.io.s3-website.us-east-2.amazonaws.com
How can I instead of passing the string pass the variable for $DOMAIN_NAME_BUILT and write it to the new json file
Use the --argjson option instead of parameter interpolation.
jq --argjson dnb "$DOMAIN_NAME_BUILT" \
'.Origins.Items[0].DomainName = $dnb' distconfig.json > "$tmp" &&
mv "$tmp" distconfig.json
(Your immediate issue is that parameter expansion doesn't occur inside single quotes, but building a static filter that takes an argument is safer than building a filter dynamically.)
I curl an endpoint for a json response and write the response to a file.
So far I've got a script that:
1). does the curl if the file does not exist and
2). else sets a variable
#!/bin/bash
instance="server1"
curl=$(curl -sk https://my-app-api.com | python -m json.tool)
json_response_file="/tmp/file"
if [ ! -f ${json_response_file} ] ; then
${curl} > ${json_response_file}
instance_info=$(cat ${json_response_file})
else
instance_info=$(cat ${json_response_file})
fi
The problem is, the file may exist with a bad response or is empty.
Possibly using bash until, I'd like to
(1). check (using JQ) that a field in the curl response contains $instance and only then write the file.
(2). retry the curl XX number of times until the response contains $instance
(3). write the file once the response contains $instance
(4). set the variable instance_info=$(cat ${json_response_file}) when the above is done correctly.
I started like this... then got stuck...
until [[ $(/usr/bin/jq --raw-output '.server' <<< ${curl}) = $instance ]]
do
One sane implementation might look something like this:
retries=10
instance=server1
response_file=filename
# define a function, since you want to run this code multiple times
# the old version only ran curl once and reused that result
fetch() { curl -sk https://my-app-api.com; }
instance_info=
for (( retries_left=retries; retries_left > 0; retries_left-- )); do
content=$(fetch)
server=$(jq --raw-output '.server' <<<"$content")
if [[ $server = "$instance" ]]; then
# Writing isn't atomic, but renaming is; doing it this way makes sure that no
# incomplete response will ever exist in response_file. If working in a directory
# like /tmp where others users may have write, use $(mktemp) to create a tempfile with
# a random name to avoid security risk.
printf '%s\n' "$content" >"$response_file.tmp" \
&& mv "$response_file.tmp" "$response_file"
instance_info=$content
break
fi
done
[[ $instance_info ]] || { echo "ERROR: Giving up after $retries retries" >&2; }
I have created a JSON file which in this case contains:
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"active"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"active"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"},
On other side I have a variable with values for example:
arr="10.1.1.2 10.1.1.3"
which comes from a subsequent check of the server status for example. For those values I want to change the status field to "inactive". In other words to grep the host and change its "status" value.
Expected output:
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"inactive"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"inactive"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"},
$ arr="10.1.1.2 10.1.1.3"
$ awk -v arr="$arr" -F, 'BEGIN { gsub(/\./,"\\.",arr); gsub(/ /,"|",arr) }
$1 ~ "\"(" arr ")\"" { sub(/active/,"in&") } 1' file
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"inactive"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"inactive"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"},
Here is a quick perl "wrap-around one-liner": that uses the JSON module and slurps with the -0 switch:
perl -MJSON -n0E '$j = decode_json($_);
for (#{$j->{hosts}}){$_->{status}=inactive if $_->{ipaddr}=~/2|3/} ;
say to_json( $j->{hosts}, {pretty=>1} )' status_data.json
might be nicer or might violate PBP recommendations for map:
perl -MJSON -n0E '$j = decode_json($_);
map { $_->{status}=inactive if $_->{ipaddr}=~/2|3/ } #{ $j->{hosts} } ;
say to_json( $j->{hosts} )' status_data.json
A shell script that resets status using jq would also be possible. Here's a quick way to parse and output changes to JSON using jq:
cat status_data.json| jq -r '.hosts |.[] |
select(.ipaddr == "10.1.1.2"//.ipaddr == "10.1.1.3" )' |jq '.status = "inactive"'
EDIT In an earlier comment I was uncertain whether the OP was more interested in an application than a quick search and replace (something about the phrases "On other side..." and "check on the server status"). Here is a (still simple) perl approach in script form:
use v5.16; #strict, warnings, say
use JSON ;
use IO::All;
my $status_data < io 'status_data.json';
my $network = JSON->new->utf8->decode($status_data) ;
my #changed_hosts= qw/10.1.1.2 10.1.1.3/;
sub status_report {
foreach my $host ( #{ $network->{hosts} }) {
say "$host->{hostname} is $host->{status}";
}
}
sub change_status {
foreach my $host ( #{ $network->{hosts} }){
foreach (#changed_hosts) {
$host->{status} = "inactive" if $host->{ipaddr} eq $_ ;
}
}
status_report;
}
defined $ENV{CHANGE_HAPPENED} ? change_status : status_report ;
The script reads the JSON file status_data.json (using IO::All which is great fun) then decodes it with JSON into a hash. It is hard to tell if this us a complete a solution because if you are "monitoring" host status then we should check the JSON data file periodically and compare it to our hash and then run the main body of the script one when changes have occurred.
To simulate changes occurring you can define/undefine CHANGE_HAPPENED in your environment with export CHANGE_HAPPENED=1 (or setenv if in in tcsh) and unset CHANGE_HAPPENED and the script will then either update the messages and the hash or "report". For this to be complete the data in our hash should be updated to match the the data file either periodically or when an event occurs. The status_report() subroutine could be changed so that it builds arrays of #inactive_hosts and #active_hosts when update_status() told it to do so: if ( something_happened() ) { update_status() }, etc.
Hope that helps.
status_data.json
{
"hosts":[
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"active"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"active"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"}
]
}
output:
~/ % perl network_status_json.pl
host2 is active
host3 is active
host4 is active
~/ % export CHANGE_HAPPENED=1
~/ % perl network_status_json.pl
host2 is inactive
host3 is inactive
host4 is active
Version 1:
Using a simple regex based transformation. This can be done in several ways. From the initial question, the list of ipaddr is in variable in arr. Example using a Bash env variable:
$ export var="... ..."
It would be a possible solution to provide this information by command line parameters.
#!/usr/bin/perl
my %inact; # ipaddr to inactivate
my $arr=$ENV{arr} ; # from external var (export arr=...)
## $arr=shift; # from command line arg
for( split(/\s+/, $arr)){ $inact{$_}=1 }
while(<>){ # one "json" line at the time
if(/"ipaddr":"(.*?)"/ and $inact{$1}){
s/"active"/"inactive"/}
print $_;
}
Version 2:
Using Json parser we can do more complex transformations; as the input is not real JSON we will process one line of "almost json" at the time:
use JSON;
use strict;
my ($line, %inact);
my $arr=$ENV{arr} ;
for( split(/\s+/, $arr)){ $inact{$_}=1 }
while(<>){ # one "json" line at the time
if(/^\{.*\},/){
s/,\n//;
$line = from_json( $_);
if($inact{$line->{ipaddr}}){
$line->{status} = "inactive" ;}
print to_json($line), ",\n"; }
else { print $_;}
}
#!/bin/ksh
# your "array" of IP
arr="10.1.1.2 10.1.1.3"
# create and prepare temporary file for sed action
SedAction=/tmp/Action.sed
# --- for/do generating SedAction --------
echo "#sed action" > ${SedAction}
#take each IP from the arr variable one by one
for IP in ${arr}
do
# prepare for a psearch pattern use
IP_RE="$( echo "${IP}" | sed 's/\./\\./g' )"
# generate sed action in temporary file.
# final action will be like:
# s/\("ipaddr":"10\.1\.1\.2".*\)"active"}/\1"inactive"}/;t
# escape(double) \ for in_file espace, escape(simple) " for this line interpretation
echo "s/\\\(\"ipaddr\":\"${IP_RE}\".*\\\)\"active\"}/\\\1\"inactive\"}/;t" >> ${SedAction}
done
# --- sed generating sed action ---------------
echo "${arr}" \
| tr " " "\n" \
| sed 's/\./\\./g
s#.*#s/\\("ipaddr":"&".*\\)"active"}/\\1"inactive"}/;t#
' \
> ${SedAction}
# core of the process (use -i for inline editing or "double" redirection for non GNU sed)
sed -f ${SedAction} YourFile
# clean temporary file
rm ${SedAction}
Self commented, tested in ksh/AIX.
2 way to generate the SedAction depending of action you want to do also (if any). You only need one to work, i prefer the second
This is very simple indeed in Perl, using the JSON module.
use strict;
use warnings;
use JSON qw/ from_json to_json /;
my $json = JSON->new;
my $data = from_json(do { local $/; <DATA> });
my $arr = "10.1.1.2 10.1.1.3";
my %arr = map { $_ => 1 } split ' ', $arr;
for my $item (#$data) {
$item->{status} = 'inactive' if $arr{$item->{ipaddr}};
}
print to_json($data, { pretty => 1 }), "\n";
__DATA__
[
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"active"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"active"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"}
]
output
[
{
"role" : "http",
"hostname" : "host2",
"status" : "inactive",
"ipaddr" : "10.1.1.2"
},
{
"hostname" : "host3",
"role" : "sql",
"ipaddr" : "10.1.1.3",
"status" : "inactive"
},
{
"ipaddr" : "10.1.1.4",
"status" : "active",
"hostname" : "host4",
"role" : "quad"
}
]
Is there a POSIX Compliant way to limit the scope of a variable to the function it is declared in? i.e.:
Testing()
{
TEST="testing"
}
Testing
echo "Test is: $TEST"
should print "Test is:". I've read about the declare, local, and typeset keywords, but it doesn't look like they are required POSIX built-ins.
It is normally done with the local keyword, which is, as you seem to know, not defined by POSIX. Here is an informative discussion about adding 'local' to POSIX.
However, even the most primitive POSIX-compliant shell I know of which is used by some GNU/Linux distributions as the /bin/sh default, dash (Debian Almquist Shell), supports it. FreeBSD and NetBSD use ash, the original Almquist Shell, which also supports it. OpenBSD uses a ksh implementation for /bin/sh which also supports it. So unless you're aiming to support non-GNU non-BSD systems like Solaris, or those using standard ksh, etc., you could get away with using local. (Might want to put some comment right at the start of the script, below the shebang line, noting that it is not strictly a POSIX sh script. Just to be not evil.) Having said all that, you might want to check the respective man-pages of all these sh implementations that support local, since they might have subtle differences in how exactly they work. Or just don't use local:
If you really want to conform fully to POSIX, or don't want to mess with possible issues, and thus not use local, then you have a couple options. The answer given by Lars Brinkhoff is sound, you can just wrap the function in a sub-shell. This might have other undesired effects though. By the way shell grammar (per POSIX) allows the following:
my_function()
(
# Already in a sub-shell here,
# I'm using ( and ) for the function's body and not { and }.
)
Although maybe avoid that to be super-portable, some old Bourne shells can be even non-POSIX-compliant. Just wanted to mention that POSIX allows it.
Another option would be to unset variables at the end of your function bodies, but that's not going to restore the old value of course so isn't really what you want I guess, it will merely prevent the variable's in-function value to leak outside. Not very useful I guess.
One last, and crazy, idea I can think of is to implement local yourself. The shell has eval, which, however evil, yields way to some insane possibilities. The following basically implements dynamic scoping a la old Lisps, I'll use the keyword let instead of local for further cool-points, although you have to use the so-called unlet at the end:
# If you want you can add some error-checking and what-not to this. At present,
# wrong usage (e.g. passing a string with whitespace in it to `let', not
# balancing `let' and `unlet' calls for a variable, etc.) will probably yield
# very very confusing error messages or breakage. It's also very dirty code, I
# just wrote it down pretty much at one go. Could clean up.
let()
{
dynvar_name=$1;
dynvar_value=$2;
dynvar_count_var=${dynvar_name}_dynvar_count
if [ "$(eval echo $dynvar_count_var)" ]
then
eval $dynvar_count_var='$(( $'$dynvar_count_var' + 1 ))'
else
eval $dynvar_count_var=0
fi
eval dynvar_oldval_var=${dynvar_name}_oldval_'$'$dynvar_count_var
eval $dynvar_oldval_var='$'$dynvar_name
eval $dynvar_name='$'dynvar_value
}
unlet()
for dynvar_name
do
dynvar_count_var=${dynvar_name}_dynvar_count
eval dynvar_oldval_var=${dynvar_name}_oldval_'$'$dynvar_count_var
eval $dynvar_name='$'$dynvar_oldval_var
eval unset $dynvar_oldval_var
eval $dynvar_count_var='$(( $'$dynvar_count_var' - 1 ))'
done
Now you can:
$ let foobar test_value_1
$ echo $foobar
test_value_1
$ let foobar test_value_2
$ echo $foobar
test_value_2
$ let foobar test_value_3
$ echo $foobar
test_value_3
$ unlet foobar
$ echo $foobar
test_value_2
$ unlet foobar
$ echo $foobar
test_value_1
(By the way unlet can be given any number of variables at once (as different arguments), for convenience, not showcased above.)
Don't try this at home, don't show it to children, don't show it your co-workers, don't show it to #bash at Freenode, don't show it to members of the POSIX committee, don't show it to Mr. Bourne, maybe show it to father McCarthy's ghost to give him a laugh. You have been warned, and you didn't learn it from me.
EDIT:
Apparently I've been beaten, sending the IRC bot greybot on Freenode (belongs to #bash) the command "posixlocal" will make it give one some obscure code that demonstrates a way to achieve local variables in POSIX sh. Here is a somewhat cleaned up version, because the original was difficult to decipher:
f()
{
if [ "$_called_f" ]
then
x=test1
y=test2
echo $x $y
else
_called_f=X x= y= command eval '{ typeset +x x y; } 2>/dev/null; f "$#"'
fi
}
This transcript demonstrates usage:
$ x=a
$ y=b
$ f
test1 test2
$ echo $x $y
a b
So it lets one use the variables x and y as locals in the then branch of the if form. More variables can be added at the else branch; note that one must add them twice, once like variable= in the initial list, and once passed as an argument to typeset. Note that no unlet or so is needed (it's a "transparent" implementation), and no name-mangling and excessive eval is done. So it seems to be a much cleaner implementation overall.
EDIT 2:
Comes out typeset is not defined by POSIX, and implementations of the Almquist Shell (FreeBSD, NetBSD, Debian) don't support it. So the above hack will not work on those platforms.
I believe the closest thing would be to put the function body inside a subshell.
E.g. try this
foo()
{
( x=43 ; echo $x )
}
x=42
echo $x
foo
echo $x
This is actually built into the design of POSIX function declarations.
If you would like a variable declared in the parent scope, to be accessible in a function, but leave its value in the parent scope unchanged, simply:
*Declare your function using an explicit subshell, i.e., use a
subshell_function() (with parentheses), not
inline_function() { with braces ;}
The behavior of inline grouping vs. subshell grouping is consistent throughout the entire language.
If you want to "mix and match", start with an inline function, then nest subshell functions as necessary. It's clunky, but works.
Here is a function that enables scoping:
scope() {
eval "$(set)" command eval '\"\$#\"'
}
Example script:
x() {
y='in x'
echo "$y"
}
y='outside x'
echo "$y"
scope x
echo "$y"
Result:
outside x
in x
outside x
If you'd like to journey down to Hell with me, I've made a more elaborated implementation of the eval concept.
This one automatically keeps an account of your quasi-scoped variables, can be called with a more familiar syntax, and properly unsets (as opposed to merely nulling) variables when leaving nested scopes.
Usage
As you can see, you push_scope to enter a scope, _local to declare your quasi-local variables, and pop_scope to leave a scope. Use _unset to unset a variable, and pop_scope will re-unset it when you back out into that scope again.
your_func() {
push_scope
_local x="baby" y="you" z
x="can"
y="have"
z="whatever"
_unset z
push_scope
_local x="you"
_local y="like"
pop_scope
pop_scope
}
Code
All of the gibberish variable name suffixes are to be extra-safe against name collisions.
# Simulate entering of a nested variable scope
# To be used in conjunction with push_scope(), pop_scope(), and _local()
push_scope() {
SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D=$(( $SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D + 1 ))
}
# Store the present value of the specified variable(s), allowing use in a new scope.
# To be used in conjunction with push_scope(), pop_scope(), and _local()
#
# Parameters:
# $# : string; name of variable to store the value of
scope_var() {
for varname_FB94CFD263CF11E89500036F7F345232 in "${#}"; do
eval "active_varnames_FB94CFD263CF11E89500036F7F345232=\"\${SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARNAMES}\""
# echo "Active varnames: ${active_varnames_FB94CFD263CF11E89500036F7F345232}"
case " ${active_varnames_FB94CFD263CF11E89500036F7F345232} " in
*" ${varname_FB94CFD263CF11E89500036F7F345232} "* )
# This variable was already stored in a previous call
# in the same scope. Do not store again.
# echo "Push \${varname_FB94CFD263CF11E89500036F7F345232}, but already stored."
:
;;
* )
if eval "[ -n \"\${${varname_FB94CFD263CF11E89500036F7F345232}+x}\" ]"; then
# Store the existing value from the previous scope.
# Only variables that were set (including set-but-empty) are stored
# echo "Pushing value of \$${varname_FB94CFD263CF11E89500036F7F345232}"
eval "SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARVALUE_${varname_FB94CFD263CF11E89500036F7F345232}=\"\${${varname_FB94CFD263CF11E89500036F7F345232}}\""
else
# Variable is unset. Do not store the value; an unstored
# value will be used to indicate its unset state. The
# variable name will still be registered.
# echo "Not pushing value of \$${varname_FB94CFD263CF11E89500036F7F345232}; was previously unset."
:
fi
# Add to list of variables managed in this scope.
# List of variable names is space-delimited.
eval "SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARNAMES=\"\${SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARNAMES}${varname_FB94CFD263CF11E89500036F7F345232} \""
;;
esac
unset active_varnames_FB94CFD263CF11E89500036F7F345232
done
unset varname_FB94CFD263CF11E89500036F7F345232
}
# Simulate declaration of a local variable
# To be used in conjunction with push_scope(), pop_scope(), and _local()
#
# This function is a convenience wrapper over scope_var().
#
# Can be called just like the local keyword.
# Example usage: _local foo="foofoofoo" bar="barbarbar" qux qaz=""
_local() {
for varcouple_44D4987063D111E8A46923403DDBE0C7 in "${#}"; do
# Example string: foo="barbarbar"
varname_44D4987063D111E8A46923403DDBE0C7="${varcouple_44D4987063D111E8A46923403DDBE0C7%%=*}"
varvalue_44D4987063D111E8A46923403DDBE0C7="${varcouple_44D4987063D111E8A46923403DDBE0C7#*=}"
varvalue_44D4987063D111E8A46923403DDBE0C7="${varvalue_44D4987063D111E8A46923403DDBE0C7#${varcouple_44D4987063D111E8A46923403DDBE0C7}}"
# Store the value for the previous scope.
scope_var "${varname_44D4987063D111E8A46923403DDBE0C7}"
# Set the value for this scope.
eval "${varname_44D4987063D111E8A46923403DDBE0C7}=\"\${varvalue_44D4987063D111E8A46923403DDBE0C7}\""
unset varname_44D4987063D111E8A46923403DDBE0C7
unset varvalue_44D4987063D111E8A46923403DDBE0C7
unset active_varnames_44D4987063D111E8A46923403DDBE0C7
done
unset varcouple_44D4987063D111E8A46923403DDBE0C7
}
# Simulate unsetting a local variable.
#
# This function is a convenience wrapper over scope_var().
#
# Can be called just like the unset keyword.
# Example usage: _unset foo bar qux
_unset() {
for varname_6E40DA2E63D211E88CE68BFA58FE2BCA in "${#}"; do
scope_var "${varname_6E40DA2E63D211E88CE68BFA58FE2BCA}"
unset "${varname_6E40DA2E63D211E88CE68BFA58FE2BCA}"
done
}
# Simulate exiting out of a nested variable scope
# To be used in conjunction with push_scope(), pop_scope(), and _local()
pop_scope() {
eval "varnames_2581E94263D011E88919B3D175643B87=\"\${SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARNAMES}\""
# Cannot iterate over $varnames by setting $IFS; $IFS does not work
# properly on zsh. Workaround using string manipulation.
while [ -n "${varnames_2581E94263D011E88919B3D175643B87}" ]; do
# Strip enclosing spaces from $varnames.
while true; do
varnames_old_2581E94263D011E88919B3D175643B87="${varnames_2581E94263D011E88919B3D175643B87}"
varnames_2581E94263D011E88919B3D175643B87="${varnames_2581E94263D011E88919B3D175643B87# }"
varnames_2581E94263D011E88919B3D175643B87="${varnames_2581E94263D011E88919B3D175643B87% }"
if [ "${varnames_2581E94263D011E88919B3D175643B87}" = "${varnames_2581E94263D011E88919B3D175643B87}" ]; then
break
fi
done
# Extract the variable name for the current iteration and delete it from the queue.
varname_2581E94263D011E88919B3D175643B87="${varnames_2581E94263D011E88919B3D175643B87%% *}"
varnames_2581E94263D011E88919B3D175643B87="${varnames_2581E94263D011E88919B3D175643B87#${varname_2581E94263D011E88919B3D175643B87}}"
# echo "pop_scope() iteration on \$SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARVALUE_${varname_2581E94263D011E88919B3D175643B87}"
# echo "varname: ${varname_2581E94263D011E88919B3D175643B87}"
if eval "[ -n \""\${SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARVALUE_${varname_2581E94263D011E88919B3D175643B87}+x}"\" ]"; then
# echo "Value found. Restoring value from previous scope."
# echo eval "${varname_2581E94263D011E88919B3D175643B87}=\"\${SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARVALUE_${varname_2581E94263D011E88919B3D175643B87}}\""
eval "${varname_2581E94263D011E88919B3D175643B87}=\"\${SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARVALUE_${varname_2581E94263D011E88919B3D175643B87}}\""
unset "SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARVALUE_${varname_2581E94263D011E88919B3D175643B87}"
else
# echo "Unsetting \$${varname_2581E94263D011E88919B3D175643B87}"
unset "${varname_2581E94263D011E88919B3D175643B87}"
fi
# Variable cleanup.
unset varnames_old_2581E94263D011E88919B3D175643B87
done
unset SCOPE${SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D}_VARNAMES
unset varname_2581E94263D011E88919B3D175643B87
unset varnames_2581E94263D011E88919B3D175643B87
SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D=$(( $SCOPENUM_CEDD88E463CF11E8A72A3F9E5F08767D - 1 ))
}
It's possible to simulate local variables in a Posix Shell using a small set of general functions.
The sample code below demonstrates two functions, called Local and EndLocal, which do the trick.
Use Local once to declare all local variables at the beginning of a routine.
Local creates a new scope, and saves the previous definition of each local variable into a new global variable.
Use EndLocal before returning from the routine.
EndLocal restores all previous definitions saved at the current scope, and deletes the last scope.
Also note that EndLocal preserves the previous exit code.
All functions are short, and use descriptive names, so they should be relatively easy to understand.
They support variables with tricky characters like spaces, single and double quotes.
Caution: They use global variables beginning with LOCAL_, so there's a small risk of collision with existing homonym variables.
The Test routine recursively calls itself 3 times, and modifies a few local and global variables.
The output shows that the A and B local variables are preserved, contrary to the global N variable.
Code:
#!/bin/sh
#-----------------------------------------------------------------------------#
# Manage pseudo-local variables in a Posix Shell
# Check if a variable exists.
VarExists() { # $1=Variable name
eval "test \"\${${1}+true}\" = \"true\""
}
# Get the value of a variable.
VarValue() { # $1=Variable name
eval "echo \"\${$1}\""
}
# Escape a string within single quotes, for reparsing by eval
SingleQuote() { # $1=Value
echo "$1" | sed -e "s/'/'\"'\"'/g" -e "s/.*/'&'/"
}
# Set the value of a variable.
SetVar() { # $1=Variable name; $2=New value
eval "$1=$(SingleQuote "$2")"
}
# Emulate local variables
LOCAL_SCOPE=0
Local() { # $*=Local variables names
LOCAL_SCOPE=$(expr $LOCAL_SCOPE + 1)
SetVar "LOCAL_${LOCAL_SCOPE}_VARS" "$*"
for LOCAL_VAR in $* ; do
if VarExists $LOCAL_VAR ; then
SetVar "LOCAL_${LOCAL_SCOPE}_RESTORE_$LOCAL_VAR" "SetVar $LOCAL_VAR $(SingleQuote "$(VarValue $LOCAL_VAR)")"
else
SetVar "LOCAL_${LOCAL_SCOPE}_RESTORE_$LOCAL_VAR" "unset $LOCAL_VAR"
fi
done
}
# Restore the initial variables
EndLocal() {
LOCAL_RETCODE=$?
for LOCAL_VAR in $(VarValue "LOCAL_${LOCAL_SCOPE}_VARS") ; do
eval $(VarValue "LOCAL_${LOCAL_SCOPE}_RESTORE_$LOCAL_VAR")
unset "LOCAL_${LOCAL_SCOPE}_RESTORE_$LOCAL_VAR"
done
unset "LOCAL_${LOCAL_SCOPE}_VARS"
LOCAL_SCOPE=$(expr $LOCAL_SCOPE - 1)
return $LOCAL_RETCODE
}
#-----------------------------------------------------------------------------#
# Test routine
N=3
Test() {
Local A B
A=Before
B=$N
echo "#1 N=$N A='$A' B=$B"
if [ $N -gt 0 ] ; then
N=$(expr $N - 1)
Test
fi
echo "#2 N=$N A='$A' B=$B"
A="After "
echo "#3 N=$N A='$A' B=$B"
EndLocal
}
A="Initial value"
Test
echo "#0 N=$N A='$A' B=$B"
Output:
larvoire#JFLZB:/tmp$ ./LocalVars.sh
#1 N=3 A='Before' B=3
#1 N=2 A='Before' B=2
#1 N=1 A='Before' B=1
#1 N=0 A='Before' B=0
#2 N=0 A='Before' B=0
#3 N=0 A='After ' B=0
#2 N=0 A='Before' B=1
#3 N=0 A='After ' B=1
#2 N=0 A='Before' B=2
#3 N=0 A='After ' B=2
#2 N=0 A='Before' B=3
#3 N=0 A='After ' B=3
#0 N=0 A='Initial value' B=
larvoire#JFLZB:/tmp$
Using the same technique, I think it should be possible to dynamically detect if the local keyword is supported, and if it's not, define a new function called local that emulates it.
This way, the performance would be much better in the normal case of a modern shell having built-in locals.
And things would still work on an old Posix shell without it.
Actually we'd need three dynamically generated functions:
BeginLocal, creating an empty pseudo-local scope, as is done in the beginning of my Local above, or doing nothing if the shell has built-in locals.
local, similar to my Local, defined only for shells not having built-in locals.
EndLocal, identical to mine, or just preserving the last exit code for shells having built-in locals.
Define the functions using the function myfunc { syntax, and use typeset myvar to define your variables. If you define functions that way rather than using the myfunc(){ syntax, all of the common Bourne shells (bash, zsh, ksh '88 and '93) will localize variables defined with typeset (and aliases to typeset like integer).
Or reinvent the wheel. Whichever floats your boat. ;)
EDIT: while the question asks for POSIX, and this is not a POSIX-compliant function definition syntax, the person who asked indicates in a later comment that he's using bash. The use of "typset" in combination with the alternative function definition syntax is the best solution there, as the true POSIX mechanism requires the additional overhead of forking a new subshell.
I have a web service that requires a list of arguments in a JSON format. For instance, the JSON it could handle is like this:
{
"args": ["basharg1", "bash arg 2 with spaces", "lastargs"]
}
How can I easily transform the arguments to a bash script to build a string that holds this kind of JSON? That string would then be used by curl to pas it to the web service.
On a Mac, a shorter version that uses glenn_jackman's approach is:
ARGS=""; for arg; do ARGS=$(printf '%s"%s"' "$ARGS", "$arg"); done;
ARGS=${ARGS#,}
JSON="{ \"args\": [$ARGS] }"
echo "$JSON"
Here's how I ended up doing it. Feel free to suggest improvements.
#!/bin/bash
ARGS="\"$1\""
shift
while (( "$#" )); do
ARGS="$ARGS, \"$1\""
shift
done
ARGUMENTS_JSON="{ \"args\": [$ARGS] }"
echo "$ARGUMENTS_JSON"