In windows, in the command line, I am not able to send packages with Aspera Fastex Command Line Interface, here are two examples of the commands I tried:
aspera faspex send -t "Send-test" -r "*Attachment" --source 1 -f y:\folder-test --metadata= { "Ticket ID":"test name" }
aspera faspex send -t "Send-test" -r "*Attachment" --source 1 -f y:\folder-test --metadata= {"metadata_fields" : {"Ticket ID": "test name"}}
The output is always ‘JSON Exception’, How should be parsed the JSON argument?
JSON Exception
Try quoting the parameter and doubling the QUOTATION MARK characters inside the parameter.
"--metadata={""metadata_fields"" : {""Ticket ID"": ""test name""}}"
Or possibly:
--metadata= "{""metadata_fields"" : {""Ticket ID"": ""test name""}}"
--metadata "{""metadata_fields"" : {""Ticket ID"": ""test name""}}"
The actual parsing of the command line depends on the operating system and shell used...
the command line tool ("aspera") will use an array of string as input, this array of string is either parsed by the invoking shell, or system call, or language runtime (C/C++)
In you case, I guess you are starting on windows, so the double quote thing solves it.
Refer to:
https://daviddeley.com/autohotkey/parameters/parameters.htm
http://www.windowsinspired.com/understanding-the-command-line-string-and-arguments-received-by-a-windows-program/
concerning the aspera command line itself, you may consider:
https://github.com/IBM/aspera-cli
Related
(Premise:)
Hi, i'm trying to read a JSON file whose content has to be copied inside a variable in a file that i'm going to need later, according to google the only way to do this is by using source. I need to avoid command execution during the sourcing tho, and, being a JSON file, all those quotes are causing me a bit of a headache.
Because of this I need to be sure that each and every one of them is escaped to be treated as plain text. I tried using sed command like this:
sed -e "s/'/\\\'/g" -e 's/"/\\"/g'
Checking again in the file i can see that every single quote has been escaped apart from the outer ones
ex: {{"foo":"bar"},{"bar":"foo}} -> VAR='{{\"foo\":\"bar\"}{\"bar\":\"foo\"}}'
Somehow when i execute the sourcing i get a lot of errors about commands and directories not existing.
(Question:)
Do you have any idea of what's going on? Is this even a viable solution to get to my goal? Is there any better way? If what i'm asking is not possible, is there any other way to use a file as a string variable store?
(Goal generic desired behaviour:)
read json
create conf file
attach json content to variable declaration string ("VAR=$(readthejson)")
attach variable declaration string to conf file
source conf file
use var as a string
(Trial specific desired behaviour:)
a=$( sed -e "s/'/\\\'/g" -e 's/"/\\"/g' myjson.json )
echo "LOCK='$a'" >> file
The lines above successfully fill my file with json content and escapes all quotes
(example from a package-lock.json file:)
LOCK='{
\"name\": \"Undecided\",
\"version\": \"0.0.1\",
\"lockfileVersion\": 2,
\"requires\": true,
\"packages\": {
\"\": {
\"version\": \"0.0.1\",
\"dependencies\": {
\"#capacitor/android\": \"3.1.2\",
\"#capacitor/app\": \"1.0.2\",
\"#capacitor/core\": \"3.1.2\",
\"#capacitor/haptics\": \"1.0.2\",
...
At this point i would expect that sourcing file would result in my strings being loaded in my script and being usable like so:
source file
echo "$LOCK"
output:
{
"name": "Undecided",
"version": "0.0.1",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"version": "0.0.1",
"dependencies": {
"#capacitor/android": "3.1.2",
"#capacitor/app": "1.0.2",
"#capacitor/core": "3.1.2",
"#capacitor/haptics": "1.0.2",
(Actual behaviour:)
The script escapes everything as needed. Though when i source it it outputs this:
usage: install [-bCcpSsv] [-B suffix] [-f flags] [-g group] [-m mode]
[-o owner] file1 file2
install [-bCcpSsv] [-B suffix] [-f flags] [-g group] [-m mode]
[-o owner] file1 ... fileN directory
install -d [-v] [-g group] [-m mode] [-o owner] directory ...
file: line 2128: },: command not found
file: line 2129: "node_modules/#hapi/bourne":: No such file or directory
file: line 2130: "version":: command not found
file: line 2131: "resolved":: command not found
file: line 2132: "integrity":: command not found
file: line 2133: "deprecated":: command not found
file: line 2134: },: command not found
file: line 2135: "node_modules/#hapi/hoek":: No such file or directory
file: line 2136: "version":: command not found
file: line 2137: "resolved":: command not found
file: line 2138: "integrity":: command not found
file: line 2139: "deprecated":: command not found
file: line 2140: },: command not found
file: line 2141: "node_modules/#hapi/joi":: No such file or directory
file: line 2142: "version":: command not found
file: line 2143: "resolved":: command not found
file: line 2144: "integrity":: command not found
file: line 2145: "deprecated":: command not found
file: line 2146: "dependencies":: command not found
file: line 2147: "#hapi/address":: No such file or directory
file: line 2148: "#hapi/bourne":: No such file or directory
file: line 2149: "#hapi/hoek":: No such file or directory
file: line 2150: "#hapi/topo":: No such file or directory
file: line 2151: syntax error near unexpected token `}'
file: line 2151: ` }'
It looks like it's ignoring backslashes, or that they get swapped with non escaped quotes and then sourced.
I mean, i would expect that echo "\"lorem\"\"ipsum\"" result in "lorem""ipsum", not in a
lorem command not found
ipsum command not found
Disclaimer: i'm not asking to code for me or debug my code (it's sad i really have to specify this)
Is there any better way?
If you want to output the value to the file to be sourced later, whatever the variable value, I like to use declare -p.
lock=$(cat myjson.json)
declare -p lock >> file_to_be_sourced.sh
declare is specific to Bash, and it will always output a properly quoted string that can be sourced later. Another way is to use printf "%q".
Do you have any idea of what's going on?
"s/'/\\\'/g" part is wrong. If you start with ' quotes, you have replace ' with '\''. \ is literal when inside ' quotes. And remove the s/"/\\"/g part - just keep inside ' quotes. The difference is cool to observe when using /bin/printf which is from coreutils vs printf as bash builtin - they use different quoting "style":
$ var="a\"b'c"
$ echo "$var"
a"b'c
$ printf "%q\n" "$var"
a\"b\'c
$ /bin/printf "%q\n" "$var"
'a"b'\''c'
$ declare -p var
declare -- var="a\"b'c"
If what i'm asking is not possible, is there any other way to use a file as a string variable store?
If it's a "store" with many strings, could be you could use an associative array.
declare -A mapvar
mapvar["abc"]=$(< file1.json)
mapvar["def"]=$(< file2.json)
declare -p mapvar > file_to_be_sourced.sh
I finally got to a solution.
Apparently if you echo "Something='$a'" it does't matter if you have single, double, triple, escaped or whatever quotes inside that variable: he just doesn't care and use the double ones.
For example in my case he just ignored the outermost ' and considered everything else as a command
My solution was very easily to:
echo "Something:\"$a\""
And now it's treated as an actual string.
I'm pretty confused about why, but that's probably caused by how bash works with quotes
I am writing a JSON template for AWS SSM document.
Once of the commands I am trying to run looks like this:
"ssh-keygen -q -t rsa -N '' <<< ""$'\n'"y" 2>&1 >/dev/null",
However, due to the "" after the <<< JSON thinks that this is a new line and is expecting a comma.
Is there a way I can tell JSON that that is a single command and need to be treated as a single line?
You need to escape the double quotes. So " inside your json string becomes \".
I'm trying to take the contents of a config file (JSON format), strip out extraneous new lines and spaces to be concise and then assign it to an environment variable before starting my application.
This is where I've got so far:
pwr_config=`echo "console.log(JSON.stringify(JSON.parse(require('fs').readFileSync(process.argv[2], 'utf-8'))));" | node - config.json | xargs -0 printf '%q\n'` npm run start
This pipes a short node.js app into the node runtime taking an argument of the file name and it parses and stringifies the JSON file to validate it and remove any unnecessary whitespace. So far so good.
The result of this is then piped to printf, or at least it would be but printf doesn't support input in this way, apparently, so I'm using xargs to pass it in in a way it supports.
I'm using the %q formatter to format the string escaping any characters that would be a problem as part of a command, but when calling printf through xargs, printf claims it doesn't support %q. I think this is perhaps because there is more than one version of printf but I'm not exactly sure how to resolve that.
Any help would be appreciated, even if the solution is completely different from what I've started :) Thanks!
Update
Here's the output I get on MacOS:
$ cat config.json | xargs -0 printf %q
printf: illegal format character q
My JSON file looks like this:
{
"hue_host": "192.168.1.2",
"hue_username": "myUsername",
"port": 12000,
"player_group_config": [
{
"name": "Family Room",
"player_uuid": "ATVUID",
"hue_group": "3",
"on_events": ["media.play", "media.resume"],
"off_events": ["media.stop", "media.pause"]
},
{
"name": "Lounge",
"player_uuid": "STVUID",
"hue_group": "1",
"on_events": ["media.play", "media.resume"],
"off_events": ["media.stop", "media.pause"]
}
]
}
Two ways:
Use xargs to pick up bash's printf builtin instead of the printf(1) executable, probably in /usr/bin/printf(thanks to #GordonDavisson):
pwr_config=`echo "console.log(JSON.stringify(JSON.parse(require('fs').readFileSync(process.argv[2], 'utf-8'))));" | node - config.json | xargs -0 bash -c 'printf "%q\n"'` npm run start
Simpler: you don't have to escape the output of a command if you quote it. In the same way that echo "<|>" is OK in bash, this should also work:
pwr_config="$(echo "console.log(JSON.stringify(JSON.parse(require('fs').readFileSync(process.argv[2], 'utf-8'))));" | node - config.json )" npm run start
This uses the newer $(...) form instead of `...`, and so the result of the command is a single word stored as-is into the pwr_config variable.*
Even simpler: if your npm run start script cares about the whitespace in your JSON, it's fundamentally broken :) . Just do:
pwr_config="$(< config.json)" npm run start
The $(<...) returns the contents of config.json. They are all stored as a single word ("") into pwr_config, newlines and all.* If something breaks, either config.json has an error and should be fixed, or the code you're running has an error and needs to be fixed.
* You actually don't need the "" around $(). E.g., foo=$(echo a b c) and foo="$(echo a b c)" have the same effect. However, I like to include the "" to remind myself that I am specifically asking for all the text to be kept together.
I have Snakefile as following:
SAMPLES, = glob_wildcards("data/{sample}_R1.fq.gz")
rule all:
input:
expand("samtools_sorted_out/{sample}.raw.snps.indels.g.vcf", sample=SAMPLES),
expand("samtools_sorted_out/combined_gvcf")
rule combine_gvcf:
input: "samtools_sorted_out/{sample}.raw.snps.indels.g.vcf"
output:directory("samtools_sorted_out/combined_gvcf")
params: gvcf_file_list="gvcf_files.list",
gatk4="/storage/anaconda3/envs/exome/share/gatk4-4.1.0.0-0/gatk-package-4.1.0.0-local.jar"
shell:"""
java -DGATK_STACKTRACE_ON_USER_EXCEPTION=true \
-jar {params.gatk4} GenomicsDBImport \
-V {params.gvcf_file_list} \
--genomicsdb-workspace-path {output}
"""
When I test it with dry run, I got error:
RuleException in line 335 of /data/yifangt/exomecapture/Snakefile:
Wildcards in input, params, log or benchmark file of rule combine_gvcf cannot be determined from output files:
'sample'
There are two places that I need some help:
The {output} is a folder that will be created by the shell part;
The {output} folder was hard-coded manually required by the command line (and the contents are unknown ahead of time).
The problem seems to be with the {output} without expansion as compared with the {input} which does.
How should I handle with this situation? Thanks a lot!
I have JSON files on my server that needs to be passed into several different Raspberry Pis running Debian. Each of the Pis has their own JSON feed that they will pull from, but essentially, I need to automatically take the value of one key-value pair and use it as an argument for a command that is run in the terminal.
For instance: Fetching https://www.example.com/api/THDCRUG2899CGF8&/manifest.json
{
"version": "1.5.6",
"update_at": "201609010000",
"body": "172.16.1.1"
}
Then that value would be dynamically output into a command that uses the body's value as an argument. EG: ping [body value]
Edit:
The point of this is to have a task that executes every minute to receive weather updates to it.
You are looking for command substition, specifically wrapped around a command that can extract values from a JSON value. First, you can use jq as the JSON-processing command.
$ jq -r '.body' tmp.json
172.16.1.1
Command substitution allows you to capture the output of jq to use as an argument:
$ ping "$(jq -r '.body' tmp.json)"
PING 172.16.1.1 (172.16.1.1): 56 data bytes
...