Have to write a ruby script which can validate that the key in s3 matches directory name.
I have a json file, path name- Apple/Employee/Background/test.tf
How can I validate that my directory path Apple/Employee/Background/ matches with the "key": "Apple/Employee/Background/" in json file.
I used jq and parsed json file to retrive the value of "key".
cat conf.json | jq '.terraform[] | .backend[] | .s3[] | .key'
"Apple/Employee/Background/terraform.tfstate"
I want to compare only Apple/Employee/Background with directory path and see if they are same or not.
json file looks like:
"terraform": [
{
"backend": [
{
"s3": [
{
"bucket": "terraform-dev",
"dynamodb": "terraform_files",
"encrypt": "true",
"key": "Apple/Employee/Background/terraform.tfstate"
}
]
}
],
"required_version": "~> 0.11.8"
}
]
the directory path looks like- Apple/Employee/Background/conf.tf
You can use File.dirname to get the directory path minus the base entry (in this case, 'terraform.tfstate')
# Assuming `key` contains the value from the JSON file
dir = File.dirname(key) # => "Apple/Employee/Background"
Note that the path is returned without a trailing slash.
Related
Before I do a release of my package and tag it, I would like to update the manifest.json file version Patch value to reflect the new version of the package every time I trigger it.
Json file:
{
"version": "0.0.0",
"timeStamp": "2022-02-14T09:41:34+00:00",
"packages": [
{
"name": "data-service",
"type": "docker-image",
"version": "REL-1.0.5"
},
{
"name": "application-service",
"type": "docker-image",
"version": "REL-1.0.6"
},
]
}
So here I am trying to increment by reading the content of the Json file from Powershell:
version = (Get-Content manifest.json | ConvertFrom-Json).version
Value for the variable "version" in the Json file is 0.0.0
But My question is - how to increase the only patch value? E.g. I need to have 0.0.1 output values after transform automatically in the manifest.json file.
Use the [version] type to parse the existing version, then increment the build number:
$data = Get-Content manifest.json | ConvertFrom-Json
# extract and increment version build number
$existingVersion = $data.version -as [version]
$nextVersion = [version]::new($existingVersion.Major, $existingVersion.Minor, $existingVersion.Build + 1)
# update original data
$data.version = "$nextVersion"
# write back to disk
$data |ConvertTo-Json |Set-Content updated-manifest.json
I would like to search a JSON file for some key or value, and have it print where it was found.
For example, when using jq to print out my Firefox' extensions.json, I get something like this (using "..." here to skip long parts) :
{
"schemaVersion": 31,
"addons": [
{
"id": "wetransfer#extensions.thunderbird.net",
"syncGUID": "{e6369308-1efc-40fd-aa5f-38da7b20df9b}",
"version": "2.0.0",
...
},
{
...
}
]
}
Say I would like to search for "wetransfer#extensions.thunderbird.net", and would like an output which shows me where it was found with something like this:
{ "addons": [ {"id": "wetransfer#extensions.thunderbird.net"} ] }
Is there a way to get that with jq or with some other json tool?
I also tried to simply list the various ids in that file, and hoped that I would get it with jq '.id', but that just returned null, because it apparently needs the full path.
In other words, I'm looking for a command-line json parser which I could use in a way similar to Xpath tools
The path() function comes in handy:
$ jq -c 'path(.. | select(. == "wetransfer#extensions.thunderbird.net"))' input.json
["addons",0,"id"]
The resulting path is interpreted as "In the addons field of the initial object, the first array element's id field matches". You can use it with getpath(), setpath(), delpaths(), etc. to get or manipulate the value it describes.
Using your example with modifications to make it valid JSON:
< input.json jq -c --arg s wetransfer#extensions.thunderbird.net '
paths as $p | select(getpath($p) == $s) | null | setpath($p;$s)'
produces:
{"addons":[{"id":"wetransfer#extensions.thunderbird.net"}]}
Note
If there are N paths to the given value, the above will produce N lines. If you want only the first, you could wrap everything in first(...).
Listing all the "id" values
I also tried to simply list the various ids in that file
Assuming that "id" values of false and null are of no interest, you can print all the "id" values of interest using the jq filter:
.. | .id? // empty
I have a config file that is like
[
"ECSClusterName=cluster",
"VPCID=vpc-xxxx",
"ALBName=ALB"
]
And with jq (or something else bash-native), I'd like to add 2 values - EnvType and KMSID - (doesn't matter where in the config file) so that the end result would look like
[
"EnvType=dev",
"KMSID=xxxxx-yyyyyy-ffffff",
"ECSClusterName=cluster",
"VPCID=vpc-xxxx",
"ALBName=ALB"
]
The closest I have been for one value is
cat config.json | jq '.[-1] += ", test=test"'
But that outputs
[
"ECSClusterName=cluster",
"VPCID=vpc-xxxx",
"ALBName=ALB, test=test"
]
Any help greatly appreciated!
Put new key=value pairs into an array, and add that array to the original one.
$ jq '. + ["EnvType=dev", "KMSID=xxxxx-yyyyyy-ffffff"]' config.json
[
"ECSClusterName=cluster",
"VPCID=vpc-xxxx",
"ALBName=ALB",
"EnvType=dev",
"KMSID=xxxxx-yyyyyy-ffffff"
]
I would like to append an element to an existing JSON file where I have the working directory as the key and the working directory + the contents as the value in a string array format.
Lets say I have the following structure:
Docs (Directory)
|
+-- RandomFile.json
|
+-- Readme (Working Directory)
| |
| +-- Readme.md
| +-- Readyou.md
What I would like to achieve is the structure below with the working directory as the prefix for every element in the array.
"Readme": ["Readme/Readme.md", "Readme/Readyou.md"]
From the output above, I would like to append that to the contents of the RandomFile.json which currently looks like this:
{
"docs": {
"Doc": ["doc1"]
}
}
to this:
{
"docs": {
"Doc": ["doc1"],
"Readme": ["Readme/Readme.md", "Readme/Readyou.md"]
}
}
Is it something that can be managed straightforward using bash and jq?
This requires jq 1.6 in order to use the --args option.
$ jq --arg wd "$(basename "$PWD")" '.docs+={($wd): $ARGS.positional | map("\($wd)/\(.)")}' ../RandomFile.json --args *
{
"docs": {
"Doc": [
"doc1"
],
"Readme": [
"Readme/Readme.md",
"Readme/Readyou.md"
]
}
}
The shell is used to pass the base name of the current working directory as the variable $wd.
The shell is also used to pass the names of all the files in the current working directory as separate arguments.
The file to edit is assumed to be ../RandomFile.json; if you only know that there is a JSON file in the parent, you can use ../*.json instead.
Use += to update the .docs object of the original with a new key (the working directory) and list of file names. map prefixes each element of $ARGS.positional with $wd.
Need your expertise here!
I am trying to load a JSON file (generated by JSON dumps) into redshift using copy command which is in the following format,
[
{
"cookieId": "cb2278",
"environment": "STAGE",
"errorMessages": [
"70460"
]
}
,
{
"cookieId": "cb2271",
"environment": "STG",
"errorMessages": [
"70460"
]
}
]
We ran into the error - "Invalid JSONPath format: Member is not an object."
when I tried to get rid of square braces - [] and remove the "," comma separator between JSON dicts then it loads perfectly fine.
{
"cookieId": "cb2278",
"environment": "STAGE",
"errorMessages": [
"70460"
]
}
{
"cookieId": "cb2271",
"environment": "STG",
"errorMessages": [
"70460"
]
}
But in reality most JSON files from API s have this formatting.
I could do string replace or reg ex to get rid of , and [] but I am wondering if there is a better way to load into redshift seamlessly with out modifying the file.
One way to convert a JSON array into a stream of the array's elements is to pipe the former into jq '.[]'. The output is sent to stdout.
If the JSON array is in a file named input.json, then the following command will produce a stream of the array's elements on stdout:
$ jq ".[]" input.json
If you want the output in jsonlines format, then use the -c switch (i.e. jq -c ......).
For more on jq, see https://stedolan.github.io/jq