Hey guys I have a json file connections.json with the following airflow connections
{
"google_cloud_default": {
"conn_type": "google_cloud_platform",
"host": null,
"login": null,
"password": null,
"schema": null,
"port": null,
"extra": "{\"extra__google_cloud_platform__key_path\": \"/usr/local/sample_path/credentials/sample.json\", \"extra__google_cloud_platform__num_retries\": 5, \"extra__google_cloud_platform__project\": \"sample-project\", \"extra__google_cloud_platform__scope\": \"https://www.googleapis.com/auth/spreadsheets\"}"
}
}
Is it possible to extend the "extra" parameter so it spans multiple lines for readability?
Like so:
{
"google_cloud_default": {
"conn_type": "google_cloud_platform",
"host": null,
"login": null,
"password": null,
"schema": null,
"port": null,
"extra": "{\"extra__google_cloud_platform__key_path\": \"/usr/local/sample_path/credentials/sample.json\",
\"extra__google_cloud_platform__num_retries\": 5,
\"extra__google_cloud_platform__project\": \"sample-project\",
\"extra__google_cloud_platform__scope\": \"https://www.googleapis.com/auth/spreadsheets\"}"
}
}
I tried wrapping the entire extra value in three double quotes i.e. """ but that still seems to throw an error. Thoughts?
Related
I successfully deployed my ARM template via ADO last week and realised that i forgot to include App Service Authentication for my Web App slots.
I messed around with the Microsoft.Web/sites/slots/config and ran into some errors that i was able to overcome with the help of few individuals on Stack.
However, now that the code should work it is failing but i am not getting any error codes on Azure DevOps when releasing.
It just says;
2020-07-02T14:20:19.0820320Z ##[error]At least one resource deployment operation failed. Please
list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
2020-07-02T14:20:19.0832558Z ##[error]Details:
2020-07-02T14:20:19.0834149Z ##[error]BadRequest:
2020-07-02T14:20:19.0835776Z ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
2020-07-02T14:20:19.0837268Z ##[error]Task failed while creating or updating the template deployment.
Here is my code:
{
"type": "Microsoft.Web/sites/slots/config",
"name": "[concat(parameters('webAppName'),'/staging/auth')]",
"apiVersion": "2018-11-01",
"location": "[resourceGroup().location]",
"dependsOn": [
"[parameters('webAppName')]",
"[concat(parameters('sqlDatabase'), 'constr')]"
],
"properties": {
"enabled": true,
"runtimeVersion": "~1",
"unauthenticatedClientAction": "RedirectToLoginPage",
"tokenStoreEnabled": true,
"allowedExternalRedirectUrls": null,
"defaultProvider": "AzureActiveDirectory",
"clientId": null,
"clientSecret": null,
"clientSecretCertificateThumbprint": null,
"issuer": null,
"allowedAudiences": [
"https://webapptest1a-staging.azurewebsites.net"
],
"additionalLoginParams": null,
"isAadAutoProvisioned": false,
"googleClientId": null,
"googleClientSecret": null,
"googleOAuthScopes": null,
"facebookAppId": null,
"facebookAppSecret": null,
"facebookOAuthScopes": [
""
],
"twitterConsumerKey": null,
"twitterConsumerSecret": null,
"microsoftAccountClientId": null,
"microsoftAccountClientSecret": null,
"microsoftAccountOAuthScopes": [
""
]
}
},
Anything sticking out like a sore thumb?
I released the code without this part in it, and it worked fine, so it is most likely this little section here that is causing an issue, but all the information i believe is correct.
According to: https://learn.microsoft.com/en-us/azure/templates/microsoft.web/2019-08-01/sites/config-authsettings
Thank you for your guidance.
So, after a bit i found an answer;
Here is the working code;
What i had previously done was this;
"name": "[concat(parameters('webAppName'), '/staging/auth')]",
What it needs to be is this;
"name": "[concat(parameters('webAppName'), '/staging/authsettings')]",
The "auth" alone will not match anything, and will give back a bad request. Which is right as nothing under "auth" exists meaning it cannot be matched.
So when you put Auth Settings, it actually works as intended as it matches!
Here is what i followed
{
"type": "Microsoft.Web/sites/slots/config",
"name": "[concat(parameters('webAppName'), '/staging/authsettings')]",
"apiVersion": "2018-11-01",
"location": "[resourceGroup().location]",
"dependsOn": [
"[parameters('webAppName')]",
"[concat(parameters('sqlDatabase'), 'constr')]"
],
"properties": {
"enabled": true,
"runtimeVersion": "1.0.0",
"unauthenticatedClientAction": "RedirectToLoginPage",
"tokenStoreEnabled": false,
"allowedExternalRedirectUrls": null,
"defaultProvider":"AzureActiveDirectory",
"clientId": null,
"clientSecret": null,
"clientSecretCertificateThumbprint": null,
"issuer": null,
"allowedAudiences": null,
"additionalLoginParams": null,
"isAadAutoProvisioned": false,
"googleClientId": null,
"googleClientSecret": null,
"googleOAuthScopes": null,
"facebookAppId": null,
"facebookAppSecret": null,
"facebookOAuthScopes": [
""
],
"twitterConsumerKey": null,
"twitterConsumerSecret": null,
"microsoftAccountClientId": null,
"microsoftAccountClientSecret": null,
"microsoftAccountOAuthScopes": [
""
]
}
},
I have a JSON as follows but need a way to remove nulls before putting it into elasticsearch. Looking for a simple jq command to remove nulls that i can incorporate into my bash script unless there's a way to do this in elasticsearch
{
"master_no": {
"master_no": 100000000,
"barcode": "E00000000",
"external_key": null,
"umid": null
},
"cust_id": {
"other_cust_id": null,
"cust_reference": null,
"external_key": null,
"list_id": null,
"cust_id": null
},
"customer_name": null,
"master_desc": "test Custom Patch - test",
"barcode": "E00000000",
"container_master_no": null,
"master_status": "I",
"length": "0:00",
"format_no": {
"format_desc": null,
"external_key": null,
"format_no": null
},
"lib_master_audio": [
{
"master_no": 10000000,
"audio_channel_no": {
"audio_channel_no": 10,
"audio_channel": "1",
"external_key": null
}
},
{
"master_no": 100000000,
"audio_channel_no": {
"audio_channel_no": 10,
"audio_channel": "2",
"external_key": null
}
}
]
}
Thanks
This GitHub issue on remove null key and values from JSON can help you, in short, some handful of command might help you like, mentioned in this link :
del(.[][] | nulls)
Please note there are several methods of doing this, please check which one works for you.
As pointed out in comments by #oguz, Please use https://github.com/stedolan/jq/issues/104#issuecomment-289637207 which work with the latest version.
Say I have two versions of the same JSON data. Each object has a unique ID key/pair. Say in one version a certain key has null as its value while the other version has the appropriate values for that key.
Can we match the objects between the two JSON files based on their unique ID value and copy over a certain key's values? I don't want to mess around with the rest of the fields in the JSON object.
Sample:
version 1:
[
{
"contentId": "ID-02",
"title": "Attendance",
"desp": "Daily Attendance",
"contentType": "service",
"url": "ATTENDANCE",
"contentCategory": "Essentials",
"employeeId": null,
"imageUrl": null,
"publishedCourseFlag": "true"
},
{
"contentId": "ID-04",
"title": "Regularise History",
"desp": "Regularise History",
"contentType": "service",
"url": "REGULARISE_HISTORY",
"contentCategory": "Non-Essentials",
"employeeId": null,
"imageUrl": null,
"publishedCourseFlag": "false"
}
]
version 2:
[
{
"contentId": "ID-02",
"title": "Attendance",
"desp": "Daily Attendance",
"contentType": "service",
"url": "ATTENDANCE",
"contentCategory": null,
"employeeId": null,
"imageUrl": null
},
{
"contentId": "ID-04",
"title": "Regularise History",
"desp": "Regularise History",
"contentType": "service",
"url": "REGULARISE_HISTORY",
"contentCategory": null,
"employeeId": null,
"imageUrl": null,
"publishedCourseFlag": "false"
}
]
Here v1 has the contentCategory set while v2 has null. I want to copy the appropriate values from v1 to v2 based on their contentId values. Is there a simple way to do this other than manually copy/paste? I'd like a scripting solution that I can modify to my needs for situations like this. Or a simple query like solution would be even better.
Additionally is it possible to add a certain missing key/value from version 1 to version 2 ? Like Update if present else Insert ? I am hoping for a lot I guess. Completely new to JSON.
I had an issue today with Filemaker on how to get the first element out of a json result without knowing the key.
Example $json result from an API call
{
"26298070": {
"task_id": "26298070",
"parent_id": "0",
"name": "DEPOT-0045 Research ODBC Model Extraction via Django To cut down on development time from Filemaker to Postgres",
"external_task_id": "32c8fd51-2066-42b9-b88b-8a2275fafc3f",
"external_parent_id": "64e7c829-d88e-48ae-9ba4-bb7a3871a7ce",
"level": "1",
"add_date": "2018-06-04 21:45:16",
"archived": "0",
"color": "#34C644",
"tags": "DEPOT-0045",
"budgeted": "1",
"checked_date": null,
"root_group_id": "91456",
"assigned_to": null,
"assigned_by": null,
"due_date": null,
"note": "",
"context": null,
"folder": null,
"repeat": null,
"billable": "0",
"budget_unit": "hours",
"public_hash": null,
"modify_time": null
}
}
I tried JSONGetElement( $json, "") and got the original json.
I tried JSONGetElement( $json, ".") and got the original json.
I tried JSONGetElement( $json, 1 ) and got nothing.
How do you get the first element out of a JSON String without knowing the name of the element in FileMaker 16 or 17?
Try this for the root element:
JSONListKeys ( $json ; "" )
result: 26298070
Once you get the root, you can get the child keys.
I remembered that FileMaker has a function to extract words from text so I thought I'd see what happened if I extracted the first word as a key.
I tried
JSONGetElement ( $json ; MiddleWords ( $json,1,1 ) )
and got the result I was looking for.
{
"add_date": "2018-06-04 21:45:16",
"archived": "0",
"assigned_by": null,
"assigned_to": null,
"billable": "0",
"budget_unit": "hours",
"budgeted": "1",
"checked_date": null,
"color": "#34C644",
"context": null,
"due_date": null,
"external_parent_id": "64e7c829-d88e-48ae-9ba4-bb7a3871a7ce",
"external_task_id": "32c8fd51-2066-42b9-b88b-8a2275fafc3f",
"folder": null,
"level": "1",
"modify_time": null,
"name": "DEPOT-0045 Research ODBC Model Extraction via Django To cut down on development time from Filemaker to Postgres",
"note": "",
"parent_id": "0",
"public_hash": null,
"repeat": null,
"root_group_id": "91456",
"tags": "DEPOT-0045",
"task_id": "26298070"
}
which makes it easy to parse simple JSON schema's that use attributes for keys.
I am trying to persist a nested object in sessionAttributes. However, it seems Amazon Lex does not recognize if I send a structure like this in response:
{
"sessionAttributes": {
"previousIntent": "TCAIntent",
"prevSlots": {
"test": 1
}
},
"dialogAction": {
"type": "Delegate",
"slots": {
"firstName": null,
"lastName": null,
"rate": null,
"rentMortgage": null,
"downPayment": null,
"purchasePrice": null,
"email": null
}
}
}
When I replace {"test": 1} with a value, it works fine. Is there a way around this or possible mistake?
Error:
I have figured out that Lex does not allow storing json object or array on sessionAttributes. As a work around, I converted the json object to string and convert it back when I need to use it.