How to delete a Json object in BASH - json

I have a couple of Json objects and I need to delete one of them if this Json contains specific information. For an example I need to delete if state of the Json object is RUNNING.
INPUT
projects {
key: "ads_evenflow.opt"
value {
name: "ads_evenflow.opt"
state: COMPLETE
result: PASSED
}
}
projects {
key: "alexandria.opt"
value {
name: "alexandria.opt"
state: RUNNING
result: PASSED
}
}
projects {
key: "android.opt"
value {
name: "android.opt"
state: COMPLETE
result: PASSED
}
}
OUTPUT
projects {
key: "ads_evenflow.opt"
value {
name: "ads_evenflow.opt"
state: COMPLETE
result: PASSED
}
}
projects {
key: "android.opt"
value {
name: "androids.opt"
state: COMPLETE
result: PASSED
}
}

Your structure isn't an valid JSON. For such structures you need some more relaxed parser. Fortunately, the JSONY perl module could parse it. From the doc:
JSONY is a data language that is simlar to JSON, just more chill. All
valid JSON is also valid JSONY (and represents the same thing when
loaded), but JSONY lets you omit a lot of the syntax that makes JSON a
pain to write.
The following perl code does what you want.
#!/usr/bin/env perl
use 5.014;
use warnings;
use JSONY;
my $string = slurp_file();
my $data = JSONY->new->load( $string );
for my $proj (#{$data}) {
next unless ref($proj);
next if $proj->{value}->{state} eq 'RUNNING';
pretty_print_proj($proj);
}
sub pretty_print_proj {
my $p = shift;
say "project {";
say qq{\tkey: "$p->{key}"};
say "\tvalue {";
say "\t\t$_: ", $p->{value}->{$_} for (qw(name state result));
say "\t}";
say "}";
}
sub slurp_file {
#change this for your real case...
return do { local $/; <DATA>};
}
__DATA__
projects {
key: "ads_evenflow.opt"
value {
name: "ads_evenflow.opt"
state: COMPLETE
result: PASSED
}
}
projects {
key: "alexandria.opt"
value {
name: "alexandria.opt"
state: RUNNING
result: PASSED
}
}
projects {
key: "android.opt"
value {
name: "android.opt"
state: COMPLETE
result: PASSED
}
}
prints:
project {
key: "ads_evenflow.opt"
value {
name: ads_evenflow.opt
state: COMPLETE
result: PASSED
}
}
project {
key: "android.opt"
value {
name: android.opt
state: COMPLETE
result: PASSED
}
}

Related

BadRequest by InvalidTemplate when deploying a bicep file containing an eventgridSubscription with a servicebus destination endpoint

What I would like to achieve is to be able to generate eventgridsubscriptions very easy using bicep. Because manually it costs a lot of time. I have to create like a over a dozen each day.
I have the following bicep file called main.bicep
param eventSubscriptionName string = 'eventSubName'
param storageAccountName string ='storeAccountName'
param deadLetterAccountName string = 'deadlttrstore'
param serviceBusQueueName string = 'queue.name.enter'
param onrampName string = 'storagecontainername'
resource storageAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: storageAccountName
}
resource deadLetterAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: deadLetterAccountName
}
resource serviceBusQueue 'Microsoft.ServiceBus/namespaces/queues#2021-11-01' existing = {
name: serviceBusQueueName
}
resource eventgridsubscription 'Microsoft.EventGrid/eventSubscriptions#2021-12-01' = {
name: eventSubscriptionName
scope: storageAccount
properties: {
deadLetterDestination: {
endpointType: 'StorageBlob'
properties: {
blobContainerName: 'storage-deadletters'
resourceId: deadLetterAccount.id
}
}
destination: {
endpointType: 'ServiceBusQueue'
properties: {
deliveryAttributeMappings: [
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: ''
}
}
]
resourceId: serviceBusQueue.id
}
}
eventDeliverySchema: 'EventGridSchema'
filter: {
enableAdvancedFilteringOnArrays: false
includedEventTypes: [
'Microsoft.Storage.BlobCreated'
]
isSubjectCaseSensitive: false
subjectBeginsWith: '/blobServices/default/containers/${onrampName}'
subjectEndsWith: '.json'
}
retryPolicy: {
eventTimeToLiveInMinutes: 1440
maxDeliveryAttempts: 5
}
}
}
When I want create the event subscription using az cli with:
az deployment group create -f main.bicep -g <resource-group>
I get the following error:
{
"status": "Failed",
"error":
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details":
[
{
"code": "BadRequest",
"message": "{\r\n \"error\":
{\r\n \"code\": \"InvalidTemplate\",\r\n \"message\": \"Unable to process template language expressions for resource '/subscriptions/x1234456-f9cc-44e5-bc40-5f02d962f2d7/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/providers/Microsoft.EventGrid/eventSubscriptions/eventSubName' at line '34' and column '5'. 'The language expression property array index '1' is out of bounds.'\",\r\n \"additionalInfo\":
[\r\n {\r\n \"type\": \"TemplateViolation\",\r\n \"info\": {\r\n \"lineNumber\": 34,\r\n \"linePosition\": 5,\r\n \"path\": \"\"\r\n }\r\n }\r\n ]\r\n }\r\n}"
}
]
}
}
I am working according to the template documented at MS here:
https://learn.microsoft.com/en-us/azure/templates/microsoft.eventgrid/eventsubscriptions?tabs=bicep
Eventually the solution was quite simple the servicebus resource was missing its parent resource namely the servicebus namespace. Once that was added it worked.
resource serviceBus 'Microsoft.ServiceBus/namespaces#2021-11-01' existing = {
name: serviceBusName
}
and
resource serviceBusQueue 'Microsoft.ServiceBus/namespaces/queues#2021-11-01' existing = {
parent: serviceBus
name: serviceBusQueueName
}
to
param eventSubscriptionName string = 'eventSubName'
param storageAccountName string ='storeAccountName'
param deadLetterAccountName string = 'deadlttrstore'
param serviceBusQueueName string = 'queue.name.enter'
param onrampName string = 'storagecontainername'
resource storageAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: storageAccountName
}
resource deadLetterAccount 'Microsoft.Storage/storageAccounts#2021-09-01' existing = {
name: deadLetterAccountName
}
resource serviceBus 'Microsoft.ServiceBus/namespaces#2021-11-01' existing = {
name: serviceBusName
}
resource serviceBusQueue 'Microsoft.ServiceBus/namespaces/queues#2021-11-01' existing = {
parent: serviceBus
name: serviceBusQueueName
}
resource eventgridsubscription 'Microsoft.EventGrid/eventSubscriptions#2021-12-01' = {
name: eventSubscriptionName
scope: storageAccount
properties: {
deadLetterDestination: {
endpointType: 'StorageBlob'
properties: {
blobContainerName: 'storage-deadletters'
resourceId: deadLetterAccount.id
}
}
destination: {
endpointType: 'ServiceBusQueue'
properties: {
deliveryAttributeMappings: [
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: 'some-value'
}
}
]
resourceId: serviceBusQueue.id
}
}
eventDeliverySchema: 'EventGridSchema'
filter: {
enableAdvancedFilteringOnArrays: false
includedEventTypes: [
'Microsoft.Storage.BlobCreated'
]
isSubjectCaseSensitive: false
subjectBeginsWith: '/blobServices/default/containers/${onrampName}'
subjectEndsWith: '.json'
}
retryPolicy: {
eventTimeToLiveInMinutes: 1440
maxDeliveryAttempts: 5
}
}
}
I tried to reproduce the error message you posted but could get the same result. I did get an error message because value: '':
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: ''
}
}
When I updated to the following, it worked:
{
name: serviceBusQueueName
type: 'Static'
properties: {
isSecret: false
value: 'some-value'
}
}
The error message I saw with the empty string was:
Null or empty value for static delivery attribute queue-name-enter. Static delivery attribute value must be a non-empty string.
After adding some random text, the deployment completed successfully.

get difference between two dates in $project

Trying to project the date difference between two dates, but I am getting error -
Invalid $project :: caused by :: Unknown expression $dateDiff
db.books.aggregate([{
$project:{
Date_diff:
{$dateDiff:{
start_dt:'$borrowers_list.borrowed_dt',
endDate:'$borrowers_list.return_dt',
unit: "day"
}
}
}
}])
The json document structure is like this -
_id:6188a5283543f7cc2f77c73f
branch_id:1
borrowers_list:Object
0:Object
borrowed_dt:2021-08-15T06:00:00.000+00:00
card_no:"ID000067"
return_dt:2021-08-25T06:00:00.000+00:00
I have no idea why the error is unknown expression $dateDiff, as my syntax is correct. Does anyone have any suggestions?
Based on your provided JSON document, the document should be as below (correct me if it is incorrect):
{
_id: ObjectId("6188a5283543f7cc2f77c73f"),
branch_id: 1,
borrowers_list: {
0: {
borrowed_dt: ISODate("2021-08-15T06:00:00.000+00:00"),
card_no: "ID000067",
return_dt: ISODate("2021-08-25T06:00:00.000+00:00")
}
}
}
]
There is no start_dt in $dateFiff field, it is startDate.
Query
db.collection.aggregate([
{
$project: {
Date_diff: {
$dateDiff: {
startDate: "$borrowers_list.0.borrowed_dt",
endDate: "$borrowers_list.0.return_dt",
unit: "day"
}
}
}
}
])
Note: Above query will perform the $dateDiff for the first document in borrowers_list.
Sample Mongo Playground
In case you need to iterate each document (with key-value pair) in borrowers_list to perform $dateDiff.
$set - Convert from object to array (via $objectToArray) for borrowers_list to new field borrowers.
$set - Iterate each document in borrowers array (1) and perform $dateDiff.
$project - Decorate the output document, convert Date_diff from array to object (via $objectToArray).
Query
db.collection.aggregate([
{
$set: {
borrowers: {
"$objectToArray": "$borrowers_list"
}
}
},
{
$set: {
Date_diff: {
$map: {
input: "$borrowers",
as: "b",
in: {
k: "$$b.k",
v: {
$dateDiff: {
startDate: "$$b.v.borrowed_dt",
endDate: "$$b.v.return_dt",
unit: "day"
}
}
}
}
}
}
},
{
$project: {
Date_diff: {
"$arrayToObject": "$Date_diff"
}
}
}
])
Sample Mongo Playground (Iterate document with key-value pair)

Return null on update mutation GraphQL and Sequelize

I would like to know why my response is "null" when I execute my mutation please.
My query works : the state is_active become true or false in the database SQL when I execute the request.
When I create, I have a correct response, but not on the update.
Graphql request :
mutation {
updateSequence(id: 4, , input: {is_active: true}) {
is_active
}
}
The response :
{
"data": {
"updateSequence": {
"is_active": null
}
}
}
resolvers.js
Mutation : {
createSequence(_, {input}) {
return models.Sequence.create(input);
},
updateSequence(_, {id, input}) {
return models.Sequence.update(input, {where: {id: id}});
}
}
schema.js
# Sequence
type Sequence {
id: Int!,
code: String,
buckets: [Bucket],
sequence_types : [SequenceType]
is_active: Boolean,
}
# SequenceInput
input SequenceInput {
is_active: Boolean
}
...
type Query {
sequences: [Sequence]
sequence(id: Int): Sequence
}
type Mutation {
createSequence(input: SequenceInput): Sequence,
updateSequence(id: ID!, input: SequenceInput): Sequence
}
schema {
query: Query,
mutation: Mutation
}
SOLVED : Migrate MySQL to Postegres to use returning option (only Postegres)
I migrate my database to use the returning option from Sequelize.
createSequence(_, {input}) {
return models.Sequence.create(input);
},
updateSequence(_, {id, input}) {
return models.Sequence.update(input, {where: {id: id}, returning: true}).then((sequence) => {
return sequence[1][0].dataValues;
});
},
According to Sequelize Documentation:
The create method returns a Promise<Model>, in your case it returns something that matches type Sequence.
The update method instead, returns a Promise<Array<number, number>> and that does not match type Sequence.
The promise returns an array with one or two elements. The first element is always the number of affected rows, while the second element is the actual affected rows (only supported in postgres with options.returning true.)
So either you change the return type of updateSequence into something that matches the return type from models.Sequence.update, OR, you have to build an object that matches type Sequence after the update.

How to pass JSON object in grpahql and strapi

When I manually write the mutation query (in graphql plugin), it's working:
mutation {
createExam(input: {
data: {
name: "myName"
desription: "ggg"
questions: [{gf: "hello"}]
time: 2
subjects: ["5c468e2d61670b25b46ccdfe"]
}
}) {
exam {
name
desription
time
}
}
}
But if I code it and pass the exact same array I get an array of the exact same object I get [null, null]
let parsedQuestion = [{gf: "hello"}];
const response = await strapi.request('POST', '/graphql', {
data: {
query: `mutation {
createExam(input: {
data: {
name: "` + examInfo.newExamName + `"
desription: "` + examInfo.newExamDescription + `"
time: ` + Number(examInfo.newExamTime) + `,
questions: `+ parsedQuestion + `,
subjects: ["` + this.state.modalSubject._id + `"]
}
}) {
exam {
name
desription
time
questions
}
}
}`
}
How can it be? Could it be a bug? I also tried with JSON.stringify but then got an error and the mutation didn't even come through
Thanks a lot in advance
Constructing a query string this way is error-prone and dangerous; it opens you up to a slew of bugs and well-known security vulnerabilities. (What if newExamName is My "super-duper" exam!!!?)
GraphQL provides variables as a better approach to pass data in. In your case since you have a complex somewhat structured object, it's probably easiest to pass the whole input in as one object (other syntaxes are possible). I would expect this to look something like:
const response = await strap.request('POST', '/graphql', {
data: {
query: `mutation CreateExam($input: CreateExamInput!) {
createExam(input: $input) {
exam { name, desription, time, questions }
}
}`,
variables: {
input: {
name: examInfo.newExamName,
desription: examInfo.newExamDescription,
time: Number(examInfo.newExamTime),
questions: [{gf: "hello"}],
subjects: [this.state.modalSubject._id]
}
}
}
});
Now the HTTP client library can take responsibility for producing well-formed JSON from your input, and you're not performing tricky string manipulation.

Express unknown string object key in TypeScript

I'm creating a TypScript Declaration file for a client library with a method that returns an object having a variable string key (a record id):
getRecords(...): {
[string]: { // ??
first: string,
last: string,
...
}
};
What is the correct way to specify that the key value is only known at runtime, and what is the reference for this?
getRecords returns something like:
{
'GuT9b...pX': {
first: 'John',
last: 'Doe',
...
},
'aMe4T...lk': {
first: 'Jane',
last: 'Doe',
....
}
}
(Note that this is a wrapper for an HTTP API written in PHP so I have no control over the result being an associative array (i.e. JavaScript hash), instead of an array with an .id field in each element.)
It's called indexable type:
function getRecords(key: string): { [s: string]: { first: string /*, ...*/ } } {
return { [key]: { first: 'John' } }
}