Related
Guy, long term looking at this board and learning a lot but now stuck with little issue. Im working with Linux shell script that reads json (no problem here). What Im trying to do is get value from entry that has specific Type.
By parsing a json with just jq -r '.', I get
{
"records": [
{
"id": 01,
"type": "SOA",
"name": "#",
"data": "1800",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
},
{
"id": 02,
"type": "A",
"name": "#",
"data": "test.com",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
}
],
"links": {},
"meta": {
"total": 2
}
}
Then, I use "jq -r '.records' " and get:
[
{
"id": 01,
"type": "SOA",
"name": "#",
"data": "1800",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
},
{
"id": 02,
"type": "A",
"name": "#",
"data": "test.com",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
}
]
What I need to do is get data value of the type A. Currently, we have type SOA and A, but I need to only get data from A.
I can use dummy way of "jq -r '.records[1].data'" and it gives me correct response of test.com, but I want a more dynamic way of searching for specific type (in this case "A") and then giving the data value.
Thanks guys!
Is the field called domain_records or just records?
Use select to match your criteria
jq -r '.records[] | select(.type == "A").data'
test.com
Demo
so I am working with Clickup API, there we have a number of lists we would like to fetch the data from and insert that data as a wordpress post for our website.
I have tested all the API requests with Postman and the data i get back is similar to this, basically would need to fetch the data with "custom fields", but I am not sure how to work it out with PHP, any help would be appreciated.
"custom_fields": [
{
"id": "39ed2191-443e-4d45-84d4-de9eff89cc97",
"name": "Cost",
"type": "currency",
"type_config": {
"default": null,
"precision": 2,
"currency_type": "USD"
},
"date_created": "1616008796860",
"hide_from_guests": false,
"required": false
},
{
"id": "bcca51b8-a903-4194-8d24-8e663de0618e",
"name": "Deadline",
"type": "short_text",
"type_config": {},
"date_created": "1613070997077",
"hide_from_guests": false,
"required": false
},
{
"id": "35af20c5-5469-40b6-afc7-d7e9d5f3c451",
"name": "Industry",
"type": "drop_down",
"type_config": {
"default": 0,
"placeholder": null,
"new_drop_down": true,
"options": [
{
"id": "227658be-5cc6-4c33-a539-029c13a18416",
"name": "Agriculture",
"color": null,
"orderindex": 0
},
{
"id": "2d5041ff-e557-42ae-859a-51698d01d2f7",
"name": "Automation",
"color": null,
"orderindex": 1
},
{
"id": "d2c33aa5-de68-4f16-ad79-5308893c66a6",
"name": "Beauty",
"color": null,
"orderindex": 2
},
{
"id": "bcbf40c4-bcb8-41c4-81ed-7afffca9eccd",
"name": "Construction",
"color": null,
"orderindex": 3
},
{
"id": "47bc5b7d-0fb9-4c64-8887-249e55d31f85",
"name": "Sanitation",
"color": null,
"orderindex": 4
},
{
"id": "10b0e274-4996-4cc9-9c1e-4d44f78694d1",
"name": "Travel",
"color": null,
"orderindex": 5
}
]
},
"date_created": "1639080879165",
"hide_from_guests": false,
"required": false
},
Here's a very abstract description of the possible steps you'll need to take:
Fetch the data using wp_remote_request()
Parse the data using json_decode()
Get all the data you need from the object(-s)
Create a post using wp_insert_post()
Save all the additional data in the post(-s) meta fields using update_post_meta()
Hello internet people,
Apparently I had the same problem of many but after searching a lot even here I could not find a reasonable answer at all!
My problem:
On shell I did enter the command:
stripe invoices list --customer MY_CUSTOMER_ID_HERE
Then the following JSON appeared:
{
"object": "list",
"data": [
{
"id": "INVOICE_ID",
"object": "invoice",
"account_country": "COUNTRY",
"account_name": "MY_BUSYNESS_NAME",
"account_tax_ids": null,
"amount_due": 1000,
"amount_paid": 1000,
"amount_remaining": 0,
"application_fee_amount": null,
"attempt_count": 1,
"attempted": true,
"auto_advance": false,
"billing_reason": "subscription_create",
"charge": "CHARGE_ID",
"collection_method": "charge_automatically",
"created": SOME_UNIXTIME,
"currency": "usd",
"custom_fields": null,
"customer": "CUSTOMER_ID",
"customer_address": null,
"customer_email": "CUSTOMER_EMAIL",
"customer_name": null,
"customer_phone": "CUSTOMER_PHONE",
"customer_shipping": null,
"customer_tax_exempt": "none",
"customer_tax_ids": [
],
"default_payment_method": null,
"default_source": null,
"default_tax_rates": [
],
"description": null,
"discount": null,
"discounts": [
],
"due_date": null,
"ending_balance": 0,
"footer": null,
"hosted_invoice_url": "INVOICE_URL_GOES_HERE",
"invoice_pdf": "INVOICE_PDF_URL_GOES_HERE",
"last_finalization_error": null,
"lines": {
"object": "list",
"data": [
{
"id": "ID",
"object": "line_item",
"amount": 1000,
"currency": "usd",
"description": "PLAN_DESCRIPTION",
"discount_amounts": [
],
"discountable": true,
"discounts": [
],
"livemode": false,
"metadata": {
},
"period": {
"end": SOME_UNIXTIME,
"start": SOME_UNIXTIME
},
"plan": {
"id": "PRICE_ID_HERE",
"object": "plan",
"active": true,
"aggregate_usage": null,
"amount": 1000,
"amount_decimal": "1000",
"billing_scheme": "per_unit",
"created": SOME_UNIXTIME,
"currency": "usd",
"interval": "month",
"interval_count": 1,
"livemode": false,
"metadata": {
},
"nickname": null,
"product": "PRODUCT_ID_HERE",
"tiers_mode": null,
"transform_usage": null,
"trial_period_days": null,
"usage_type": "licensed"
},
"price": {
"id": "MORE_ID_HERE",
"object": "price",
"active": true,
"billing_scheme": "per_unit",
"created": SOME_UNIXTIME,
"currency": "usd",
"livemode": false,
"lookup_key": null,
"metadata": {
},
"nickname": null,
"product": "PRODUCT_ID_HERE",
"recurring": {
"aggregate_usage": null,
"interval": "month",
"interval_count": 1,
"trial_period_days": null,
"usage_type": "licensed"
},
"tiers_mode": null,
"transform_quantity": null,
"type": "recurring",
"unit_amount": 1000,
"unit_amount_decimal": "1000"
},
"proration": false,
"quantity": 1,
"subscription": "SUBSCRIPTION_ID_HERE",
"subscription_item": "SUBS_ITEM_ID_HERE",
"tax_amounts": [
],
"tax_rates": [
],
"type": "subscription"
}
],
"has_more": false,
"total_count": 1,
"url": "IV_URL_HERE"
},
"livemode": false,
"metadata": {
},
"next_payment_attempt": null,
"number": "NUMBER_IV_XXXX",
"on_behalf_of": null,
"paid": true,
"payment_intent": "PAYMENT_INTENT_ID_HERE",
"payment_settings": {
"payment_method_options": null,
"payment_method_types": null
},
"period_end": SOME_UNIXTIME,
"period_start": SOME_UNIXTIME,
"post_payment_credit_notes_amount": 0,
"pre_payment_credit_notes_amount": 0,
"receipt_number": null,
"starting_balance": 0,
"statement_descriptor": null,
"status": "paid",
"status_transitions": {
"finalized_at": SOME_UNIXTIME,
"marked_uncollectible_at": null,
"paid_at": SOME_UNIXTIME,
"voided_at": null
},
"subscription": "SUBSCRIPTION_ID_HERE",
"subtotal": 1000,
"tax": null,
"total": 1000,
"total_discount_amounts": [
],
"total_tax_amounts": [
],
"transfer_data": null,
"webhooks_delivered_at": SOME_UNIXTIME
}
],
"has_more": false,
"url": "/some_path"
}
I'm able to access the second level with this command:
stripe invoices list --customer MY_CUSTOMER_ID_HERE | [.id,.attempt_count,.billing_reason,.created,.customer,.customer_email,.lines.data[],.paid,.status]'
Which returns a nice response, but I do want to filter few values inside data (data.lines.data[]) like id, plan and inside plan the id, currency and product for example.
My question:
Does any one knows how to do that with this kind of commands?
Just in case, those are the questions and answer that I tried but couldn't figured how to do it at all!
How to filter array of objects by element property values using jq?
How to filter arrays in json based on specific value using jq
How to filter nested arrays in JSON while maintaining structure using jq
filtering from JSON output from curl using JQ
Filtering JSON by object name using jq
How to filter JSON using jq stream (duplicate)
How to filter and replace values in json with jq
Filtering JSON list in shell using jq
I did also try a lot attempts on my own, like:
stripe invoices list --customer MY_CUSTOMER_ID_HERE | [.id,.attempt_count,.billing_reason,.created,.customer,.customer_email,.lines.data[.id.plan[][.id,.plan[.id,.currency,.product]],.paid,.status]'
Response:
jq: error (at :179): Cannot index string with string "plan"
stripe invoices list --customer MY_CUSTOMER_ID_HERE | [.id,.attempt_count,.billing_reason,.created,.customer,.customer_email,.lines.data[][],.paid,.status]'
This one return a null value for the second data content.
Response:
[
"DESIRED_ID",
1,
"value_retrieved_ok",
unixtime_ok,
"SOME_ID_OK",
"customer_mail#ok.com",
null, <<< ???
true,
"paid"
]
EDIT:
Desired result should look like:
[
"DESIRED_ID",
1,
"value_retrieved_ok",
unixtime_ok,
"SOME_ID_OK",
"customer_mail#ok.com",
"id" // the id inside lines.data{}
["id", // the id inside lines.data.plan
"currency", // the currency inside lines.data.plan
"product" // the product inside lines.data.plan
]
true,
"paid"
]
Anyway I do hope that there's a good soul out there have mercy!
Your requirements are a bit difficult to follow, especially since the "expected output" does not correspond exactly to the given input, but the following is either what you're looking for, or very close to it:
.data[]
| [.id,.attempt_count,.billing_reason,.created,.customer,.customer_email]
+ (.lines.data[] | [.id, [.plan.id, .plan.currency, .plan.product]])
+ [.paid,.status]
Im using Debezium to sync data from MySQL to S3. Now I want to make some changes.
Sample insert:
create table new (id int);
insert into new (1);
1. Custom Payload
{
"schema": {
"type": "struct",
bla bla bla
"optional": false,
"name": "_72.31.84.129.test.new.Envelope"
},
"payload": {
"before": null,
"after": {
"id": 10
},
"source": {
"version": "0.10.0.Final",
"connector": "mysql",
"name": "11.11.84.129",
"ts_ms": 1576605998000,
"snapshot": "false",
"db": "test",
"table": "new",
"server_id": 1,
"gtid": "3a7b90e9-207e-11ea-b3ed-121a0cbac3cb:51",
"file": "mysql-bin.000003",
"pos": 12770,
"row": 0,
"thread": 47,
"query": null
},
"op": "c",
"ts_ms": 1576605998231
}
}
I want to only push the payload option with some custom changes. I need to include the source,op,ts_ms are inside the payload.after.
Expected output:
{
"id": 10,
"source": {
"version": "0.10.0.Final",
"connector": "mysql",
"name": "11.11.84.129",
"ts_ms": 1576605998000,
"snapshot": "false",
"db": "test",
"table": "new",
"server_id": 1,
"gtid": "3a7b90e9-207e-11ea-b3ed-121a0cbac3cb:51",
"file": "mysql-bin.000003",
"pos": 12770,
"row": 0,
"thread": 47,
"query": null
},
"op": "c",
"ts_ms": 1576605998231
}
I don't want schema, payload.before. Im not sure how to get this output.
Check out the SMT for extracting the new record state. It will only propagate what's in after. Optionally, you can let it add chosen field from source, too.
...
transforms=unwrap
transforms.unwrap.type=io.debezium.transforms.ExtractNewRecordState
transforms.unwrap.add.source.fields=table,lsn
...
You cannot insert the op and ts_ms fields atm., but they can be propgated as message headers.
I have a JSON of the below format
{
"board_title": "test",
"read_only": false,
"isIntegration": false,
"board_bgtype": "board_graph",
"created": "2017-08-16T06:40:47.158868+00:00",
"original_title": "Revised_CID_Templating-test(cloned)",
"modified": "2017-08-31T11:52:22.115661+00:00",
"disableEditing": false,
"height": 111,
"width": "100%",
"template_variables": [
{
"default": "identity",
"prefix": "v1",
"name": "env"
}
],
"created_by": {
"disabled": false,
"handle": "xx.com",
"name": null,
"is_admin": false,
"role": null,
"access_role": "st",
"verified": true,
"email": "xx.com",
"icon": "https://secure.gravatar.com/avatar/86fd6c17deba27cfc4081134a5bc0c6a?s=48&d=retro"
},
...
}
I need to load this JSON using Python and edit the value of
"template_variables": [
{
"default": "identity"
To some other value, say:
default : "com"
I load it using Python script and traverse and print the key value pair but not able to understand how to modify it.
How can I assign a value to the first child of template variables and save the JSON in the same file and post it using request?
You can think of json object as a dictionary.
try this.
import json
jsonData = '''{
"board_title": "test",
"read_only": false,
"isIntegration": false,
"board_bgtype": "board_graph",
"created": "2017-08-16T06:40:47.158868+00:00",
"original_title": "Revised_CID_Templating-test(cloned)",
"modified": "2017-08-31T11:52:22.115661+00:00",
"disableEditing": false,
"height": 111,
"width": "100%",
"template_variables": [
{
"default": "identity",
"prefix": "v1",
"name": "env"
}
],
"created_by": {
"disabled": false,
"handle": "xx.com",
"name": null,
"is_admin": false,
"role": null,
"access_role": "st",
"verified": true,
"email": "xx.com",
"icon": "https://secure.gravatar.com/avatar/86fd6c17deba27cfc4081134a5bc0c6a?s=48&d=retro"
}
}'''
jsonToPython = json.loads(jsonData)
print (jsonToPython['template_variables'][0]['default'])
jsonToPython['template_variables'][0]['default'] = 'test'
print (jsonToPython['template_variables'][0]['default'])
as you can see jsonToPython is being modified.