Have a JSON response
{
"id": "1",
"result": {
"data": [
{
"title": "lorem ipsum",
"size": 92,
"Links for query #1 random23": 27,
"Links for query #2 random122": 227,
"Links for query #3 random15": 13,
},
{
"title": "deloren ima",
"size": 11,
"Links for query #1 random23": 27,
"Links for query #2 random122": 227,
"Links for query #3 random15": 13,
}
]
}
}
There have fields starts with "Links for query #1/2/3" and random suffix.
Can I validate this fields names with RegEXP in JSON schema?
Exampl
"properties": {
"data": {
"title": "The data Schema",
"type": "array",
"default": [],
"items": {
"title": "A Schema",
"type": "object",
"required": [
"title",
"size",
"^(Links for query #1+\\s\\w+)$",
"^(Links for query #2+\\s\\w+)$",
"^(Links for query #3+\\s\\w+)$"
],
Related
I have a simple JSON schema that's used to define a tree of objects that are labeled with an Object Identifier. These objects are recursive and look identical, so the JSON schema just uses a "$ref": "#".
What I would like to do is validate that the actual numeric OID structure is constructed correctly. Right now I'm validating that it starts with the right prefix (1.2.3), but if the subsequent structure is screwed up (e.g. the parent is 1.2.3.6 but someone adds children of 1.2.3.6.1 and 1.2.3.2.2) it won't be caught.
I'm not sure how I can reference the oid field in the parent object to check against the child. Can someone point me in the right direction?
My schema:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://my-site/my-schema.json",
"title": "My JSON Schema",
"description": "A fancy JSON schema for a tree of objects",
"type": "object",
"properties": {
"oid": {
"description": "The OID",
"type": "string",
"pattern": "^1.2.3"
},
"description": {
"description": "A description",
"type": "string"
},
"children": {
"description": "Children",
"type": [
"array",
"null"
],
"items": {
"$ref": "#"
}
}
},
"required": [
"oid",
"description",
"children"
]
}
Example data:
{
"oid": "1.2.3",
"description": "Parent",
"children": [
{
"oid": "1.2.3.1",
"description": "Group 1",
"children": [
{
"oid": "1.2.3.1.1",
"description": "Group 1.1",
"children": [
{
"oid": "1.2.3.1.1.1",
"description": "Item 1.1.1",
"children": null
}
]
},
{
"oid": "1.2.3.1.2",
"description": "Group 1.2",
"children": [
{
"oid": "1.2.3.1.2.1",
"description": "Item 1.2.1",
"children": null
},
{
"oid": "1.2.3.1.2.2",
"description": "Item 1.2.2",
"children": null
}
]
}
]
}
]
}
Hellow jq experts!
I'm a jq learner and have a json obect composed lists as follows:
{
"image_files": [
{
"id": "img_0001",
"width": 32,
"heigt": 32,
"file_name": "img_0001.png"
},
{
"id": "img_0002",
"width": 128,
"heigt": 32,
"file_name": "img_0002.png"
},
{
"id": "img_0003",
"width": 32,
"heigt": 32,
"file_name": "img_0003.png"
},
{
"id": "img_0004",
"width": 160,
"heigt": 32,
"file_name": "img_0004.png"
}
],
"annotations": [
{
"id": "ann_0001",
"image_id": "img_0001",
"label": "A",
"attributes": {
"type": "letter",
"augmented": false
}
},
{
"id": "ann_0002",
"image_id": "img_0002",
"label": "Good",
"attributes": {
"type": "word",
"augmented": false
}
},
{
"id": "ann_0003",
"image_id": "img_0003",
"label": "C",
"attributes": {
"type": "letter",
"augmented": false
}
},
{
"id": "ann_0004",
"image_id": "img_0004",
"label": "Hello",
"attributes": {
"type": "word",
"augmented": false
}
}
]
}
image_id in the annotations list are foreign key referencing the id in the image_files list.
I want to join image_files and annotations with condition of annotations.attribute.type == "letter".
Expecting following ouptut:
{
"letter_image_files_with_label": [
{
"id": "img_0001",
"width": 32,
"heigt": 32,
"file_name": "img_0001.png",
"label": "A"
},
{
"id": "img_0003",
"width": 32,
"heigt": 32,
"file_name": "img_0003.png",
"label": "C"
}
]
}
How can I produce above result from the json data input?
join explained in jq manual does not seem to use this kind task.
Is there a way for this? Please show me the rope.
Thanks for your generous reading.
Indexing image_files with ids makes this pretty trivial.
INDEX(.image_files[]; .id) as $imgs | [
.annotations[]
| select(.attributes.type == "letter")
| $imgs[.image_id] + {label: .label}
]
Online demo
I have been working on this for a couple days and cannot get past this error. I have 2 activities in this pipeline. The first activity copies data from an ODBC connection to an Azure database, which is successful. The 2nd activity transfers the data from Azure table to another Azure table and keeps failing.
The error message is:
Copy activity met invalid parameters: 'UnknownParameterName', Detailed message: An item with the same key has already been added..
I do not see any invalid parameters or unknown parameter names. I have rewritten this multiple times using their add activity code template and by myself, but do not receive any errors when deploying on when it is running. Below is the JSON pipeline code.
Only the 2nd activity is receiving an error.
Thanks.
Source Data set
{
"name": "AnalyticsDB-SHIPUPS_06shp-01src_AZ-915PM",
"properties": {
"structure": [
{
"name": "UPSD_BOL",
"type": "String"
},
{
"name": "UPSD_ORDN",
"type": "String"
}
],
"published": false,
"type": "AzureSqlTable",
"linkedServiceName": "Source-SQLAzure",
"typeProperties": {},
"availability": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"external": true,
"policy": {}
}
}
Destination Data set
{
"name": "AnalyticsDB-SHIPUPS_06shp-02dst_AZ-915PM",
"properties": {
"structure": [
{
"name": "SHIP_SYS_TRACK_NUM",
"type": "String"
},
{
"name": "SHIP_TRACK_NUM",
"type": "String"
}
],
"published": false,
"type": "AzureSqlTable",
"linkedServiceName": "Destination-Azure-AnalyticsDB",
"typeProperties": {
"tableName": "[olcm].[SHIP_Tracking]"
},
"availability": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"external": false,
"policy": {}
}
}
Pipeline
{
"name": "SHIPUPS_FC_COPY-915PM",
"properties": {
"description": "copy shipments ",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "RelationalSource",
"query": "$$Text.Format('SELECT COMPANY, UPSD_ORDN, UPSD_BOL FROM \"orupsd - UPS interface Dtl\" WHERE COMPANY = \\'01\\'', WindowStart, WindowEnd)"
},
"sink": {
"type": "SqlSink",
"sqlWriterCleanupScript": "$$Text.Format('delete imp_fc.SHIP_UPS_IntDtl_Tracking', WindowStart, WindowEnd)",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
},
"translator": {
"type": "TabularTranslator",
"columnMappings": "COMPANY:COMPANY, UPSD_ORDN:UPSD_ORDN, UPSD_BOL:UPSD_BOL"
}
},
"inputs": [
{
"name": "AnalyticsDB-SHIPUPS_03shp-01src_FC-915PM"
}
],
"outputs": [
{
"name": "AnalyticsDB-SHIPUPS_03shp-02dst_AZ-915PM"
}
],
"policy": {
"timeout": "1.00:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval",
"retry": 3,
"longRetry": 0,
"longRetryInterval": "00:00:00"
},
"scheduler": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"name": "915PM-SHIPUPS-fc-copy->[imp_fc]_[SHIP_UPS_IntDtl_Tracking]"
},
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "SqlSource",
"sqlReaderQuery": "$$Text.Format('select distinct ups.UPSD_BOL, ups.UPSD_BOL from imp_fc.SHIP_UPS_IntDtl_Tracking ups LEFT JOIN olcm.SHIP_Tracking st ON ups.UPSD_BOL = st.SHIP_SYS_TRACK_NUM WHERE st.SHIP_SYS_TRACK_NUM IS NULL', WindowStart, WindowEnd)"
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
},
"translator": {
"type": "TabularTranslator",
"columnMappings": "UPSD_BOL:SHIP_SYS_TRACK_NUM, UPSD_BOL:SHIP_TRACK_NUM"
}
},
"inputs": [
{
"name": "AnalyticsDB-SHIPUPS_06shp-01src_AZ-915PM"
}
],
"outputs": [
{
"name": "AnalyticsDB-SHIPUPS_06shp-02dst_AZ-915PM"
}
],
"policy": {
"timeout": "1.00:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval",
"retry": 3,
"longRetryInterval": "00:00:00"
},
"scheduler": {
"frequency": "Day",
"interval": 1,
"offset": "04:15:00"
},
"name": "915PM-SHIPUPS-AZ-update->[olcm]_[SHIP_Tracking]"
}
],
"start": "2017-08-22T03:00:00Z",
"end": "2099-12-31T08:00:00Z",
"isPaused": false,
"hubName": "adf-tm-prod-01_hub",
"pipelineMode": "Scheduled"
}
}
Have you seen this link?
They get the same error message and suggest using AzureTableSink instead of SqlSink
"sink": {
"type": "AzureTableSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
}
It would make sense for you too since your 2nd copy activity is Azure to Azure
It could be a red herring but I'm pretty sure "tableName" is a require entry in the typeProperties for a sqlSource. Yours is missing this for the input dataset. Appreciate you have a join in the sqlReaderQuery so probably best to put a dummy (but real) table name in there.
Btw, not clear why you are using $$Text.Format and WindowStart/WindowEnd on your queries if you're not transposing these values into the query; you could just put the query between double quotes.
I am trying to access completed form field data in ServiceM8 including uploaded images. The api reference does not detail how to access them. Can anyone provide an example or documentation?
It's not publicly documented, but you can access completed forms via the FormResponses endpoint.
https://api.servicem8.com/api_1.0/FormResponse.json
[{
"edit_date": "2015-02-17 11:12:22",
"active": 1,
"form_uuid": "578344cc-8d57-441b-a713-02207430f7cb",
"staff_uuid": "",
"regarding_object": "job",
"regarding_object_uuid": "8b9bfe73-c43c-4275-bb4e-1aac9b6f8adb",
"field_data": "removed (see below)",
"timestamp": "2015-02-17 11:12:20",
"form_by_staff_uuid": "0e72af0f-22db-4ddb-ac7d-eba2c1ef2d4b",
"document_attachment_uuid": "6b73bfa3-3dd8-48b7-83b2-af1dcc78262b",
"uuid": "ff1926c8-8879-41a5-9f2d-e4e4455e92eb"
}]
The actual responses to the questions are in the field_data property, which is itself a JSON-encoded string. Here's an example:
[
{
"UUID": "1ea64222-2b0e-4dc4-a99b-698603a9f59b",
"FieldType": "Date",
"Response": "Tue Feb 17 2015 00:00:00 GMT+0930 (ACST)",
"SortOrder": 1,
"Question": "Date"
},
{
"UUID": "9b4ea087-d154-4fb7-831e-c17cb882157b",
"FieldType": "Multiple Choice",
"Response": "Blah",
"SortOrder": 2,
"Question": "Supplier"
},
{
"UUID": "24fdd846-8fc8-4ddf-9d58-6c5fa1b5310b",
"FieldType": "Text",
"Response": "1",
"SortOrder": 3,
"Question": "Part 1 Qty"
},
{
"UUID": "6162749f-5945-455c-95ca-ca31e9bd6a7b",
"FieldType": "Text",
"Response": "p1",
"SortOrder": 4,
"Question": "Item Part No. 1"
},
{
"UUID": "fd5c8f67-c68d-4dcb-96a7-05341fbb3dab",
"FieldType": "Text",
"Response": "part one",
"SortOrder": 5,
"Question": "Item Name 1"
},
{
"UUID": "6e591103-78d4-42a0-8657-5229b5b469fb",
"FieldType": "Photo",
"Response": "cf91bae2-b9da-42e2-87ee-b416109bb44b",
"SortOrder": 14,
"Question": "Take Photo of Items 1"
},
{
"UUID": "c8719af5-b550-40cf-9d83-0ea67ab30e5b",
"FieldType": "Signature",
"Response": "90ac5b56-fc0e-4a3a-8935-f43cdd23c05b",
"SortOrder": 17,
"Question": "Signature"
}
]
For photo questions (FieldType == Photo), the Response property contains a UUID which you can retrieve from the Attachments endpoint.
JSON data for the attachment: https://api.servicem8.com/api_1.0/Attachment/cf91bae2-b9da-42e2-87ee-b416109bb44b.json
File data (i.e. the attachment itself): https://api.servicem8.com/api_1.0/Attachment/cf91bae2-b9da-42e2-87ee-b416109bb44b.file
I am trying to use Validate JSON schema component in Mule flow and I am getting com.fasterxml.jackson.core.JsonParseException for the json that is passed. Below is the json schema, json sample and code of Mule flow. Can you please point me where am I doing mistake?
Json schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "object",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"minimum": 0,
"exclusiveMinimum": true
}
},
"required": ["id", "name", "price"]
}
Json passed to POST method:
[
{
"id": 2,
"name": "An ice sculpture",
"price": 12.50,
},
{
"id": 3,
"name": "A blue mouse",
"price": 25.50,
}
]
Error :
Root Exception stack trace:
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('}' (code 125)): was expecting double-quote to start field name
at [Source: java.io.InputStreamReader#6e7f030; line: 6, column: 5]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
Mule Flow:
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration"/>
<flow name="jsonschemavalidationFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/" allowedMethods="POST" doc:name="HTTP"/>
<json:validate-schema schemaLocation="jsonschema.json" doc:name="Validate JSON Schema"/>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
There are few errors in both the JSON and the Schema as follows:-
There is an extra comma, after "price": 12.50
So the Valid JSON will be :-
[
{
"id": 2,
"name": "An ice sculpture",
"price": 12.50
},
{
"id": 3,
"name": "A blue mouse",
"price": 25.50
}
]
In JSON Schema file jsonschema.json There are two errors:-
You need to put "type": "array" instead of "type": "object"
and next is "exclusiveMinimum": true seems to be invalid with this JSON input...
So, the correct working JSON schema will be :-
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "array",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"minimum": 0
}
},
"required": ["id", "name", "price"]
}
Try it.. it will work fine :)
The json u r passing is not proper. It has an extra comma ',' after price: 12.50. Even after second price element also an extra comma is added.
Just remove it and it works fine.