Related
Hi I am trying to select Status from Json Array in azure data factory
{
"dataRead": 2997,
"dataWritten": 2714,
"filesWritten": 1,
"sourcePeakConnections": 1,
"sinkPeakConnections": 1,
"rowsRead": 11,
"rowsCopied": 11,
"copyDuration": 3,
"throughput": 0.976,
"errors": [],
"effectiveIntegrationRuntime": "DefaultIntegrationRuntime (East US)",
"usedDataIntegrationUnits": 4,
"billingReference": {
"activityType": "DataMovement",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.06666666666666667,
"unit": "DIUHours"
}
]
},
"usedParallelCopies": 1,
"executionDetails": [
{
"source": {
"type": "AzureSqlDatabase",
"region": "East US"
},
"sink": {
"type": "AzureBlobStorage",
"region": "East US"
},
"status": "Succeeded",
"start": "2020-03-19T06:24:39.0666585Z",
"duration": 3,
"usedDataIntegrationUnits": 4,
"usedParallelCopies": 1,
I have tried selecting #activity('Copy data From CCP TO Blob').output.executionDetails.status.It throws an error:
'Array elements can only be selected using an integer index'.
Any way to resolve it?
executionDetails is an array, you have to set index to refer elements in it.
Please try:
#activity('Copy data From CCP TO Blob').output.executionDetails[0].status
Thank you for the reply
Yes, we have to use slicing and indexing the lists and Dictionaries
I have tried Dispensing_Unit_Master_Dim
#activity('Copy data From CCP TO Blob').output.executionDetails[0]['status'] and it works
0 and status there is no Dot
I have a JSON list with 253 entries and 7 records where the 7th record is a list with usually 2 entries.
I'm trying to convert this in Power Query to give me a table output. I've only really ever used the basic connection in Excel to do this automatically without problems.
My current error is:
Expression.Error: We cannot convert a value of type Record to type Text.
Details:
Value=Record
Type=Type
I'm not sure where to go and I can't see examples for my specific situation, although I'm sure there are plenty of examples - just that I'm not good enough with this to understans
[
{
"Id": "lorum-ipsum1",
"Description": "sitename 1",
"Latitude": 1.0,
"Longitude": -1.0,
"Postcode": "AB1 2CD",
"CountryCode": "GB",
"Connectors": [
{
"Id": "lorum-ipsum1-a",
"Number": 1,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP00000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
},
{
"Id": "lorum-ipsum1-b",
"Number": 2,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP0000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
}
]
},
{
"Id": "lorum-ipsum2",
"Description": "sitename 2",
"Latitude": 1.0,
"Longitude": -1.0,
"Postcode": "AB1 2CD",
"CountryCode": "GB",
"Connectors": [
{
"Id": "lorum-ipsum2-a",
"Number": 1,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP00000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
},
{
"Id": "lorum-ipsum2-b",
"Number": 2,
"Status": 1,
"ErrorCode": "NoError",
"ChargepointName": "GP0000",
"Shape": 0,
"Mode": 0,
"Volts": 240,
"Amps": 0,
"Phase": 2,
"PricingInformationUrl": "hrefhere",
"UsageRestrictions": []
}
]
},
I think you want to use Table.FromRecords function to transform the JSON into a table. However, since the Connectors field of each record is itself a list of records, I think you will need to call it a second time.
To give you an example:
let
serialised = "[{""Id"":""lorum-ipsum1"",""Description"":""sitename 1"",""Latitude"":1,""Longitude"":-1,""Postcode"":""AB1 2CD"",""CountryCode"":""GB"",""Connectors"":[{""Id"":""lorum-ipsum1-a"",""Number"":1,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP00000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]},{""Id"":""lorum-ipsum1-b"",""Number"":2,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP0000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]}]},{""Id"":""lorum-ipsum2"",""Description"":""sitename 2"",""Latitude"":1,""Longitude"":-1,""Postcode"":""AB1 2CD"",""CountryCode"":""GB"",""Connectors"":[{""Id"":""lorum-ipsum2-a"",""Number"":1,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP00000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]},{""Id"":""lorum-ipsum2-b"",""Number"":2,""Status"":1,""ErrorCode"":""NoError"",""ChargepointName"":""GP0000"",""Shape"":0,""Mode"":0,""Volts"":240,""Amps"":0,""Phase"":2,""PricingInformationUrl"":""hrefhere"",""UsageRestrictions"":[]}]}]",
deserialised = Json.Document(serialised),
toTable = Table.FromRecords(deserialised),
transformConnectors = Table.TransformColumns(toTable, {{"Connectors", Table.FromRecords}})
in
transformConnectors
which gives me:
From there, you can explore/continue the rest of the transformation yourself (since you haven't specified what you want).
You can expand some/all nested columns inside of the Connectors column by either clicking the icon highlighted in the image above -- or writing any necessary M code.
I am using AWS Step Functions which utilizes JSONPath for providing JSON paths. I have the following input :
{
"response": {
"isSuccess": true,
"error": "",
"body": {
"count": 2,
"fields": [
{
"fieldId": 1,
"tabId": 100,
"title": "First Name"
},
{
"fieldId": 2,
"tabId": 100,
"title": "Last Name"
}
]
}
},
"iteration": {
"totalCount": 2,
"currentCount": 0,
"step": 1
}
}
I want to query the fields array as:
$.response.body.fields[$.iteration.currentCount]
The value of currentCount is incremented by 1 as part of an iteration.
I am getting an invalid XPath exception when trying to use the above.
Can someone please advice on how to provide a dynamic property value to read array values?
As described on https://github.com/json-path/JsonPath#operators you can index an array with a number only. However, you can use a filter expression to select a specific item from the array as follows.
Assuming you have another field that denotes the index such as:
{
"index": 0,
"fieldId": 2,
"tabId": 100,
"title": "Last Name"
}
You can then do
$.response.body.fields[?(#.index==$.iteration.currentCount)]
LR script throws warning and fails for the JSON Response.
Below is the script
char *request_json_base;
char *request_json;
web_add_header("Content-Type", "application/json; charset=utf-8");
request_json_base= "{\"GLDefaultId\":0,\"ProjectId\":{ContactProjectId},\"CoaMainId\":10,\"UserId\":{UserId},\"Notes\":\"\",\"PRO_GLDefaultValue\":[{\"GLDefaultValueId\":0,\"GLDefaultId\":0,\"SegmentId\":16,\"Alias\":\"ACT\",\"SegmentType\":\"DETAIL\",\"FunctionTypeId\":0,\"Value\":\"&&&\",\"Mask\":\"&&&\",\"IsRequired\":true,\"LastModified\":null,\"IsCoaRequired\":true},{\"GLDefaultValueId\":0,\"GLDefaultId\":0,\"SegmentId\":17,\"Alias\":\"ST\",\"SegmentType\":\"SET\",\"FunctionTypeId\":0,\"Value\":\"&&\",\"Mask\":\"&&\",\"IsRequired\":false,\"LastModified\":null,\"IsCoaRequired\":false}]}",
request_json = lr_eval_string(request_json_base);
lr_save_string(request_json, "REQUEST_JSON_PARAM");
web_custom_request("PRO_GLDefault",
"URL=https://{URL}/PayrollWebApi/V1/PRO_GLDefault?userId={UserId}",
"Method=POST",
"TargetFrame=",
"Resource=0",
"RecContentType=application/json",
"Referer=",
"Snapshot=t299.inf",
"Mode=HTML",
"EncType=application/json",
"Body={REQUEST_JSON_PARAM}",
LAST);
Below is Replay Log :
Action3_SUNP_ProjectSetup_RatesandSetupInfo.c(81): Warning: The string '"GLDefaultValueId":0,"GLDefaultId":0,"SegmentId":16,"Alias":"ACT","SegmentType":"DETAIL","FunctionTypeId":0,"Value":"&&&","Mask":"&&&","IsRequired":true,"LastModified":null,"IsCoaRequired":true' with parameter delimiters is not a parameter.
Action3_SUNP_ProjectSetup_RatesandSetupInfo.c(81): Warning: The string '"GLDefaultValueId":0,"GLDefaultId":0,"SegmentId":17,"Alias":"ST","SegmentType":"SET","FunctionTypeId":0,"Value":"&&","Mask":"&&","IsRequired":false,"LastModified":null,"IsCoaRequired":false' with parameter delimiters is not a parameter.
Action3_SUNP_ProjectSetup_RatesandSetupInfo.c(83): Notify: Saving Parameter "REQUEST_JSON_PARAM = {"GLDefaultId":0,"ProjectId":1810,"CoaMainId":10,"UserId":67,"Notes":"","PRO_GLDefaultValue":[{"GLDefaultValueId":0,"GLDefaultId":0,"SegmentId":16,"Alias":"ACT","SegmentType":"DETAIL","FunctionTypeId":0,"Value":"&&&","Mask":"&&&","IsRequired":true,"LastModified":null,"IsCoaRequired":true},{"GLDefaultValueId":0,"GLDefaultId":0,"SegmentId":17,"Alias":"ST","SegmentType":"SET","FunctionTypeId":0,"Value":"&&","Mask":"&&","IsRequired":false,"LastModified":null,"IsCoaRequired":false}]}".
Action3_SUNP_ProjectSetup_RatesandSetupInfo.c(85): {"$id":"1","Message":"An error has occurred."}
Below is Body of Original Request call:
"Body={\"GLDefaultId\":0,\"ProjectId\":{ContactProjectId},\"CoaMainId\":10,\"UserId\":{UserId},\"Notes\":\"\",\"PRO_GLDefaultValue\":[{\"GLDefaultValueId\":0,\"GLDefaultId\":0,\"SegmentId\":16,\"Alias\":\"ACT\",\"SegmentType\":\"DETAIL\",\"FunctionTypeId\":0,\"Value\":\"&&&\",\"Mask\":\"&&&\",\"IsRequired\":true,\"LastModified\":null,\"IsCoaRequired\":true},{\"GLDefaultValueId\":0,\"GLDefaultId\":0,\"SegmentId\":17,\"Alias\":\"ST\",\"SegmentType\":\"SET\",\"FunctionTypeId\":0,\"Value\":\"&&\",\"Mask\":\"&&\",\"IsRequired\":false,\"LastModified\":null,\"IsCoaRequired\":false}]}",
LAST);
Below is the Successful Response during the recording:
Request :
{
"GLDefaultId": 0,
"ProjectId": 1806,
"CoaMainId": 10,
"UserId": 2,
"Notes": "",
"PRO_GLDefaultValue": [
{
"GLDefaultValueId": 0,
"GLDefaultId": 0,
"SegmentId": 16,
"Alias": "ACT",
"SegmentType": "DETAIL",
"FunctionTypeId": 0,
"Value": "&&&",
"Mask": "&&&",
"IsRequired": true,
"LastModified": null,
"IsCoaRequired": true
},
{
"GLDefaultValueId": 0,
"GLDefaultId": 0,
"SegmentId": 17,
"Alias": "ST",
"SegmentType": "SET",
"FunctionTypeId": 0,
"Value": "&&",
"Mask": "&&",
"IsRequired": false,
"LastModified": null,
"IsCoaRequired": false
}
]
}
Response:
{
"$id": "1",
"GLDefaultId": 1,
"ProjectId": 1806,
"CoaMainId": 10,
"Notes": "",
"PRO_GLDefaultValue": [
{
"$id": "2",
"GLDefaultValueId": 1,
"GLDefaultId": 1,
"Alias": "ACT",
"FunctionTypeId": 0,
"Value": "&&&",
"SegmentId": 16,
"IsRequired": true,
"ModifiedOn": "7/1/2017 4:48:06 PM",
"ModifiedBy": 2,
"PRO_GLDefault": {
"$ref": "1"
},
"LastModified": null
},
{
"$id": "3",
"GLDefaultValueId": 2,
"GLDefaultId": 1,
"Alias": "ST",
"FunctionTypeId": 0,
"Value": "&&",
"SegmentId": 17,
"IsRequired": false,
"ModifiedOn": "7/1/2017 4:48:06 PM",
"ModifiedBy": 2,
"PRO_GLDefault": {
"$ref": "1"
},
"LastModified": null
}
]
}
Please advise me how to overcome this
Thanks,
Raj
Warning is just that, a warning, not an error, and expected if you have curly braces in your post
What is line 85, where the error is generated?
See these two lines:
request_json_base= "{\"GLDefaultId\":0,\"ProjectId\":{ContactProjectId},\"CoaMainId\":10,\"UserId\":{UserId},\"Notes\":\"\",\"PRO_GLDefaultValue\":[{\"GLDefaultValueId\":0,\"GLDefaultId\":0,\"SegmentId\":16,\"Alias\":\"ACT\",\"SegmentType\":\"DETAIL\",\"FunctionTypeId\":0,\"Value\":\"&&&\",\"Mask\":\"&&&\",\"IsRequired\":true,\"LastModified\":null,\"IsCoaRequired\":true},{\"GLDefaultValueId\":0,\"GLDefaultId\":0,\"SegmentId\":17,\"Alias\":\"ST\",\"SegmentType\":\"SET\",\"FunctionTypeId\":0,\"Value\":\"&&\",\"Mask\":\"&&\",\"IsRequired\":false,\"LastModified\":null,\"IsCoaRequired\":false}]}",
request_json = lr_eval_string(request_json_base);
They are not doing what you think they are doing. You are neither allocating space for this data ( see malloc() in combination with a char * variable), nor are you assigning the value to the variable correctly for the C language (see strcpy() )
Another reference here on Stackoverflow
How to correctly assign a new string value?
I'm playing with Google Custom Search API with Custom Search Engine (CSE) via JSON API. I successfully obtained search result, but I'm clueless about how to obtain the nextPageToken.
https://www.googleapis.com/customsearch/v1?key=MY_API_KEY&cx=MY_SEARCH_ENGINE_ID&q=Testing
The JSON response is as follow:
{
"kind": "customsearch#search",
"url": {
"type": "application/json",
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&cref={cref?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json"
},
"queries": {
"nextPage": [
{
"title": "Google Custom Search - Testing",
"totalResults": "2900",
"searchTerms": "Testing",
"count": 10,
"startIndex": 11,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "MY_SEARCH_ENGINE_ID"
}
],
"request": [
{
"title": "Google Custom Search - Testing",
"totalResults": "2900",
"searchTerms": "Testing",
"count": 10,
"startIndex": 1,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "MY_SEARCH_ENGINE_ID"
}
]
},
"context": {
"title": "Test Search Engine"
},
"searchInformation": {
"searchTime": 0.299265,
"formattedSearchTime": "0.30",
"totalResults": "2900",
"formattedTotalResults": "2,900"
},
"items": [
// ... Search Result here
]
}
But how to obtain result of next page via nextPageToken ?
Instead of using nextPageToken, there are 2 parameters I can add into the query string to change the page of results:
start: the start index of the result, valid values are integers > 0.
num: number of results per page, valid values are 1 ~ 10 (max 10 records can be obtained in 1 page)
Therefore to change to page 2, I have to issue:
start=11&num=10
in query string (given that record per page is 10).
Hope this helps.