How to set pipeline variable getting databrick return value?
Databricks/Notebook exited json:
"runOutput":
[
"{\"last_up\":\"2022-04-04 15:14:20\"}"
]
What I tried to get the date value
#activity('notename').output.runOutput.last_up
What should I try?
From your post, the json appears to contain an array as the result of runOutput.
Can you try the following?
#activity('notename').output.runOutput[0].last_up
Related
My aim is to use Azure Data Factory to copy data from one place to another using REST API.
The first part of the copying is using the ForEach activity to select parameters from a nested JSON/array. I have had problems with this nested JSON because of various error messages in the ForEach activity.
My JSON is of the following form:
(
{
"key_1": "value_1",
"key_2": [
"value_2_1",
"value_2_2"
]
}
)
and first I'm setting the #json conversion function in front of it:
#json(
'{
"key_1": "value_1",
"key_2": [
"value_2_1",
"value_2_2"
]
}'
)
Here you can see the Execute Pipeline object and its parameters:
I am setting this JSON as a parameter in the Execute Pipeline object. I am setting its type as an "Array". (For the record, I have still had similar error messages even though I have tried to change the parameter type to "String" or "Object".)
The ForEach activity is used to select an item from the nested JSON, which is written into a parameter.
This immediately produces the following error in the Execute Pipeline activity:
Operation on target... ...failed: The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'.
So, even though I set the JSON parameter type to "Array", it is changed into "Object" when I debug the pipeline activity. See the error below:
Next, I tried to use the ADF #createArray function before my JSON text.
#createArray(
'{
"key_1": "value_1",
"key_2": [
"value_2_1",
"value_2_2"
]
}'
)
When debugging, the forEach activity throws me an error in the first Copy data activity:
The expression 'concat(item().SELECTING_key_1_FROM_MY_JSON))' cannot be evaluated because property 'key_1' cannot be selected. Property selection is not supported on values of type 'String'.
Please help me, what I am doing wrong in my attempts of converting the JSON to an array? What should I change in my code?
So far I have tried changing the parameter type and using various functions in the JSON dynamic content but with no luck.
update:
If you want change the filename in Copy Data --> Sink.
You can key in the dynamic content #concat(pipeline().parameters.Pip_Object.key_1,'.json') to rename the file.
Please correct me if I understand you wrong.
First, we should use Parameters to store the Json array as follows. Because Variables are not support to store the Json array. Variables are only support to store simple data type such as ["1","2","3"...].
Json array format should as follows:
[{"key_1": "value_1"},{"key_2": ["value_2_1","value_2_2"]}]
I created a simple test here. Then ForEach the Json array:
Inside ForEach1 activity, pass #item() to the object type parameter Pip_Object.
The Input is as follows:
Add dynamic content above using any combination of expressions, functions and system variables at for each settings
#activity('Get Metadata1').output.childItems
We ended up using two pipelines: a generic one and a specific one.
The generic one has the JSON as a parameter (JSON_PARAMETER), and we set it a default value in the following form
[{"key_1":"value_1","key_2":["value_2"]},{"key_1":"value_3","key_2":["value_2"]}, ...etc. ...}]
The generic pipeline has a forEach loop, in which that mentioned JSON parameter is called in Settings -> Items:
#JSON(pipeline().parameters.JSON_PARAMETER)
In the specific pipeline, there is an Execute Pipeline activity, where the JSON_PARAMETER is found in Settings -> Parameters. If the default value of JSON_PARAMETER is used, the field is left blank. If we want to change the parameter, before Execute Pipeline, we put a Set Variable activity where we change the Variables -> Value to:
#concat('
[
{"key_1":"value_1",
"key_2":
["value_2",
"value_3"
]
},
{"key_1":"value_3",
...and so on...
}
]
')
I am running a query in ASP.NET to get some data from collection in Mongo DB.
The query is running fine but the datatype that it is returning is different that i expect.
variable= Newtonsoft.Json.JsonConvert.DeserializeObject<class>(var);
Data that it returns
I am using Newtonsoft to parse the Json:
variable is an object of class
and var is the Json of the form > "{ \"Name\" : \"XYZ\", \"Value\" : [\"Value\"] }"
Class contains propeties Name and Value.
Exception it returns :
{"Unexpected character encountered while parsing value: [. Path 'Value', line 1, position 43."}
Any Ideas how i should correct this.
so i have this data which is outputted from stringify(doc)
{
"id": "8467fdae-c38c-4b6e-9492-807d7c9eb97e",
"message": "send to nodejs"
}
but im not sure how can i go ahead in getting a single value from it, e.g message. Any help would be appreciated as ive tried different methods and they seem to not be working
After you use stringify method the value you have is string.
So You can not get single property from it, except you convert it become object again and get property it. Like this:
object = {
"id": "8467fdae-c38c-4b6e-9492-807d7c9eb97e",
"message": "send to nodejs"
};
// JSON string is value which you get after use Stringify method
jsonString = JSON.stringify(object);
// Convert jsonString to object again and get message property
message = JSON.parse(jsonString).message
stringify() will convert json to string.
Before running stringify(), doc is already json and you can get message by doc.message.
after running stringify(), the result will be string, if you want to get json back, you can use JSON.parse()
I have a JSON object returned in my console, and I want to display those data named "offers".
the JSON object is returned like that :
I tried to display my JSON Object data with :
console.log(JSON.stringify(data));
The thing is, it says that "data is not defined"
Does anyone know what happens ? :)
You should add full path to element of json, for example if your json looks like:
var json = {"par":22, "par2":555, "elems":[{"attr1":53, "attr2":99}] };
and if you want to get attr1 value, you should do something like this:
console.log(json.elems[0].attr1); // 53
so in your case that could be something like:
variableName.result.data.offers //variableName is variable that your "consoling"
Method JSON.stringify doesn't get yout specified value from JSON structure, it's converts JSON object to string.
console.dir provides a good representation of object than console.log().U can try with both
console.log(result.data.offers[0]);
console.dir(result.data.offers[0]);
I am trying to parse a single value from a REST web service that I am testing.
I understand how to make the call and I see the JSON response in the Output window.
let request = Alamofire.request(.GET, "http://IP:PORT/jsonTest", parameters: ["s": "Ping?"])
.responseJSON{(_,_,data,_) in
var json = JSON(data!)
println(json)
The Console Output shows me:
{"NewDataSet":[
{"Table1":[
{"Column-A":"FirstA",
"Column-B":"FirstB"
},
{"Column-A":"SecondA",
"Column-B":"SecondB"
},
{"Column-A":"ThirdA",
"Column-B":"ThirdB"
}
]}
]}
What I would like to do now, is to display only the first value from Column-A - which in this example would be "FirstA".
I've been trying to use a code like this, but so far I am not getting anywhere...
println(json[0][0]["Column-A"].stringValue)
Any pointers much appreciated!
json["NewDataSet"][0]["Table1"][0]["Column-A"].stringValue
This is what you want. This is because your json starts with a dictionary and is formatted as dictionary>array>dictionary>array>dictionary. Note that json dictionaries are noted by { : , : } while arrays are noted as [ , ].