I wondering how you could add an additional key value pair to an already existing tcl dictionay. For example:.
dict set greetings english Hi
$greetings add key_pair German Hallo
You have just described exactly what dict set does. Several other dict subcommands also do that.
However, note that you are logically reading the dictionary out of the variable, creating a new dictionary from it with the new key in it, and writing the new dictionary back to the variable. (The implementation is more efficient than that, but the model is definitely read/create-new/write.) If another variable also contains the pre-modification dictionary, it will not see the change. This includes if you pass it as an argument to a command. Tcl values are immutable. (Or rather they behave like it when their references are shared.) Only named entities like variables are mutable.
Related
I am using Azure Data Factory. I'm trying to use a String variable to lookup a Key in a JSON array and retrieve its Value. I can't seem to figure out how to do this in ADF.
Details:
I have defined a Pipeline Parameter named "obj", type "Object" and content:
{"values":{"key1":"value1","key2":"value2"}}
Parameter definition
I need to use this pipeline to find a value named "key1" and return it as "value1"; "key2" and return it as "value2"... and so on. I'm planning to use my "obj" as a dictionary, to accomplish this.
Technically speaking, If i want to find the value for key2, I can use the code below, and it will be returned "value2":
#pipeline().parameters.obj.values.key2
What i can't figure out is how to do it using a variable (instead of hardcoded "key2").
To clear things out: I have a for-loop and, inside it, i have just a copy activity: for-each contents
The purpose of the copy activity is to copy the file named item().name, but save it in ADLS as whatever item().name translates to, according to "obj"
This is how the for-loop could be built, using Python: python-for-loop
In ADF, I tried a lot of things (using concat, replace...), but none worked. The simpliest woult be this:
#pipeline().parameters.obj.values.item().name
but it throws the following error:
{"code":"BadRequest","message":"ErrorCode=InvalidTemplate, ErrorMessage=Unable to parse expression 'pipeline().parameters.obj.values.item().name'","target":"pipeline/name_of_the_pipeline/runid/run_id","details":null,"error":null}
So, can you please give any ideas how to define my expression?
I feel this must be really obvious, but I'm not getting there.....
Thanks.
Hello fellow Pythonista!
The solution in ADF is actually to reference just as you would in Python by enclosing the 'variable' in square brackets.
I created a pipeline with a parameter obj like yours
and, as a demo, the pipeline has a single Set Variable activity that got the value for key2 into a variable.
This is documented but you need X-ray vision to spot it here.
Based on your comments, this is the output of a Filter activity. The Filter activity's output is an object that contains an array named value, so you need to iterate over the "output.value":
Inside the ForEach you reference the name of the item using "item().name":
EDIT BASED ON MORE INFORMATION:
The task is to now take the #item().name value and use it as a dynamic property name against a JSON array. This is a bit of a challenge given the limited nature of the Pipeline Expression Language (PEL). Array elements in PEL can only be referenced by their index value, so to do this kind of complex lookup you will need to loop over the array and do some string parsing. Since you are already inside a FOR loop, and nested FOR loops are not supported, you will need to execute another pipeline to handle this process AND the Copy activity. Warning: this gets ugly, but works.
Child Pipeline
Define a pipeline with two parameters, one for the values array and one for the item().name:
When you execute the child pipeline, pass #pipeline.parameters.obj.values as "valuesArray" and #item().name as "keyValue".
You will need several string parsing operations, so create some string variables in the Pipeline:
In the Child Pipeline, add a ForEach activity. Check the Sequential box and set the Items to the valuesArray parameter:
Inside the ForEach, start by cleaning up the current item and storing it as a variable to make it a little easier to consume.
Parse the object key out of the variable [this is where it starts to get a little ugly]:
Add an IF condition to test the value of the current key to the keyValue parameter:
Add an activity to the TRUE condition that parses the value into a variable [gets really ugly here]:
Meanwhile, back at the Pipeline
At this point, after the ForEach, you will have a variable (IterationValue) that contains the correct value from your original array:
Now that you have this value, you can use that variable as a DataSet parameter in the Copy activity.
I'm an intermediate-to-advanced user of jq 1.5 and 1.6. I'm iterating over an array of objects (let's say, keyed by the names of the first ten digits). I have an array of strings stored in variable $odd_fields (["one","three",...,"nine"]). Without using a convoluted iterative process (since I can do this statically with only map({one,three,...,nine}), I'd like to be able to re-cast my array of objects into an array of objects with reference to my array of keys, and in that order, like map({$odd_fields[]}).
After hundreds of attempts, I can't seem to do it in one throw without a static list of members. I have been able to approximate this by creating an array of UNwanted keys and mapping del($unwanted_keys[]), but I don't care what keys I'm throwing away (and I have no confidence that the 50th object doesn't have one extra key that I don't want), only what keys I want to keep (or create, if missing), so that every object in the array has the same arbitrary keys (my keys) in the same order (suitable for #csv).
This is a general need not directly related to a specific JSON glob or specific set of keys, hence no sample JSON.
Most attempts that don't generate syntax or reference errors give me an array of objects [{"odd_fields":["one","three",...,"nine"]},...], or, some iterative processes give me just the values of the keys I want. I want whole consistent subset objects as if any key not in my list was never present.
Additionally, any wizard out there who can tell me -- while iterating through an object, is there any way to determine what is/was the key to "."? Similar to: {"one":1,"two":2,"three":3}|map(.|my_key) (where function my_key reveals what .'s key is). I realize there's an easier way to do the above example, but I would like to be able to determine .'s key in other, less direct, code. I don't need to use it to rip all keys from an object, just one key from an arbitrary ".".
To recast an object using the keys and key ordering in an array, $array:
def recast($array):
. as $in
| reduce $array[] as $k ({}; .[$k] = $in[$k]);
You could use this with your array as input as follows:
map(recast($odd_fields))
You may wish to tweak this with respect to missing and/or extra keys.
As a one-liner
A compact, reduce-free version:
map([$array[] as $p|{($p):.[$p]}]|add)
Suppose I have a simple map, example-map:
(def example-map {"s" {"f" "g"}
"m" {"r" "q"}})
I can use clojure.data.json/write-str to JSON-ify this map as such:
(clojure.data.json/write-str example-map) =>
"{\"s\":{\"f\":\"g\"},\"m\":{\"r\":\"q\"}}"
I'd like to conditionally place some of the values into lists according to the value of their keys.
write-str provides an optional :key-fn, which applies some function to key value pairs. For example, the desired function might specify that all values associated with entries that match "s" are placed in lists.
(clojure.data.json/write-str example-map :key-function desired-function) =>
"{\"s\":[{\"f\":\"g\"}],\"m\":{\"r\":\"q\"}}"
Does anyone know how to specify such a key function that checks for membership of a key in a set and places the values associated with members into an array rendered in the output JSON?
Like your previous question, this is not a job for the JSON parser. You don't need to rely on write-time features of your JSON library to adjust the shape of your JSON maps. Instead, you have a fully functional Turing complete language at your disposal: Clojure! If the maps don't already look the way you want them to be output, then write a function that takes one Clojure map as input and produces a different one as output; then ask your JSON library to write the output map, without any special rules for fiddling with the output.
Now, as it happens this particular JSON library does provide an option named value-fn (not key-function as you claim) to let you modify a value in a map based on its key. So you could use that, in which case you simply need to write a function with a signature like:
(fn [k v]
(...compute new value...))
There are many ways you could write such a function, but they are all entirely divorced from your JSON parser. If you need help writing it, mention some specific things you need help with, so you can get a clear explanation for the part of the process that is actually giving you trouble.
I got an anonymous array which I want to deserialize, here the example of the first array object
[
{ "time":"08:55:54",
"date":"2016-05-27",
"timestamp":1464332154807,
"level":3,
"message":"registerResourcePath ('', '/sap/bc/ui5_ui5/ui2/ushell/resources/')",
"details":"","component":"sap.ui.ModuleSystem"},
{"time":"08:55:54","date":"2016-05-27","timestamp":1464332154808,"level":3,"message":"URL prefixes set to:","details":"","component":"sap.ui.ModuleSystem"},
{"time":"08:55:54","date":"2016-05-27","timestamp":1464332154808,"level":3,"message":" (default) : /sap/bc/ui5_ui5/ui2/ushell/resources/","details":"","component":"sap.ui.ModuleSystem"}
]
I tried deserializing using CL_TREX_JSON_SERIALIZER, but it is corrupt and does not work with my JSON, here is why
Then I tried /UI2/CL_JSON, but it needs a "structure" that perfectly fits the object given by the JSON Object. "Structure" means in my case an internal table of objects with the attributes time, date, timestamp, level, messageanddetails. And there was the problem: it does not properly handle references and uses class description to describe the field assigned to the field-symbol. Since I can not have a list of objects but only a list of references to objects that solution also doesn't works.
As a third attempt I tried with the CALL TRANSFORMATION as described by Horst Keller, but with this method I was not able to read in an anonymous array, and here is why
My major points:
I do not want to change the JSON, since that is what I get from sap.ui.log
I prefere to use built-in functionality and not a thirdparty framework
Your problem comes out not from the anonymity of array, but from the awkwardness of SAP JSON (De)serializer, which doesn't respect double quotes, which enclose JSON attributes. The issue is thoroughly described in this answer.
If you don't want to change your JSON on-the-fly, the only way you have is to change CL_TREX_JSON_DESERIALIZER class like this.
/UI5/CL_JSON_PARSER parses JSONs with unknown format.
Note that it's got "for internal use" written on it so many times that you probably should take it seriously and clone its code to fixate it.
When I'm working with Hash Tables/Dictionaries I sometimes struggle with how to specify keys.
For example: if I create a simple Dictionary (using Python for this example),
foo = {'bar': 'baz', 'foobar': 'foobaz' }
I can access values (in other modules) with the key values: (foo['bar']) and get baz back.
In the words of Dr. Evil, "pretty standard, really."
Unfortunately, using static strings for keys tightly couples any modules using this Dictionary to its implementation. Of course, this can also apply when using other key types (e.g. Enums, Objects, etc.); anyway you slice it, all modules which access the Dictionary need to know the values for the keys.
To resolve this, I typically use static constant string values (or Enums if available in the language) for keys, and either store them publicly in the local class/module, or in a separate module/class. Therefore any changes to the dictionary keys themselves are kept in a single location.
This usually looks like this:
BAR_KEY = 'bar'
foo[BAR_KEY] = 'foobar'
Are there better ways of specifying keys such that the use of the Dictionary doesn't necessarily couple a module/class to its implementation?
Note: I've seen a few responses in SO which address this (e.g. property-to-reference-a-key-value-pair-in-a-dictionary), but the topics didn't seem to address this issue specifically. The answers were helpful, but I'd like a wider range of experience.
Why not make a class for this, which only contains properties? This is done nicely with Python (from what I know), and works well with other languages, too. Refactoring the names is trivial with today's tools, too.
In cases where I'm passing the object around and I've got known keys, I'd always prefer adding an attribute to an object. IMO, the use case of dictionaries is when you don't know what the keys are.
Python is trivial:
foo.bar=baz
Java is pretty much the same:
class Foo { public String bar="baz"; }
The Python performance would be pretty much identical, since a property lookup is just a dictionary lookup and the Java performance would be better.
I sometimes create a separate class to hold the dictionary keys. That gives them their own namespace, as well as the regular benefits of having the keys be in const strings, namely that you don't have the risk of typos, you get code completion, and the string value is easy to change. If you don't want to create a separate class, you get all of the benefits except a namespace just from have const strings.
That said, I think you're getting close to soft coding territory. If the keys in the dictionary change, it's OK for code using the dictionary to change.
Personally, I use your method. It's pretty sensible, simple, and solves the actual problem.
I usually do the same thing; if the key is always going to be the same, make a 'constant static' in whatever language to hold the key.