Liquid Map Arithmetic Operations - json

Hello wise people of StackOverflow,
I have an exchange rates file that I am mapping from XML to JSON in an Azure logic app using Transform XML To JSON. I have two rates (Ask and Bid) within the XML. A node in the XML file looks like so:
<quote>
<ask>110.0668027</ask>
<bid>108.1461645</bid>
<currency>AFA</currency>
<date>2022-07-27T23:59:59+0000</date>
<midpoint>109.1064836</midpoint>
</quote>
I then need to map to JSON but I wish for the JSON output to only show a single rate which must be calculated as (Ask+Bid)/2 to give me the average.
I have created a liquid map like so:
{
"EffectiveDate": "2022-06-06",
"BaseCurrency": "GBP",
"ExchangeRates":
[
{% for data in content.response.quotes %}
{
"ExchangeRate": "({{data.quote.ask}} + {{data.quote.bid}})/2",
"TargetCurrency": "{{data.quote.currency}}"
},
{%endfor%}
]
}
But when I run the logic app the JSON output looks like so:
{
"ExchangeRate": "(110.0668027 + 108.1461645)/2",
"TargetCurrency": "AFA"
},
Which is obviously not what I want to see! What is wrong with the syntax in my Liquid map?

In Liquid you have to use math filters to manipulate integers.
So I would do something like that:
First calculate your value to divide.
{% assign value_2 = value_2 | divided_by:2 %}
Then output your values by adding them:
{{ value_1 | plus: value_2}}
More info about math filters here: Documentation about math filters

Related

Format for storing json in Amazon DynamoDB

I've got JSON file that looks like this
{
"alliance":{
"name_part_1":[
"Ab",
"Aen",
"Zancl"
],
"name_part_2":[
"aca",
"acia",
"ythrae",
"ytos"
],
"name_part_3":[
"Alliance",
"Bond"
]
}
}
I want to store it in dynamoDB.
The thing is that I want a generator that would take random elements from fields like name_part_1, name_part_2 and others (number of name_parts_x is unlimited and overalls number of items in each parts might be several hundreds) and join them to create a complete word. Like
name_part_1[1] + name_part_2[10] + name_part[3]
My question is that what format I should use to do that effectively? Or NoSQL shouldn't be used for that? Should I refactor JSON for something like
{
"name": "alliance",
"parts": [ "name_part_1", "name_part_2", "name_part_3" ],
"values": [
{ "name_part_1" : [ "Ab ... ] }, { "name_part_2": [ "aca" ... ] }
]
}
This is a perfect use case for DynamoDB.
You can structure like this,
NameParts (Table)
namepart (partition key)
range (hash key)
namepart: name_part_1 range: 0 value: "Ab"
This way each name_part will have its own range and scalable. You can extend it to thousands or even millions.
You can do a batch getitem from the sdk of your choice and join those values.
REST API reference:
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html
Hope it helps.
You can just put the whole document as it is in DynamoDB and then use document path to access the elements you want.
Document Paths
In an expression, you use a document path to tell DynamoDB where to
find an attribute. For a top-level attribute, the document path is
simply the attribute name. For a nested attribute, you construct the
document path using dereference operators.
The following are some examples of document paths. (Refer to the item
shown in Specifying Item Attributes.)
A top-level scalar attribute: ProductDescription A top-level list
attribute. (This will return the entire list, not just some of the
elements.) RelatedItems The third element from the RelatedItems list.
(Remember that list elements are zero-based.) RelatedItems[2] The
front-view picture of the product. Pictures.FrontView All of the
five-star reviews. ProductReviews.FiveStar The first of the five-star
reviews. ProductReviews.FiveStar[0] Note The maximum depth for a
document path is 32. Therefore, the number of dereferences operators
in a path cannot exceed this limit.
Note that each document requires a unique Partition Key.

ansible json-query path to select item by content

Does anyone know what json-query filter can be used to select Tigger's food in the sample JSON below? The JSON is a simplified stand-in for a massive and relatively complicated AWS blob.
Some background: I was rather pleased to discover that Ansible has a json-query filter. Given that I was trying to select an element from an AWS JSON blob this looked as if it was just what I needed. However I quickly ran into trouble because the AWS objects have tags and I needed to select items by tag.
I tried selector paths equivalent to Foods[Tags[(Key='For') & (Value='Tigger')]] and similar but didn't manage to get it to work. Using a standalone json-query library such as https://www.npmjs.com/package/json-query I can use the parent attribute but that does not appear to be in Ansible, quite apart from being a deviation from the core idea of json-query.
It might be better to sidestep the problem and use a jsonpath selector. jsonpath is similar to json-query and is a translation from xpath.
{ "Foods" :
[ { "Id": 456
, "Tags":
[ {"Key":"For", "Value":"Heffalump"}
, {"Key":"Purpose", "Value":"Food"}
]
}
, { "Id": 678
, "Tags":
[ {"Key":"For", "Value":"Tigger"}
, {"Key":"Purpose", "Value":"Food"}
]
}
, { "Id": 911
, "Tags":
[ {"Key":"For", "Value":"Roo"}
, {"Key":"Purpose", "Value":"Food"}
]
}
]
}
References
json_query in ansible:
http://docs.ansible.com/ansible/playbooks_filters.html#json-query-filter
json-query standalone node: https://www.npmjs.com/package/json-query
jmespath, the library ansible uses: http://jmespath.org/
json-query standalone python: https://pypi.python.org/pypi/jsonquery/ (red herring)
Do you need list of ids? If so, try:
- debug: msg="{{ lookup('file','test.json') | from_json | json_query(query) }}"
vars:
query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
First construct simple objects with necessary fields and then pipe it to a filter.

Creating list of ip's in ansible using given range within jinja template

i want to generate a list of ip addresses (using range of last 8bits so 120-190 translates to x.x.x.120 - x.x.x.190) in defaults/main.yml in my role, and later use it to create new network interfaces and generate a new config file. I tried this approach:
defaults/main.yml:
ip_list: "{%for address_end in range(50,99)%} 192.168.0.{{address_end}}{%endfor%}"
conf_list: "{%for ip in ip_list%}server {{ip}}:6666 {%endfor%}"
and then to use it in template
template.conf.j2:
{% for conf_line in conf_list %}
{{conf_line}}
{% endfor %}
and all i got in generated config file was:
s
e
r
v
e
r
:
6
6
6
6
s
e
r
v
e
r
1
:
6
so my guess is that i'm not generating a list but just a long string and when I use for loop in template.conf.j2 I iterate over single chars. I tried using answer to this problem but all i got was syntax error. Any ideas for what might help me ?
You should format your vars as JSON lists if you want them to be lists.
1.1.1.1 2.2.2.2 3.3.3.3 is a string.
['1.1.1.1', '2.2.2.2', '3.3.3.3'] will be converted to list.
But there is alternative approach for you:
ip_list: "{{ lookup('sequence', 'start=50 count=12 format=192.168.0.%d', wantlist=True) }}"
conf_list: "{{ ip_list | map('regex_replace','(.*)','server \\1:6666') | list }}"
Kostantin answer was of much help, but i found just realized that generating config entries in my case could be solved in an less complex way. Instead of trying to iterate over list or a string variable variable in jinja template file template.conf.j2 like did with :
{% for conf_line in conf_list %}
{{conf_line}}
{% endfor %}
you could just enter insert new line signs while generating string in defaults/main.yml:
conf_list: "{%for ip in ip_list%}server {{ip}}:6666\n{%endfor%}"
and then just insert that whole string into a template.conf.j2 like this:
{{conf_line}}
nevertheless i have no other idea how to generate list of ip addresses other than the one Konstantin proposed.

Get the value of a JSON array with _attribute

I have a strange looking JSON file (I think?) generated from elasticsearch.
I was wondering if anyone know how I could retrieve the data from a JSON object looking like this:
u'hits : {
u'hits : [{
u'_score' : 2.1224,
u'_source' : {u'content': u'SomethingSomething' }
}],
u'total: 8 }
u'took: 2 }
I can retrieve the total by writing {{ results.hits.hits.total }}, however, the underscore symbol (_) in front of the attribute name "_score" makes it impossible to retrieve the value of that attribute.
Any suggestions?
Try:
{{ results.hits.hits[0]._score }}
{{ results.hits.hits[0]._source }}

CSV Parser through angularJS

I am building a CSV file parser through node and Angular . so basically a user upload a csv file , on my server side which is node the csv file is traversed and parsed using node-csv
. This works fine and it returns me an array of object based on csv file given as input , Now on angular end I need to display two table one is csv file data itself and another is cross tabulation analysis. I am facing problem while rendering data, so for a table like
I am getting parse responce as
For cross tabulation we need data in a tabular form as
I have a object array which I need to manipulate in best possible way so as to make easily render on html page . I am not getting a way how to do calculation on data I get so as to store cross tabulation result .Any idea on how should I approach .
data json is :
[{"Sample #":"1","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"2","Gender":"Male","Handedness;":"Left-handed;"},{"Sample #":"3","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"4","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":"5","Gender":"Male","Handedness;":"Left-handed;"},{"Sample #":"6","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":"7","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"8","Gender":"Female","Handedness;":"Left-handed;"},{"Sample #":"9","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":";"}
There are many ways you can do this and since you have not been very specific on the usage, I will go with the simplest one.
Assuming you have an object structure such as this:
[
{gender: 'female', handdness: 'lefthanded', id: 1},
{gender: 'male', handdness: 'lefthanded', id: 2},
{gender: 'female', handdness: 'righthanded', id: 3},
{gender: 'female', handdness: 'lefthanded', id: 4},
{gender: 'female', handdness: 'righthanded', id: 5}
]
and in your controller you have exposed this with something like:
$scope.members = [the above array of objects];
and you want to display the total of female members of this object, you could filter this in your html
{{(members | filter:{gender:'female'}).length}}
Now, if you are going to make this a table it will obviously make some ugly and unreadable html so especially if you are going to repeat using this, it would be a good case for making a directive and repeat it anywhere, with the prerequisite of providing a scope object named tabData (or whatever you wish) in your parent scope
.directive('tabbed', function () {
return {
restrict: 'E',
template: '<table><tr><td>{{(tabData | filter:{gender:"female"}).length}}</td></tr><td>{{(tabData | filter:{handedness:"lefthanded"}).length}}</td></table>'
}
});
You would use this in your html like so:
<tabbed></tabbed>
And there are ofcourse many ways to improve this as you wish.
This is more of a general data structure/JS question than Angular related.
Functional helpers from Lo-dash come in very handy here:
_(data) // Create a chainable object from the data to execute functions with
.groupBy('Gender') // Group the data by its `Gender` attribute
// map these groups, using `mapValues` so the named `Gender` keys persist
.mapValues(function(gender) {
// Create named count objects for all handednesses
var counts = _.countBy(gender, 'Handedness');
// Calculate the total of all handednesses by summing
// all the values of this named object
counts.Total = _(counts)
.values()
.reduce(function(sum, num) { return sum + num });
// Return this named count object -- this is what each gender will map to
return counts;
}).value(); // get the value of the chain
No need to worry about for-loops or anything of the sort, and this code also works without any changes for more than two genders (even for more than two handednesses - think of the aliens and the ambidextrous). If you aren't sure exactly what's happening, it should be easy enough to pick apart the single steps and their result values of this code example.
Calculating the total row for all genders will work in a similar manner.