Elasticsearch search templates - json

I tried using ES's search template to do a conditional clause as specified here. I'm sending my request to the /[my_index]/_search/template endpoint. The request fails because of JSON parsing issues, which makes sense because after adding the conditional clause the payload is no longer a valid JSON. How than am I supposed to use the search templates? Is there a designated endpoint for non-JSON templates?

When using conditional clauses,the template will not be a valid JSON because it will include the section markers {{# like this }}.For this reason, the template should either be stored in a file or, when used via the REST API, should be written as a string.
Method 1 : tempalte stored in a file
Save the query part of the template in config/scripts
ES installation>Config>scripts
test_template.mustache
{
"query":{ whatever query }
}
you can use the saved template by this method through sense
GET /_search/template
{
"template": "test_template",
"params": {
whatever params
}
}
Method 2: template written as a string
convert the template to string form and use via rest api
POST /_search/template/test_template
{
"template": "{\"query\":{ whatever query; remember to escape quotes}}"
}
To search using this template,
GET /_search/template
{
"template": {
"id": "test_template"
},
"params": {
whatever params
}
}
Reference: https://www.elastic.co/guide/en/elasticsearch/reference/1.7/search-template.html#_conditional_clauses

You need to escape the template in wrapping string.
From the same link you referenced:
As written above, this template is not valid JSON because it includes
the section markers like {{#line_no}}. For this reason, the template
should either be stored in a file (see the section called
“Pre-registered templateedit”) or, when used via the REST API, should
be written as a string:
"template":
"{\"query\":{\"filtered\":{\"query\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"

Related

Azure Logic App - Parse JSON with dynamic key/name

just want to know if and how I can parse a HTTP response with a dynamic name in a JSON?
I used the Azure Management API to receive the managed identities (system- and user assigned managed identities) to receive all managed identities.
With a foreach I am iterating the results.
If a resource has a system assigned managed identity and user assigned managed identity, the response looks like this:
{
"principalId": "<principalId1>",
"tenantId": "<tenantId>",
"type": "SystemAssigned, UserAssigned",
"userAssignedIdentities": {
"/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<userAssignedIdentitiesName>": {
"principalId": "<principalId2>",
"clientId": "<clientId>"
}
}
}
Now, I would like to get the <principalId2>.
Unfortunately, the Name of the object is dynamic related to the scope of the resource /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<userAssignedIdentitiesName>.
How can I parse the JSON to receive the needed <principalId2>?
For all other responses I can easily use the Data operations Parse JSON with the payload I inserted from the HTTP response.
Is there a way to use a wildcard? Otherwise, could I somehow just select the first object of userAssignedIdentities to receive the needed value?
Ok, this should work for you. This is the flow I tested with ...
Initialise JSON
Your JSON as a string, how you do that in your solution may differ slightly.
Initialize XPath Result
Defined as an Array and the expression is ...
xpath(xml(json(concat('{ root: ', replace(variables('JSON'), 'PrincipalId', 'principalId'), '}'))), '(//principalId)[2]')
Initialize Result
A bit more work again but defined as a String and the expression is ...
array(xpath(xml(base64ToString(variables('XPath Result')[0]?['$content'])), '//text()'))[0]
The end result should be your value ...

How to parse Json with Array list

I have below response from an API get call.
How do i read the value of dummyId and testcontactId. Below is the sample response:
{
"pageCount":1,
"pageIndex":0,
"pageSize":200,
"totalCount":1,
"dto":{
"data":[
{
"callDuration":"0:00:07",
"dummyId":20,
"testcontactId":"3002",
"id":54
}
],
"columns":[
{
"fieldName":"test1",
"displayName":"test Id",
"show":true,
"sortable":true,
"fieldType":"string",
"order":1
}
]
}
}
I can't place a comment to ask what language you're using, Here is how you'd do it in JavaScript
Parse your API Result to a Object with var data = JSON.parse() JSON.parse()
Then you can just go data.dto.data[0].dummyId
Since dto.data is an array, You can just use the indexes to access each option.
if you need to find specific values, You might want to take a look at Array.find
In the future, Please add a tag with your programming language and things you've tried.

freeradius 3.0.17 rlm_rest parsing json response

I'm trying to authenticate RADIUS Requests against a RESTful API (provided by Customer) using rlm_rest.
The problem I am facing is that
response JSON format (of REST API provided by Customer), is different from rlm_rest default format (indicated in etc/raddb/mods-enabled/rest).
My Virtual Server configuration as below:
Default
authorize {
...
...
rest
if (ok) {
update control {
Auth-Type := rest
}
}
}
mods-enabled/rest
authorize {
uri = "https://3rd-party-API/auth"
method = 'post'
body = 'json'
chunk = 0
tls = ${..tls}
data = '{
"code": 1,
"identifier": %I,
"avps": {
"User-Name": ["%{User-Name}"],
"NAS-IP-Address": ["%{NAS-IP-Address}"],
"Called-Station-Id": ["%{Called-Station-Id}"],
"Calling-Station-Id": ["%{Calling-Station-Id}"],
"NAS-Identifier": ["%{NAS-Identifier}"]
}
}'
}
Result
/sbin/radiusd -Xxx
HTTP response code
200
JSON Body
{
"code": "2",
"identifier": "91",
"avps": {
"Customer-Attributes": "Hello"
...
...
"Acct-Interim-Interval": "300"
}
}
The JSON structure is different from the example, and xlat parse
"code"
"identifier"
"avps"
And, of course, xlat finds no attributes match with the dictionary, while it cannot find "avps" and won't dig deeper.
So I was wondering is there anyway to either
Define the response JSON structure for xlat to parsing
Insert a "is_json" or "do_xlat" flag into the JSON ("avps"), and hope xlat will then dig deeper
Save the JSON and parse with exec/rlm_exec (using JQ or any other bash/JSON tools)
Please advise if there is any workaround. Thanks!
In FreeRADIUS version 4, there's a rlm_json module, which implements a custom node query language based on xpath (jpath), it is extremely limited and only supports some very basic queries (feel free to enhance it via PR :) ).
Below is an example I pulled out of my library of customer configurations. You can see here it's pulling out two keys (externalID and macAddress) from the root level of the JSON doc and assigning them to a couple of custom attributes (Subscriber-ID and Provisioned-MAC).
map json "%{rest_api:https://${modules.rest[rest_api].server}/admin/api/${modules.rest[rest_api].api_key}/external/getDeviceBySerialNumber?certificateSerialNumber=%{lpad:&TLS-Client-Cert-Serial 40 0}}" {
&Subscriber-ID := '$.externalId'
&Provisioned-MAC := '$.macAddress'
}
The xlat expansion can also be modified to send HTTP body data. Just put a space after the URL and pass your custom JSON blob.

JMeter Variables in JSON request

When I pass an variable in JMeter HTTP request, I'm getting an exception
"Unexpected escape character after back slash"
The request body:
"Draft":{
"id": 123654656,
"draftdata":{\\\"accCat\\\":\\\"207\\\",\\\"accNumber\\\":\\\"656565
\\\",\\\"id\\\":${Var_ID},...}
}
When I send the request, one of the two back-slashes are omitted. I guess the variable ${Var_ID} should be passed in a way that does not conflict with the json body
I don't think you need these \\\ signs
I believe you need to surround ${Var_ID} with quotation marks
Something like:
{
"id": 123654656,
"draftdata": {
"accCat": "207",
"accNumber": "656565 ",
"id": "${Var_ID}"
}
}
You can use online JSON validation tools like Online JSON Viewer to test your JSON payload. Also check out Testing SOAP/REST Web Services Using JMeter article for some initial information on testing REST APIs using JMeter
Maybe making changes like:
{
"Draft": {
"id": 123654656,
"draftdata": {
\"accCat\":\"207\",
\"accNumber\":\"656565\",
\"id\":\"${Var_ID}\",...}
}
I don't see any need to have \ signs, just one will escape original " signs.
The variables, or the request to some function of JMeter, into json body must be passed without quotation marks, something like this:
"Draft":{
"id": 123654656,
"draftdata":{
"accCat":"207",
"accNumber":"656565",
"id":${Var_ID},...}
}
Also yout don't need the backslashes signs. Hope this help.

Converting velocity response to JSON

I am using struts 2 and velocity templates to generate JSON response.
Now the catch is the response is not generated using some velocity JSON plugin
it's just a String that comes out once velocity is done with its parsing and rendering of
response, and on client side I do eval to get the response from string to JSON.
What I really need is some solution on velocity's or struts' side where, once the result is
generated by velocity, the framework should call my API where I can convert the response output of vm file into JSON using my own logic. How do achieve this?
For example:
On browser using JavaScript I have designed a tree widget that I use for displaying comments in tree structure.
Say user clicks on comments button.
My UI widget will fire an AJAX to get data for comments.
This request is intercepted by STRUTS 2 framework.
It will call, say, getComments() action API and will populate an arrayList with comment object say cmt.
Now the response is handled by a velocity template(*.vm).
Now in vm I am writing code like this:
{ "CommentsData" : [
#set($sep="")
#foreach($c in $cmt)
$sep
{
"commentText" : $c.getText()
}
#set($sep=",")
#end
}
Now the final response may turn out like this:
{ "CommentsData" : [
{
"commentText" : "This is comment 1"
},
{
"commentText" : "This is comment 2"
},
{
"commentText" : "This is comment 3"
},
{
"commentText" : "This is comment 4"
}`
]
}
Now this may look like JSON, but its not strict JSON; I mean if I miss
some , somewhere then on client side in JavaScript my eval might fail or JSON.parse()
will fail, but on velocity template I have now clue if JSON is malformed.
So once the above velocity template is generated I need some control, where I can write some Java code to do some validations on the response.
I see that my approach to use velocity template to generate JSON output (actully a String that looks like JSON) may be wrong. But still I need to handle the response of every velocity template I have written.
Not sure how you are using velocity. We don't use velocity when outputting JSON; we just create a JSON convertible object and output it directly from controllers using response.write(jsonObject.toJson()). This way, proper JSON is always generated.