On our Stack users by default have discoverer permissions on resources. I was surprised that this also gives users the ability to query the last_transaction_rid of the dataset (using catalog/datasets/<rid>/reverse-transactions2/<branch>), so this method to check if a user has access is not working.
What would be the recommended and most performant API call to check if I, with my current Foundry token, can read the actual content of a dataset? Note: I don't want to query the content, but just understand if I would have the permissions to do so.
Would Data Lineage App (Monocle) work for you?
Open it workspace/data-integration/monocle/ > Find the dataset(s) > top right dropdown ("Resource Type" -> "Permissions")
The API that powers it can be extracted by checking the Dev Tools > Network Tab if you want / need to automate it (I've done this in the past and worked well for my use case)
I have resorted back to calling the compass/resources API and checking if response['operations'] contains compass:view:
def get_dataset_details(api_base: str, dataset_path_or_rid: str, headers: dict) -> dict:
"""
Returns the resource information of a dataset
Args:
dataset_path_or_rid: The full path or rid to the dataset
Returns: (dict) the json response of the api
"""
if 'ri.foundry.main.dataset' in dataset_path_or_rid:
response = requests.get(f"{api_base}/compass/api/resources/{dataset_path_or_rid}",
headers=headers,
params={'decoration': 'path'})
else:
response = requests.get(f"{api_base}/compass/api/resources",
headers=headers,
params={
'path': dataset_path_or_rid,
'decoration': 'path'
})
response.raise_for_status()
if response.status_code != 200:
raise ValueError(f"Dataset {dataset_path_or_rid} not found; "
f"If you are sure your dataset_path is correct, "
f"check if your jwt token "
f"is still valid!")
return response.json()
details = get_dataset_details(...)
if 'compass:view' not in dataset_details['operations']:
raise ValueError("No compass:view access to dataset")
I am trying to POST API request where I am getting API response as
"d": {
"__metadata": {
"uri": "http://ev-qa02.zs.local/IncentiveManager/0002i1/wcf/v5.svc/InDataRequestCreators('9f31c6da-ec56-4360-8589-d21b6320f99b')",
"type": "ZSAssociates.Javelin.ETL.Rest.v5.InDataRequestCreator"
},
"ScenarioId": "9f31c6da-ec56-4360-8589-d21b6320f99b",
"CallbackUrl": "",
"DataExpiresOnUtc": "/Date(4103913600000)/",
"CreateScenarioIfMissing": false,
"AdapterId": "0fcbd8d2-f5cb-4e2a-bda8-bb37037b022d",
"InDataRequestIdOut": "eb36f8a9-5b7d-4835-88f6-4af67830c1e9",
"InDataRequestUrlOut": "/InDataRequests('eb36f8a9-5b7d-4835-88f6-4af67830c1e9')"
}
}
Now I am trying to hit another API request where my URL would be kind of
http://ev-qa02.zs.local/IncentiveManager/0002i1/WCF/V5.svc/InDataRequests('eb36f8a9-5b7d-4835-88f6-4af67830c1e9')/FileCreator
*InDataRequests('eb36f8a9-5b7d-4835-88f6-4af67830c1e9') This random number is generated from above response value "InDataRequestIdOut"
How can I append the URL taking previous API response and adding in my 2nd POST request.
I am not able to capture my response and used it in other API POST request? i would realy appreciate if you can help me here,been stuck in this issue since couple of days,I went through doc and examples too but couldn't resolve this.I have attached screenshot too.PostUrlFailureScreenshot
My main problem is line number 26 and 27 from eclipe screenshot
Scenario: Verify that JIM Idr request ofr Post
Given header Content-Type = 'Application/JSON'
And header Accept = 'Application/JSON'
And header Authorization = 'Basic
UUEwMl9JbmNlbnRpdmVNYW5hZ2VyXzAwMDJpMTpZWkxaRjlGclR1eWhlcVNJbXlkTlBR'
Given path 'InDataRequestCreators'
* def user =
"""
{
"ScenarioId":"9f31c6da-ec56-4360-8589-d21b6320f99b",
"AdapterId":"0fcbd8d2-f5cb-4e2a-bda8-bb37037b022d",
"DataExpiresOnUtc":"2100-01-18T00:00:00",
"CreateScenarioIfMissing":"false"
}
"""
And request user
When method post
Then status 201
* print 'the value of response is:', response
And def app = response
And path 'app.InDataRequestIdOut' + '/FileCreators'
* def body =
"""
{
"InDataRequestId": "1d6326a2-d25f-41d2-9303-8a6e6101efcc",
"ProcedureName": "",
"SourceWorkspacePath": ""
}
"""
And request body
When method post
Then status 201
First, it looks to me you are using the wrong Eclipse plugin for Cucumber, please refer to this issue and make sure: https://github.com/intuit/karate/issues/90
There are so many things you are doing wrong. For example it should be application/json (lowercase). There are many places you are mixing upper case and lower case in your above description, please take care.
And there is no way to understand how the URL is being set up, without this - I can't provide proper help.
You have a fundamental misunderstanding of how to use Karate expressions, for example this is just concatenating two strings:
And path 'app.InDataRequestIdOut' + '/FileCreators'
This may give you some hints, instead of the above:
When url baseUrl
And path "InDataRequests('" + response.InDataRequestIdOut + "')/FileCreator"
And is it FileCreator or FileCreators. You seem to be quite careless :(
In my controller file I have a method that reads the incoming HTTP request, reads the user data from the Database, encodes the result in JSON (using jsx) and sends it in response.
sensorusersdetails('GET', []) ->
Headers = [{'Access-Control-Allow-Origin', "*"},
{'Access-Control-Allow-Methods', "GET, OPTIONS"},
{'Content-Type', "application/json"},
{'Access-Control-Allow-Headers', "X-Requested-With"},
{'Access-Control-Max-Age', "180"}],
Building = Req:query_param("bld"),
io:format("User Data request from Node.js server~n~p~n",
[Req:query_params()]),
{{Year,Month,Day},{_,_,_}} = erlang:localtime(),
StrDate = lists:flatten(io_lib:format("~4..0w-~2..0w-~2..0w",
[Year,Month,Day])),
BUserDataList = boss_db:find(sensoruser_data, [{building, 'equals', Building}]),
io:format("Current Users Data stored in the database: ~n~p~n",[BUserDataList]),
MyUserJSONList = sensor_preapre_data(BUserDataList, StrDate),
io:format("The Present Date Sensor Users Data with Binary 1: ~n~p~n",[MyUserJSONList]),
MyUserJSONListLength = length(MyUserJSONList),
if MyUserJSONListLength > 0 ->
MyFinalList = sensor_data_final(MyUserJSONList),
io:format("The Present Date Sensor Users Data without Binary 2: ~n~p~n",[MyFinalList]),
{200, [MyFinalList], Headers};
%%{json, MyFinalList};
true ->
{200, "NO DATA FOUND", Headers}
%%{json, [{error, "NO DATA FOUND"}]}
end.
In the Chicagoboss Server logs I'm getting:
The Present Date Sensor Users Data with Binary 1:
[[<<"{\"username\":\"KPBatman1\",\"building\":\"A\",\"device\":\"Fitbit\",\"date\":\"2017-07-23\",\"calorie\":732,\"distance\":6.4399999999999995,\"elevation\":0,\"floor\":0,\"steps\":8}">>],
[<<"{\"username\":\"KPSuperman1\",\"building\":\"A\",\"device\":\"Jawbone\",\"date\":\"2017-07-23\",\"calorie\":0,\"distance\":0.0,\"elevation\":0,\"floor\":0,\"steps\":0}">>]]
The Present Date Sensor Users Data without Binary 2:
[["{\"username\":\"KPBatman1\",\"building\":\"A\",\"device\":\"Fitbit\",\"date\":\"2017-07-23\",\"calorie\":732,\"distance\":6.4399999999999995,\"elevation\":0,\"floor\":0,\"steps\":8}"],
["{\"username\":\"KPSuperman1\",\"building\":\"A\",\"device\":\"Jawbone\",\"date\":\"2017-07-23\",\"calorie\":0,\"distance\":0.0,\"elevation\":0,\"floor\":0,\"steps\":0}"]]
However, when I send the HTTP request - the JSON response I am getting:
{"username":"KPBatman1","building":"A","device":"Fitbit","date":"2017-07-23","calorie":732,"distance":6.4399999999999995,"elevation":0,"floor":0,"steps":8}
{"username":"KPSuperman1","building":"A","device":"Jawbone","date":"2017-07-23","calorie":0,"distance":0.0,"elevation":0,"floor":0,"steps":0}
What is the correct way to send JSON response?
However, when I send the HTTP request - the JSON response I am
getting:
{"username":"KPBatman1","building":"A", ...}
{"username":"KPSuperman1","building":"A", ...}
And? What did you expect/want to get?
The following code works for me because the output is what I expected to see:
-module(cb_tutorial_greeting_controller, [Req]).
-compile(export_all).
hello('GET', []) ->
Headers = [
{'Access-Control-Allow-Origin', "*"},
{'Access-Control-Allow-Methods', "GET, OPTIONS"},
{'Content-Type', "application/json"},
{'Access-Control-Allow-Headers', "X-Requested-With"},
{'Access-Control-Max-Age', "180"}
],
Data = [
[<<"{\"username\":\"KPBatman1\",\"building\":\"A\"}">>],
[<<"{\"username\":\"KPSuperman1\",\"building\":\"A\"}">>]
],
Json = jsx:encode(Data),
{200, Json, Headers}.
In my browser, I see:
[["{\"username\":\"KPBatman1\",\"building\":\"A\"}"],["{\"username\":\"KPSuperman1\",\"building\":\"A\"}"]]
Note that MyFinalList isn't even valid JSON:
13> Data = [["{\"a\":\"Batman\"}"], ["{\"b\":\"Superman\"}"]].
[["{\"a\":\"Batman\"}"],["{\"b\":\"Superman\"}"]]
14> jsx:is_json(Data).
false
See what I did there?
EDIT: Here's a bit more context to how the JSON is received. I'm using the ApiAI API to generate a request to their platform, and they have a method to retrieve it, like this:
# instantiate ApiAI
ai = apiai.ApiAI(CLIENT_ACCESS_TOKEN)
# declare a request obect, fill in in lower lines
request = ai.text_request()
# send ApiAI the request
request.query = "{}".format(textobject.body)
# get response from ApiAI
response = request.getresponse()
response_decode = response.read().decode("utf-8")
response_data = json.loads(response_decode)
I'm coding a webapp in Django and trying to read through a JSON response POSTed to a webhook. The code to read through the JSON, after it has been decoded, is:
if response_data['result']['action'] != "":
Request.objects.create(
request = response_data['result']['resolvedQuery']
)
When I try to run this code, I get this error:
KeyError: 'result'
on the line
if response_data['result']['action'] != "":
I'm confused because it looks to me like 'result' should be a valid key to this JSON that is being read:
{
'id':'65738806-eb8b-4c9a-929f-28dc09d6a333',
'timestamp':'2017-07-10T04:59:46.345Z',
'lang':'en',
'result':{
'source':'agent',
'resolvedQuery':'Foobar',
'action':'Baz'
},
'alternateResult':{
'source':'domains',
'resolvedQuery':'abcdef',
'actionIncomplete':False,
},
'status':{
'code':200,
'errorType':'success'
}
}
Is there another way I should be reading this JSON in my program?
Try:
import JSON
if 'action' in response_data:
parsed_data = json.loads(response_data)
if parsed_data['result']['action'] != "":
Request.objects.create(request = parsed_data['result']['resolvedQuery'])
Thanks for everyone's thoughts. It turned out there was an another error with how I was trying to implement the ApiAI API, and that was causing this error. It now reads through the JSON fine, and I'm using #sasuke's suggestion.
POST ing json from javascript to server in Play Framework:
var myJson = {"name": "joe", "age":20};
var obj = JSON.parse(myJson);
$.ajax(jsRoutes.controllers.MyController.create(obj));
Now, I have the javascript router configured fine. If i recieve the obj as a string I can print it out to the console just fine.
routes.conf:
POST /person/add controllers.MyController.createFromAjax(ajax: String)
BUT, I want to write the json to MongoDB using an Async promise which Activator gives the compile time error:
scala.concurrent.Future[play.api.mvc.Result][error] cannot be applied to (String)
I have other routes that take no parameters that receive json using Postman and write it to MongoDB just fine
routes.conf
POST /heartrates/bulk controllers.HRController.createFromJson
If I omit the parameter on the route that receives the json from Ajax instead of using Postman I get a HTTP 400 error in the browser.
POST http://localhost:9000/person/add 400 (Bad Request)
SO, my question is, Ajax needs a parameter but String wont work. Play documentation says json is always received as a String. What am I doing wrong here?
Scala Controller Code taken from Lightbend seed Play.Reactive.MongoDB:
def createBulkFromAjax = Action.async(parse.json) { request =>
val documents = for {
heartRate <- request.body.asOpt[JsArray].toStream
maybeHeartRate <- heartRate.value
validHeartRate <- maybeHeartRate.transform(transformer).asOpt.toList
} yield validHeartRate
for {
heartRate <- hrFuture
multiResult <- heartRate.bulkInsert(documents = documents, ordered = true)
} yield {
Logger.debug(s"Successfully inserted with multiResult: $multiResult")
Created(s"Created ${multiResult.n} heartRate")
}
}
I think you're getting mixed up between the parameters you pass to your Action as part of the jsRoutes call, and parameters that get passed to endpoints (i.e. the query string, query parameters etc).
Play will return a 400 Bad Request if you've declared a non-optional parameter (like you did with ajax: String) and you don't then actually supply it in your request.
While conceptually you are passing obj to your action, it's not as a query parameter - you've declared that your endpoint expects an HTTP POST - so the JSON should be in the HTTP request body. Notice your other endpoints don't take any query parameters.
So step 1 is to fix your routes file (I've renamed your method to match your other existing working one):
POST /person/add controllers.MyController.createFromJson
If you look at the Play documentation for the Javascript reverse router, you'll see that you'll need to set the type (aka HTTP method) if you're doing something other than a GET. So, step 2, here's what your Javascript should look like to achieve a POST:
var myJson = {"name": "joe", "age":20};
var obj = JSON.stringify(myJson);
var r = controllers.MyController.createFromJson;
$.ajax({url: r.url, type: r.type, data: obj });
After those changes you should be good; your controller code looks fine. If you still get 400 Bad Request responses, check that jQuery is setting your Content-Type header correctly - you may need to use the contentType option in the jQuery $.ajax call.
Edit after still getting 400 errors:
I've just noticed that you were using JSON.parse in your Javascript - as per this answer you should be using JSON.stringify to convert an object into something jQuery can send - otherwise it may try to URLEncode the data and/or send the fields as query parameters.
The other thing to look at is whether the JSON you are sending actually agrees with what you're trying to parse it as. I'm not sure if you've provided a simplified version for this question but it looks like you're trying to parse:
{"name": "joe", "age":20}
Using:
request.body.asOpt[JsArray]
Which will always result in a None - you didn't give it an array.
The Answer to ajax javascript routes in Play Framework 2.5 for ReativeMongo:
routes.conf:
GET /javascriptRoutes controllers.HRController.javascriptRoutes
HRController:
def javascriptRoutes = Action { implicit request =>
Ok(
JavaScriptReverseRouter("jsRoutes")(
routes.javascript.HRController.createBulkFromAjax
)
).as("text/javascript")
}
routes.conf:
POST /heartrates/add controllers.HRController.createBulkFromAjax
main.scala.html:
<script type="text/javascript" src="#routes.HRController.javascriptRoutes"></script>
javascript:
var r = jsRoutes.controllers.HRController.createBulkFromAjax();
$.ajax({url: r.url, type: r.type, contentType: "application/json", data: JsonString });
HRController:
def createBulkFromAjax = Action.async(parse.json) { request =>
//Transformation silent in case of failures.
val documents = for {
heartRate <- request.body.asOpt[JsArray].toStream
maybeHeartRate <- heartRate.value
validHeartRate <- maybeHeartRate.transform(transformer).asOpt.toList
} yield validHeartRate
for {
heartRate <- hrFuture
multiResult <- heartRate.bulkInsert(documents = documents, ordered = true)
} yield {
Logger.debug(s"Successfully inserted with multiResult: $multiResult")
Created(s"Created ${multiResult.n} heartRate")
}
}
HRController.createBulkFromAjax was built from a Lightbend activator ui seed example called play.ReactiveMogno