My professor wants me to write a little tutorial on how to deploy Ballerina services. So I'm trying to learn it. I'm using Version 1.2 and I'm a bit overwhelmed by the concept of taint checking and the variable types...
I'm trying to write a minimal REST-Service with an endpoint that requests json data from another api and then uses that JSON to do stuff.
What's working so far is the following:
service tutorial on new http:Listener(9090) {
// Resource functions are invoked with the HTTP caller and the incoming request as arguments.
resource function getName(http:Caller caller, http:Request req) {
http:Client clientEP = new("https://api.scryfall.com/");
var resp = clientEP->get("/cards/random");
if (resp is http:Response) {
var payload = resp.getJsonPayload();
if (payload is json) {
var result = caller->respond(<#untainted>(payload));
} else {
log:printError("");
}
} else {
log:printError("");
}
}
That responds with the JSON that is returned from https://api.scryfall.com/cards/random
But lets now say, that I want to access a single value from that JSON. e.G. "name".
If I try to access it like this: payload["name"]
I get: invalid operation: type 'json' does not support indexing
I just figured out that it works if I create a map first like that:
map mp = <map>payload;
If I then access mp["name"] it works. BUT WHY? What is the json type good for if you still have to create a map and then cast the payload? And how would I access json inside the json? For example mp["data"][0]... invalid operation: type 'json' does not support indexing again...
And I'm still trying to understand the concept of taint checking....
do I just cast everything that is tainted to <#untainted> after checking the content?
Sometimes I really do not get what the documentation is trying to tell me....
I would recommend you to use Ballerina Swan Lake versions. Swan Lake versions contain enhancements to various language features. Here is a sample code that covers your use case. You can download Swan Lake Alpha2 at https://ballerina.io/
import ballerina/io;
import ballerina/http;
service tutorial on new http:Listener(9090) {
resource function get payload() returns json|error {
http:Client clientEP = check new ("https://api.scryfall.com/");
json payload = <json> check clientEP -> get("/cards/random", targetType = json);
// Processing the json payload
// Here the type of the `payload.name` expression is json | error
// You can eliminate with check: it returns from this resource with this error
json nameField = check payload.name;
io:println(nameField);
// You can traverse the json tree as follows
json standardLegality = check payload.legalities.standard;
io:println(standardLegality);
// colors is an array
// The cast is necessary because `check payload.colors` gives you a json
json colors = <json[]> check payload.colors;
io:println(colors);
// Responding with the complete payload recived from api.scryfall.com
return payload;
}
}
Taint analysis helps you to write code with no security vulnerabilities. However, we've disabled taint analysis in Swan Lake versions. If you want to enable it, you can use the option --taint-check with bal build
Related
I'm calling a REST service using Refit and I want to deserialize the JSON that is returned as a dynamic type.
I tried defining the interface as
[Get("/foo")]
Task<dynamic> GetFoo();
but the call times out.
I know I can deserialize to a dynamic like this
var mockString = "{ title: { name: 'fred', book: 'job'} }";
dynamic d = JsonConvert.DeserializeObject(mockString);
but I can't figure out what to pass to Refit to get it to do the same.
Another option would be to get Refit to pass the raw JSON back so I can deserialize it myself but I can't see a way to do that either.
Any ideas?
You can define your interface to return a string and get the raw JSON that way:
[Get("/foo")]
Task<string> GetFoo();
As described here:
https://github.com/paulcbetts/refit#retrieving-the-response
Refit uses JSON.NET under the hood, so any deserialization that works with that will work with Refit, including dynamic. The interface you have described is exactly right.
Here's a real working example:
public interface IHttpBinApi
{
[Get("/ip")]
Task<dynamic> GetIp();
}
var value = await RestService.For<IHttpBinApi>("http://httpbin.org").GetIp();
If you are using iOS and Refit 4+, you might be seeing this bug: https://github.com/paulcbetts/refit/issues/359
As Steven Thewissen has stated, you can use Task<string> as your return type (or Task<HttpResponseMessage>, or even Task<HttpContent>) to receive the raw response and deserialize yourself, but you shouldn't have to -- the whole point of Refit is that it's supposed to save you that hassle.
I have implemented Play framework's WebSocket so as to perform server communication using a WebSocket instead of Http. I have created a function as WebSocket.using[JsValue]. My json response is stored inside a Future[JsValue] variable and I am trying to fetch and return the json value from within Future[JsValue] variable. However I have been unable to return the json data from the Future[JsValue] variable. When I tried creating the WebSocket function as WebSocket.using[Future[JsValue]], in this case I was unable to create a json FrameFormatter for it.
def socketTest = WebSocket.using[JsValue] { request =>
val in = Iteratee.ignore[JsValue]
val out = Enumerator[JsValue](
Json.toJson(futureJsonVariable)
).andThen(Enumerator.eof)
(in, out)
}
futureJsonVariable is a variable of type Future[JsValue] In the above code the error at runtime is No Json serializer found for type scala.concurrent.Future[play.api.libs.json.JsValue]. Try to implement an implicit Writes or Format for this type. How can I return a json using a WebSocket method in Scala ? How can it be achieved using an Actor class instance ? If anyone knows best available online tutorials for WebSocket in Play framework. Any help is appreciated.
Use tryAccept to return either the result of the future when it is redeemed, or an error:
def socketTest = WebSocket.tryAccept[JsValue] { request =>
futureJsonVariable.map { json =>
val in = Iteratee.ignore[JsValue]
val out = Enumerator(json).andThen(Enumerator.eof)
Right((in, out))
} recover {
case err => Left(InternalServerError(err.getMessage))
}
}
This is similar to using but returns a Future[Either[Result, (Iteratee[A, _], Enumerator[A])]]. The Either[Result, ...] allows you to handle the case where something unexpected occurs calculating the future value A by providing a play.api.mvc.Result in the Left branch. The corollary is that you need to also wrap the "happy path" (where nothing goes wrong) in Right, in this case the iteratee/enumerator tuple you'd ordinarily return from using.
You can do something similar with the tryAcceptWithActor function.
I have the following code which publishes the json data in the specified url using mqtt.The initial data is retrieved from http.
var request = require('request');
var JSONStream = require('JSONStream');
var es = require('event-stream');
var mqtt = require('mqtt');
request({url: 'http://isaacs.couchone.com/registry/_all_docs'})
.pipe(JSONStream.parse('rows.*'))
.pipe(es.mapSync(function (data) {
console.info(data);
var client = mqtt.createClient(1883, 'localhost');
client.publish('NewTopic', JSON.stringify(data));
client.end();
return data;
}))
The following is the subscriber code which subscribes the data that is published (in the above code) through mqtt
var mqtt = require('mqtt');
var client = mqtt.createClient();
client.subscribe('NewTopic');
client.on('message', function(topic, message) {
console.info(message);
});
In the above code, I get all json data in the specified url in 'message'.I need to extract 'id' and 'value' from the received data and make it as a single JSON object and need to publish it to mqtt,so that another client can subscribe only the 'id' and 'value' as json data.
To convert a JSON text into an object, you can use the eval() function. eval() invokes the JavaScript compiler. Since JSON is a proper subset of JavaScript, the compiler will correctly parse the text and produce an object structure. The text must be wrapped in parens to avoid tripping on an ambiguity in JavaScript's syntax.
var myObject = eval(message);
The eval function is very fast. However, it can compile and execute any JavaScript program, so there can be security issues. The use of eval is indicated when the source is trusted and competent. It is much safer to use a JSON parser. In web applications over XMLHttpRequest, communication is permitted only to the same origin that provide that page, so it is trusted. But it might not be competent. If the server is not rigorous in its JSON encoding, or if it does not scrupulously validate all of its inputs, then it could deliver invalid JSON text that could be carrying dangerous script. The eval function would execute the script, unleashing its malice.
To defend against this, a JSON parser should be used. A JSON parser will recognize only JSON text, rejecting all scripts. In browsers that provide native JSON support, JSON parsers are also much faster than eval.
var myObject = JSON.parse(message);
And use it as a Object:
myObject.id;
myObject.value;
Create a object with just id and value :
var idAndValueObj = {};
idAndValueObj.id = myObject.id;
idAndValueObj.value = myObject.value;
Convert to JSON string:
var jsonStr = JSON.stringify(idAndValueObj);
I am trying to access a REST Service in my MVC application.I am calling getJSON method to get the data from a controller which internally calls the REST service which returns data in json format.But I am getting the a lot of "\ in my output of DownLoadString method and my return Json is not returning proper JSON data and hence my client side script is not able to access the JSON properties.
My Script in my view is
$.getJSON("#Url.Action("GetManufacturers", "Home")",function(data){
console.debug("Status is : "+data.Status)
});
My Action method looks like this
public ActionResult GetManufacturers()
{
string restURL ="http://mytestserver/myapi/Manufacturers";
using (var client = new WebClient())
{
var data = client.DownloadString(restURL);
//data variable gets "\" everywhere
return Json(data,JsonRequestBehavior.AllowGet);
}
}
I used visual studio breakpoints in my action method and i am seeing a lot of \"
And i checked what is coming out to my getJSON callback and the JSON tab is empty.
But my response tab has content like this
I belive if there is no \", i would be able to parse it nicely.
I used fiddler to see whether i am getting correct (JSON format) data from my REST service and it seems fine.
Can anyone help me to tackle this ? I would like to return proper JSON from my action method. Sometime i may want to read the json properties in the C# code itself. I saw some example of doing it with DataContractJsonSerializer. But that needs a concrete type to be converted to. I don't want to do that. because other clients would also access my RESTService and how will expect them to write a fake entity for this ?
You need to return the data as is:
public ActionResult GetManufacturers()
{
string restURL ="http://mytestserver/myapi/Manufacturers";
using (var client = new WebClient())
{
var data = client.DownloadString(restURL);
return Content(data, "application/json");
}
}
I have the following NancyFX unit test. I use the Shouldly assertion library to give the set of extensions methods that start .Should---
[Fact]
public void Assessment__Should_return_assessment_state_for_specified_user()
{
const AssessmentState assessmentState = AssessmentState.Passed;
var user = Fake.Mentor();
using (var db = Fake.Db())
{
db.Save(user);
Fake.Assessment(user.Id, db, assessmentState);
db.ClearStaleIndexes();
}
var response = Fake.Browser(user.UserName, user.Password)
.Get("/assessment/state/" + user.Id, with => with.HttpRequest());
//var result = (dynamic)body.DeserializeJson<ExpandoObject>();
var result = (dynamic) JsonConvert.DeserializeObject<ExpandoObject>(response.Body.AsString());
result.ShouldNotBe(null);
((AssessmentState) result.State).ShouldBe(assessmentState);
}
This test calls a AssessmentService uri defined as /assessment/state/" + user.Id which returns a simple JSON object definition that has a single property State of type (enum) AssessmentState, either Passed, Failed or NotStarted.
Here is the service handler so you can see there are no tricks.
Get["/assessment/state/{userid}"] = parameters =>
{
var assessment = AssessmentService.GetByUserId(Db, (string)parameters.userid);
return assessment == null ? HttpStatusCode.NotFound : Response.AsJson(new
{
assessment.State
});
};
And here is an example the JSON this service call returns:
{"State":1}
Everything works fine until I try to Deserialize the JSON returned by the fake Nancy browser. First I tried to use the built in method provided by Nancy's BrowserResponse.Body object:
var result = (dynamic)response.Body.DeserializeJson<ExpandoObject>();
This deserializes to an empty object. Which is no good. However, if we use the Newtonsoft equivalent then everything is fine (almost).
var result = (dynamic) JsonConvert.DeserializeObject<ExpandoObject>(response.Body.AsString());
The JSON deserialization now works and so the following Shouldly assertion passes with flying colours:
((AssessmentState) result.State).ShouldBe(assessmentState);
However, for reasons that I suspect have to do with anonymous types, the following line fails at run-time (it compiles fine).
result.ShouldNotBe(null);
That is quite a lot of information. Let me distil it down to two questions:
Why does Nancy's built in JSON deserializer not work given that the Newtonsoft version does?
How do I work with the dynamic types generated by the JSON de-serialisation so that the Shouldly extension methods do not cause a run-time exception?
Thanks
I can't answer the first question, but WRT Shouldly and dynamic types, Shouldly's ShouldNotBe method is an extension method on object. The DLR doesn't allow you to call extension methods on objects typed as dynamic (hence the runtime binder exception you're seeing)
I'd suggest that if you want to call ShouldNotBe(null) on result, you'd have to cast it to an object first (ie: ((object)result).ShouldNotBe(null))
-x