How to parse JSON and urlencoded responses with Jetty HttpClient? - json

Please recommend the optimal approach for parsing urlencoded or JSON-encoded responses when using Jetty HttpClient.
For example, I have created the following utility class for sending ADM-messages and use BufferingResponseListener there, with UrlEncoded.decodeUtf8To​ (for parsing bearer token response) and JSON.parse (for parsing message sending response):
private final HttpClient mHttpClient;
private final String mTokenRequest;
private String mAccessToken;
private long mExpiresIn;
public Adm(HttpClient httpClient) {
mHttpClient = httpClient;
MultiMap<String> params = new MultiMap<>();
params.add("grant_type", "client_credentials");
params.add("scope", "messaging:push");
params.add("client_id", "amzn1.application-oa2-client.XXXXX");
params.add("client_secret", "XXXXX");
mTokenRequest = UrlEncoded.encode(params, null, false);
}
private final BufferingResponseListener mMessageListener = new BufferingResponseListener() {
#Override
public void onComplete(Result result) {
if (!result.isSucceeded()) {
if (result.getResponse().getStatus() % 100 == 4) {
String jsonStr = getContentAsString(StandardCharsets.UTF_8);
Map<String, String> resp = (Map<String, String>) JSON.parse(jsonStr);
String reason = resp.get("reason");
if ("AccessTokenExpired".equals(reason)) {
postToken();
} else if ("Unregistered".equals(reason)) {
// delete the invalid ADM registration id from the database
}
}
return;
}
String jsonStr = getContentAsString(StandardCharsets.UTF_8);
Map<String, String> resp = (Map<String, String>) JSON.parse(jsonStr);
String oldRegistrationId = (String) result.getRequest().getAttributes().get("registrationID");
String newRegistrationId = resp.get("registrationID");
if (newRegistrationId != null && !newRegistrationId.equals(oldRegistrationId)) {
// update the changed ADM registration id in the database
}
}
};
private final BufferingResponseListener mTokenListener = new BufferingResponseListener() {
#Override
public void onComplete(Result result) {
if (result.isSucceeded()) {
String urlencodedStr = getContentAsString(StandardCharsets.UTF_8);
MultiMap<String> params = new MultiMap<>();
UrlEncoded.decodeUtf8To(urlencodedStr, params);
long now = System.currentTimeMillis() / 1000;
mExpiresIn = now + Long.parseLong(params.getString("expires_in"));
mAccessToken = params.getString("access_token");
}
}
};
public void postMessage(String registrationId, int uid, String jsonStr) {
long now = System.currentTimeMillis() / 1000;
if (mAccessToken == null || mAccessToken.length() < 32 || mExpiresIn < now) {
postToken();
return;
}
mHttpClient.POST(String.format("https://api.amazon.com/messaging/registrations/%1$s/messages", registrationId))
.header(HttpHeader.ACCEPT, "application/json")
.header(HttpHeader.CONTENT_TYPE, "application/json")
.header(HttpHeader.AUTHORIZATION, "Bearer " + mAccessToken)
.header("X-Amzn-Type-Version", "com.amazon.device.messaging.ADMMessage#1.0")
.header("X-Amzn-Accept-Type", "com.amazon.device.messaging.ADMSendResult#1.0")
.attribute("registrationID", registrationId)
.content(new StringContentProvider(jsonStr))
.send(mMessageListener);
}
private void postToken() {
mHttpClient.POST("https://api.amazon.com/auth/O2/token")
.header(HttpHeader.ACCEPT, "application/json")
.header(HttpHeader.CONTENT_TYPE, "application/x-www-form-urlencoded")
.content(new StringContentProvider(mTokenRequest))
.send(mTokenListener);
}
The above class works okay, but seeing that there are Jetty-methods with InputStream in arguments, like
UrlEncoded.decodeTo​(java.io.InputStream in, MultiMap map, java.lang.String charset, int maxLength, int maxKeys)
and
JSON.parse​(java.io.InputStream in)
I wonder if there is a smarter way to fetch and parse... maybe with something more effective than BufferingResponseListener?
In other words my question is please:
How to use the "streaming" version of the above parsing methods with HttpClient?

Related

Signing the Header and Payload with json private key in C#

I have to generate a Client assertion string signing with the json private key.
I have the private key in the private.json file. How do I use the private key in the file to sign the Header and payload?
Except the signing part, I have managed to get the other things to work. Following is the code I use for getting the Signed Client assertion string
public string GetSignedClientAssertion()
{
var header = new Dictionary<string, string>()
{
{ "typ" , "JWT"},
{ "alg", "ES512"},
{ "kid", "TST_Staging" }
};
string token = Encode(Encoding.UTF8.GetBytes(JObject.FromObject(header).ToString())) + "." + Encode(Encoding.UTF8.GetBytes(JObject.FromObject(GetClaims()).ToString()));
var rsa = new RSACryptoServiceProvider();
var filename = System.Web.HttpContext.Current.Server.MapPath("Private.json");
string data = File.ReadAllText(filename);
string key = Encode(Encoding.UTF8.GetBytes(data));
byte[] databyte = File.ReadAllBytes(filename);
//-----I am stuck here on how to sign the token with the Private key from the private.key file
//---the method used for SignData does not work
string signature = SignData(token, data);
signature = Encode(Encoding.UTF8.GetBytes(signature));
return signedClientAssertion = string.Concat(token, ".", signature);
}
Private.json has something like the following
{
"kty": "EC",
"d": "AfJ_hlRFCP0g2PghjghjghjtryrtytyFpbALpoG0gqh9tyaSv8JIZuhKYOgvbAzkI6pi2gdCce3fvWb5csiL24PiS9Ke5CKlh3QyW-YOO",
"use": "sig",
"crv": "P-521",
"kid": "TST_Staging",
"x": "ADRSCG8Acsqj6SlShpEJYa9UhA7ojghjgjK4eUVHj9CDqbH4j2_F84j7qtK4fdH94xGzYqQwV0rLfJrAISknoudPQm743H",
"y": "AYnLkWp3Up69WQoc-kZ8ugvSiCNChMiBra3jLHmSotDdzSJ6MgMCokfRdHsfsF-z4VAGq3zam1Z604_rC5N9xrtyrtyufV",
"alg": "ES512"
}
Can someone point out how to sign the token with the Private key will be much helpful. Thanks.
I have managed to find the solution myself. I am using EC Json public and private keys. Previously I was using the Algorithm ES512 which had some issues while decrypting the JWE response from the server, so I had changed my JWT key algorithm to ES256 and with this encryption and decryption works perfectly with Jose.JWT.
I have been struggling for more than a month to find a solution for the EC Json Public and private key Encryption and Decryption. All the solution available out there was for PEM key method only and nothing much for the Json EC keys. It took me a lot of time find the necessary pieces and put them together and make it work. I am providing the entire code for the page where the encryption and decryption happens, so that someone might find it useful.
using System.Net;
using System.Text; // For class Encoding
using System.IO; // For StreamReader
using System;
using System.Collections.Generic;
using System.Configuration;
using Jose;
using System.Security.Cryptography;
using System.Web;
using Jose.keys;
using Stream = System.IO.Stream;
using System.Web.Script.Serialization;
using Newtonsoft.Json;
public partial class holding : System.Web.UI.Page
{
string ClientID = "mlDBKnXXXXXTYRTYRTYyYdmzFGD6HcBPsHZ2W";
string RemoteURL = "https://www.example.org";
string RemoteTokenAppend = "/token";
string RedirectURL = "https://www.exampleabc.com/holding.aspx";
//-----private key for Encrypting the token
string PrivateKey = #"
{
""x"": ""4kELRTR545GDGDGtvilOLrtr5luaQaWgaTlpqUf7o"",
""y"": ""iCyNdwX73FWKJTjn1Q19gdjEILKjEILK3Y_XwgY3Y_XwgY"",
""d"": ""wiYrwNa5SgBNgdqRtSMpaUvRmipaBJ6hfmL1CUMpwlQ"",
""kty"": ""EC"",
""crv"": ""P-256""
}";
Dictionary<string, object> PrivateKeyHeader = new Dictionary<string, object>
{
{ "typ", "JWT" },
{ "kid", "TST_SERVER" },
{ "alg", "ES256" }
};
//----Public Enc key for Decrypting the JWE token from the remote host
string PublicKey_Enc = #"
{
""x"": ""_ylhMfdVwaRrLx8HL8z7X1ixVkk2rbpwD9oU-uAqyhE"",
""y"": ""aCRo4kY2dTl7wZXjsp2NJyF9Tcmzk1XZN5ueJWNq7Lk"",
""d"": ""Jz9aEpbt_4aKL5FVdCLlux7U-Ubt_4aKL5VdCLLTR2Y"",
""kty"": ""EC"",
""crv"": ""P-256""
}";
public void Page_Load(object sender, EventArgs e)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(RemoteURL + RemoteTokenAppend);
ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;
request.ProtocolVersion = HttpVersion.Version10;
var postData = "client_assertion_type=" + HttpUtility.UrlEncode("urn:ietf:params:oauth:client-assertion-type:jwt-bearer");
postData += "&client_id=" + HttpUtility.UrlEncode(ClientID);
postData += "&grant_type=" + HttpUtility.UrlEncode("authorization_code");
postData += "&redirect_uri=" + HttpUtility.UrlEncode(RedirectURL);
postData += "&code=" + HttpUtility.UrlEncode(Request.QueryString["code"]);
postData += "&client_assertion=" + EncryptTokenJose();
var data = Encoding.ASCII.GetBytes(postData);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("Content-Encoding", "ISO-8859-1");
request.ContentLength = data.Length;
try
{
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var webResponse = (HttpWebResponse)request.GetResponse();
var webStream = webResponse.GetResponseStream();
var responseReader = new StreamReader(webStream);
var response = responseReader.ReadToEnd();
//---Decrypt the Response from Remote
string JoseRes = DecryptTokenJose(response.ToString());
//---close the response reader
responseReader.Close();
Response.Write("<br><br>Payload = " + JoseRes + "<br><br>");
//---Get the json string and values from the decrypted Token
JsonTextReader reader = new JsonTextReader(new StringReader(JoseRes));
var sData = JsonSerializer.CreateDefault().Deserialize<payload>(reader);
//----Sample Payload
//---- Payload = { "sub":"s=S775566X,u=c57acrtereb8-d102-455a-860a-ae7dretef4b8d","aud":"mlDBKnGyYfgdgdmzwx770mqXKb6HcBPsHZ2W","amr":["pwd","swk"],"iss":"https:\/\/www.exampleabc.com","exp":1637810980,"iat":1637810380,"nonce":"TJtv85f586sTgxjUlFm5"}
//----get the sub from Payload
string sub = sData.sub;
Response.Write("<br><br>Sub = " + sub + "<br><br>");
//---Get the list array of sub to extract the IC
string[] subList = sub.Split(',');
//---Get the first item of the array to get the IC
Response.Write("<br><br>NRIC = " + subList[0] + "<br><br>");
//---Retrieved IC value will be s=S775566X, so replace s= to "" to get the exact NRIC value and set the Session value
Session["NRIC"] = subList[0].Replace("s=", "");
//---if Session["RedirctPage"] is set then go to that page
if (Session["RedirctPage"] != null)
Response.Redirect(Session["RedirctPage"].ToString());
else
Response.Redirect("Register_uat.aspx");
}
catch (WebException ex)
{
using (WebResponse response = ex.Response)
{
string ErrorString = "Error from the Server:-----<br><br>";
HttpWebResponse httpResponse = (HttpWebResponse)response;
using (Stream data1 = response.GetResponseStream())
using (var reader = new StreamReader(data1))
{
ErrorString += reader.ReadToEnd();
Response.Write(ErrorString);
}
}
}
}
public string EncryptTokenJose()
{
const uint JwtToAadLifetimeInSeconds = 60 * 2;
DateTime validFrom = DateTime.UtcNow;
long exp = ConvertToTimeT(validFrom + TimeSpan.FromSeconds(JwtToAadLifetimeInSeconds));
long iat = ConvertToTimeT(validFrom);
//---Payload string
string payloadStr = "{\"aud\":\"" + RemoteURL + "\",\"exp\":" + exp.ToString() + ",\"iss\":\"" + ClientID + "\",\"iat\": " + iat.ToString() + ",\"sub\":\"" + ClientID + "\"}";
//---get the Private key to form the public key
JsonTextReader reader = new JsonTextReader(new StringReader(PrivateKey));
var jwk = JsonSerializer.CreateDefault().Deserialize<JWK>(reader);
var publicECCKey = EccKey.New(Base64Url.Decode(jwk.x), Base64Url.Decode(jwk.y), Base64Url.Decode(jwk.d), usage: CngKeyUsages.KeyAgreement);
string token = Jose.JWT.Encode(payloadStr, publicECCKey, JwsAlgorithm.ES256, extraHeaders: PrivateKeyHeader);
return token;
}
public string DecryptTokenJose(string Res)
{
var jss = new JavaScriptSerializer();
string json = Res;
Dictionary<string, string> sData = jss.Deserialize<Dictionary<string, string>>(json);
string AccessToken = sData["access_token"].ToString();
string TokenType = sData["token_type"].ToString();
string IdToken = sData["id_token"].ToString();
Response.Write("<br><br>" + IdToken + "<br><br>");
//---get the Public key Enc to decrypt the JWE token
JsonTextReader reader = new JsonTextReader(new StringReader(PublicKey_Enc));
var jwk = JsonSerializer.CreateDefault().Deserialize<JWK>(reader);
var publicECCKey = EccKey.New(Base64Url.Decode(jwk.x), Base64Url.Decode(jwk.y), Base64Url.Decode(jwk.d), usage: CngKeyUsages.KeyAgreement);
//---get the decrypted token
string token = Jose.JWT.Decode(IdToken, publicECCKey, JweAlgorithm.ECDH_ES_A128KW, JweEncryption.A256CBC_HS512);
//----todo: Verify the signature of the decoded JWS token
//----5 parts token with dot(.) as separator
string[] toklist = token.Split('.');
//----get the 2nd item of the list which will have the Payload with User IC details
string Payload = toklist[1];
//----decode the payload to bytes and from bytes to readable string
var base64EncodedBytes = Base64Url.Decode(Payload);
return System.Text.Encoding.UTF8.GetString(base64EncodedBytes);
}
public static long ConvertToTimeT(DateTime dt)
{
return (long)(dt - new DateTime(1970, 1, 1, 0, 0, 0, 0)).TotalSeconds;
}
public class payload
{
public string sub { get; set; }
public string aud { get; set; }
public string iss { get; set; }
public string exp { get; set; }
public string iat { get; set; }
}
public class JWK
{
public string x { get; set; }
public string y { get; set; }
public string d { get; set; }
}
}

Get data array from sqlite database and post to API via json object... can be possible?

newbie here... i was developing app that send my data to api via retrofit. my code was working but it sends 1 data input only at the time.... in my case, i've like to do is I want to get more saved data in my sqlite (example 5 data saved) and send it all on api via json object.
This is my Activity:
DatabaseHelper databaseHelper2 = new
DatabaseHelper(getApplicationContext());
SQLiteDatabase db2 =
databaseHelper2.getWritableDatabase();
Cursor cursor =
databaseHelper2.retrieveSettingFromLocalDatabase(db2);
while (cursor.moveToNext()) {
ADDRESS =
cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_ADDRESS));
PORT =
cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_PORT));
TIMEINTERVAL=cursor.getString
(cursor.getColumnIndex(DatabaseHelper.SETTING_TIME_INTERVAL));
}
portInts=Integer.parseInt(PORT);
MapDetails mapDetails = new MapDetails(gg, lat, lon,
well, "0", portInts); //Datas ive get to send in api
List<MapDetails> data = new ArrayList<>();
data.add(mapDetails);
Retrofit.Builder builder = new Retrofit.Builder()
.baseUrl("http://" + ADDRESS + ":" + PORT)
.addConverterFactory(GsonConverterFactory.create());
Retrofit retrofit = builder.build();
Api locate = retrofit.create(Api.class);
Call<MapDetails> call = locate.mapDetailLocation(data);
call.enqueue(new Callback<MapDetails>() {
#Override
public void onResponse(Call<MapDetails> call, Response<MapDetails> response) {
Snackbar.make(view, "" + response,
Snackbar.LENGTH_INDEFINITE)
.setAction("Action", null).show();
}
#Override
public void onFailure(Call call, Throwable t) {
Snackbar.make(view, "" + t.getMessage(),
Snackbar.LENGTH_INDEFINITE)
.setAction("Action", null).show();
}
});
This is my code in API:
public interface Api {
#POST("/api/Database/NewLocation")
Call<MapDetails> mapDetailLocation(#Body List<MapDetails> mapDetails)
}
This is my sample Client:
public class MapDetails {
#SerializedName("SerialNumber")
#Expose
private String SerialNumber;
#SerializedName("Coordinate1")
#Expose
private String Coordinate1;
#SerializedName("Coordinate2")
#Expose
private String Coordinate2;
#SerializedName("DateTime")
#Expose
private String DateTime;
#SerializedName("Speed")
#Expose
private String Speed;
#SerializedName("Port")
#Expose
private int Port;
public MapDetails(String serialNumber, String coordinate1, String
coordinate2, String dateTime, String speed, int port) {
SerialNumber = serialNumber;
Coordinate1 = coordinate1;
Coordinate2 = coordinate2;
DateTime = dateTime;
Speed = speed;
Port = port;
}
public String getSerialNumber() {
return SerialNumber;
}
public void setSerialNumber(String serialNumber) {
SerialNumber = serialNumber;
}
public String getCoordinate1() {
return Coordinate1;
}
public void setCoordinate1(String coordinate1) {
Coordinate1 = coordinate1;
}
public String getCoordinate2() {
return Coordinate2;
}
public void setCoordinate2(String coordinate2) {
Coordinate2 = coordinate2;
}
public String getDateTime() {
return DateTime;
}
public void setDateTime(String dateTime) {
DateTime = dateTime;
}
public String getSpeed() {
return Speed;
}
public void setSpeed(String speed) {
Speed = speed;
}
public int getPort() {
return Port;
}
public void setPort(int port) {
Port = port;
}
}
this is my sqlite database ive like to retrieve:
this is the sample posting ive created at the top
but in my case, ive like to do is this one, getting the saved data from my database and send it like this,:
The reason why only one is being sent is that you are sending outside of the while loop that traverses the Cursor, so only the last is sent.
That is you have :-
while (cursor.moveToNext()) {
ADDRESS = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_ADDRESS));
PORT = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_PORT));
TIMEINTERVAL=cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_TIME_INTERVAL));
}
So say the query extracted a Cursor with 10 rows as address 1,2,3....10 (for explantory purposes) then
The loop is entered ADDRESS is set to 1, the next iteration sets it to 2, the next to 3 ..... and finally ADDRESS is set to 10 (same for PORT and TIMEINTERVAL)
After the loop the data is sent so only one is sent (ADDRESS 10).
What you need is along the lines of :-
List<MapDetails> data = new ArrayList<>();
MapDetails mapDetails
Retrofit.Builder builder;
Retrofit retrofit;
Call<MapDetails> call;
Api locate;
while (cursor.moveToNext()) {
ADDRESS = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_ADDRESS));
PORT = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_PORT));
TIMEINTERVAL=cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_TIME_INTERVAL));
portInts=Integer.parseInt(PORT);
mapDetails = new MapDetails(gg, lat, lon, well, "0", portInts);
data.clear(); //<<<<<<<< remove previous entries if required????
data.add(mapDetails);
builder = new Retrofit.Builder()
.baseUrl("http://" + ADDRESS + ":" + PORT)
.addConverterFactory(GsonConverterFactory.create());
retrofit = builder.build();
locate = retrofit.create(Api.class);
call = locate.mapDetailLocation(data);
call.enqueue(new Callback<MapDetails>() {
#Override
public void onResponse(Call<MapDetails> call, Response<MapDetails> response) {
Snackbar.make(view, "" + response,
Snackbar.LENGTH_INDEFINITE).setAction("Action", null).show();
}
#Override
public void onFailure(Call call, Throwable t) {
Snackbar.make(view, "" + t.getMessage(),
Snackbar.LENGTH_INDEFINITE).setAction("Action", null).show();
}
}
Note the above is in-principle code. it has not been checked or tested and may therefore contain some errors.
It may be that you can send an entire set e.g. data with populated in which case you may only need up to data.add(mapDetails); in the loop and then have the following code outside the loop.
If I understood your question properly, you need to send JSONArray as payload for the request. But, you made a little mistake while preparing payload from SQLite database. #Mike T pointed out that mistake in his answer to your question.
Follow these codes to fix the problem.
DatabaseHelper databaseHelper2 = new DatabaseHelper(getApplicationContext());
SQLiteDatabase db2 = databaseHelper2.getWritableDatabase();
Cursor cursor = databaseHelper2.retrieveSettingFromLocalDatabase(db2);
List<MapDetails> data = new ArrayList<>(); // declare ArrayList outside and before while loop
while (cursor.moveToNext()) {
ADDRESS = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_ADDRESS));
PORT = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_PORT));
TIMEINTERVAL = cursor.getString(cursor.getColumnIndex(DatabaseHelper.SETTING_TIME_INTERVAL));
// pass arguments to MapDetails class constructor
portInts = Integer.parseInt(PORT);
MapDetails mapDetails = new MapDetails(gg, lat, lon, well, "0", portInts); //Datas ive get to send in api
// add prepared data to ArrayList
data.add(mapDetails);
}
// and finally execute network call
Retrofit.Builder builder = new Retrofit.Builder()
.baseUrl("http://" + ADDRESS + ":" + PORT)
.addConverterFactory(GsonConverterFactory.create());
Retrofit retrofit = builder.build();
Api locate = retrofit.create(Api.class);
Call<MapDetails> call = locate.mapDetailLocation(data);
call.enqueue(new Callback<MapDetails>() {
#Override
public void onResponse(Call<MapDetails> call, Response<MapDetails> response) {
Snackbar.make(view, "" + response, Snackbar.LENGTH_INDEFINITE).setAction("Action", null).show();
}
#Override
public void onFailure(Call call, Throwable t) {
Snackbar.make(view, "" + t.getMessage(), Snackbar.LENGTH_INDEFINITE).setAction("Action", null).show();
}
});
PS: I'm not sure why are you taking ADDRESS and PORT from SQLite database. If they're same in every single row you don't need to take it from database right?

Increasing maxjsonlength on MVC post from Javascript

I have a controller action Export which accepts a List of models like below. This is sending back and manipulated dataset back from the view where the user could interact with it. So we have been able to send the data down with much more information.
[HttpPost]
public JsonResult Export(List<MappingExportModel> sources){}
This works fine in all cases but there is one where we have a bigger than normal dataset. This is causing an issue with the export. So far I have tried just passing the values as an object or string but I am unable to convert them into any usable instance after the data is into the controller.
Is it possible to preemptively increase this maxjsonlength value somewhere. The value from the web.config is being ignored from what I have come across so far.
The error I receive is
"Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property"
I need to be able to accept this directly from the ajax request into the controller action. Spinning up a version of JsonResult and then setting the max value will not work because the error is thrown the the data is trying to be deserialized into the object var presented above. We get the value in the original GET request and do set the value before the view is loaded. Now we are taking the data from this view and sending it back plus all the manipulations the users have created.
User posts data to server, the controller action is hit with the data. The error is encountered and spit back out to the browser which handles the error.
You can use custom json length. add the following file in your project and edit your global.asax.cs
Global.asax
public class MvcApplication : System.Web.HttpApplication
{
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
WebApiConfig.Register(GlobalConfiguration.Configuration);
FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
RouteConfig.RegisterRoutes(RouteTable.Routes);
BundleConfig.RegisterBundles(BundleTable.Bundles);
///// **********
JsonValueProviderFactory jsonValueProviderFactory = null;
foreach (var factory in ValueProviderFactories.Factories)
{
if (factory is JsonValueProviderFactory)
{
jsonValueProviderFactory = factory as JsonValueProviderFactory;
}
}
//remove the default JsonVAlueProviderFactory
if (jsonValueProviderFactory != null) ValueProviderFactories.Factories.Remove(jsonValueProviderFactory);
//add the custom one
ValueProviderFactories.Factories.Add(new CustomJsonValueProviderFactory());
/////*************
}
}
///******** for json length
public sealed class CustomJsonValueProviderFactory : ValueProviderFactory
{
private static void AddToBackingStore(Dictionary<string, object> backingStore, string prefix, object value)
{
IDictionary<string, object> d = value as IDictionary<string, object>;
if (d != null)
{
foreach (KeyValuePair<string, object> entry in d)
{
AddToBackingStore(backingStore, MakePropertyKey(prefix, entry.Key), entry.Value);
}
return;
}
IList l = value as IList;
if (l != null)
{
for (int i = 0; i < l.Count; i++)
{
AddToBackingStore(backingStore, MakeArrayKey(prefix, i), l[i]);
}
return;
}
// primitive
backingStore[prefix] = value;
}
private static object GetDeserializedObject(ControllerContext controllerContext)
{
if (!controllerContext.HttpContext.Request.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase))
{
// not JSON request
return null;
}
StreamReader reader = new StreamReader(controllerContext.HttpContext.Request.InputStream);
string bodyText = reader.ReadToEnd();
if (String.IsNullOrEmpty(bodyText))
{
// no JSON data
return null;
}
JavaScriptSerializer serializer = new JavaScriptSerializer();
serializer.MaxJsonLength = int.MaxValue; //increase MaxJsonLength. This could be read in from the web.config if you prefer
object jsonData = serializer.DeserializeObject(bodyText);
return jsonData;
}
public override IValueProvider GetValueProvider(ControllerContext controllerContext)
{
if (controllerContext == null)
{
throw new ArgumentNullException("controllerContext");
}
object jsonData = GetDeserializedObject(controllerContext);
if (jsonData == null)
{
return null;
}
Dictionary<string, object> backingStore = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
AddToBackingStore(backingStore, String.Empty, jsonData);
return new DictionaryValueProvider<object>(backingStore, CultureInfo.CurrentCulture);
}
private static string MakeArrayKey(string prefix, int index)
{
return prefix + "[" + index.ToString(CultureInfo.InvariantCulture) + "]";
}
private static string MakePropertyKey(string prefix, string propertyName)
{
return (String.IsNullOrEmpty(prefix)) ? propertyName : prefix + "." + propertyName;
}
}
JsonValueProviderFactory.cs
public sealed class JsonValueProviderFactory : ValueProviderFactory
{
private static void AddToBackingStore(Dictionary<string, object> backingStore, string prefix, object value)
{
IDictionary<string, object> d = value as IDictionary<string, object>;
if (d != null)
{
foreach (KeyValuePair<string, object> entry in d)
{
AddToBackingStore(backingStore, MakePropertyKey(prefix, entry.Key), entry.Value);
}
return;
}
IList l = value as IList;
if (l != null)
{
for (int i = 0; i < l.Count; i++)
{
AddToBackingStore(backingStore, MakeArrayKey(prefix, i), l[i]);
}
return;
}
// primitive
backingStore[prefix] = value;
}
private static object GetDeserializedObject(ControllerContext controllerContext)
{
if (!controllerContext.HttpContext.Request.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase))
{
// not JSON request
return null;
}
StreamReader reader = new StreamReader(controllerContext.HttpContext.Request.InputStream);
string bodyText = reader.ReadToEnd();
if (String.IsNullOrEmpty(bodyText))
{
// no JSON data
return null;
}
JavaScriptSerializer serializer = new JavaScriptSerializer();
serializer.MaxJsonLength = int.MaxValue; //increase MaxJsonLength. This could be read in from the web.config if you prefer
object jsonData = serializer.DeserializeObject(bodyText);
return jsonData;
}
public override IValueProvider GetValueProvider(ControllerContext controllerContext)
{
if (controllerContext == null)
{
throw new ArgumentNullException("controllerContext");
}
object jsonData = GetDeserializedObject(controllerContext);
if (jsonData == null)
{
return null;
}
Dictionary<string, object> backingStore = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
AddToBackingStore(backingStore, String.Empty, jsonData);
return new DictionaryValueProvider<object>(backingStore, CultureInfo.CurrentCulture);
}
private static string MakeArrayKey(string prefix, int index)
{
return prefix + "[" + index.ToString(CultureInfo.InvariantCulture) + "]";
}
private static string MakePropertyKey(string prefix, string propertyName)
{
return (String.IsNullOrEmpty(prefix)) ? propertyName : prefix + "." + propertyName;
}
}
by this you can pass lengthy json through ajax to controller and if you want to retrieve a lengthy string back to ajax result from controller then add below code in your controller also
//add this for getting large json string
protected override JsonResult Json(object data, string contentType, System.Text.Encoding contentEncoding, JsonRequestBehavior behavior)
{
return new JsonResult()
{
Data = data,
ContentType = contentType,
ContentEncoding = contentEncoding,
JsonRequestBehavior = behavior,
MaxJsonLength = Int32.MaxValue
};
}

Using XPages to get data from managed bean

I am trying to create a list of Twitter users, populating it with the number of followers for the user and their profile image. Because of Twitter's API, you need to get an access token for your application prior to using their REST API. I thought the best way to do this was via Java and a managed bean. I posted the code below, which currently works. I get the access token from Twitter, then make the API call to get the user info, which is in JSON.
My question is, what is the best way to parse the JSON and iterate over a list of user names to create a table/grid on the XPage?
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Serializable;
import java.io.UnsupportedEncodingException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLEncoder;
import javax.net.ssl.HttpsURLConnection;
import org.apache.commons.codec.binary.Base64;
import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
public class TwitterUser implements Serializable {
private static final String consumerKey = "xxxx";
private static final String consumerSecret = "xxxx";
private static final String twitterApiUrl = "https://api.twitter.com";
private static final long serialVersionUID = -2084825539627902622L;
private static String accessToken;
private String twitUser;
public TwitterUser() {
this.twitUser = null;
}
public String getTwitterUser(String screenName) {
try {
this.requestTwitterUserInfo(screenName);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return twitUser;
}
public void setTwitterUser() {
twitUser = twitUser;
}
//Encodes the consumer key and secret to create the basic authorization key
private static String encodeKeys(String consumerKey, String consumerSecret) {
try {
String encodedConsumerKey = URLEncoder.encode(consumerKey, "UTF-8");
String encodedConsumerSecret = URLEncoder.encode(consumerSecret, "UTF-8");
String fullKey = encodedConsumerKey + ":" + encodedConsumerSecret;
byte[] encodedBytes = Base64.encodeBase64(fullKey.getBytes());
return new String(encodedBytes);
}
catch (UnsupportedEncodingException e) {
return new String();
}
}
//Constructs the request for requesting a bearer token and returns that token as a string
private static void requestAccessToken() throws IOException {
HttpsURLConnection connection = null;
String endPointUrl = twitterApiUrl + "/oauth2/token";
String encodedCredentials = encodeKeys(consumerKey,consumerSecret);
String key = "";
try {
URL url = new URL(endPointUrl);
connection = (HttpsURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Host", "api.twitter.com");
connection.setRequestProperty("User-Agent", "Your Program Name");
connection.setRequestProperty("Authorization", "Basic " + encodedCredentials);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
connection.setRequestProperty("Content-Length", "29");
connection.setUseCaches(false);
writeRequest(connection, "grant_type=client_credentials");
// Parse the JSON response into a JSON mapped object to fetch fields from.
JSONObject obj = (JSONObject)JSONValue.parse(readResponse(connection));
if (obj != null) {
String tokenType = (String)obj.get("token_type");
String token = (String)obj.get("access_token");
accessToken = ((tokenType.equals("bearer")) && (token != null)) ? token : "";
}
else {
accessToken = null;
}
}
catch (MalformedURLException e) {
throw new IOException("Invalid endpoint URL specified.", e);
}
finally {
if (connection != null) {
connection.disconnect();
}
}
}
private void requestTwitterUserInfo(String sn) throws IOException {
HttpsURLConnection connection = null;
if (accessToken == null) {
requestAccessToken();
}
String count = "";
try {
URL url = new URL(twitterApiUrl + "/1.1/users/show.json?screen_name=" + sn);
connection = (HttpsURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod("GET");
connection.setRequestProperty("Host", "api.twitter.com");
connection.setRequestProperty("User-Agent", "Your Program Name");
connection.setRequestProperty("Authorization", "Bearer " + accessToken);
connection.setRequestProperty("Content-Type", "text/plain");
connection.setUseCaches(false);
}
catch (MalformedURLException e) {
throw new IOException("Invalid endpoint URL specified.", e);
}
finally {
if (connection != null) {
connection.disconnect();
}
}
twitUser = readResponse(connection);
}
//Writes a request to a connection
private static boolean writeRequest(HttpsURLConnection connection, String textBody) {
try {
BufferedWriter wr = new BufferedWriter(new OutputStreamWriter(connection.getOutputStream()));
wr.write(textBody);
wr.flush();
wr.close();
return true;
}
catch (IOException e) { return false; }
}
// Reads a response for a given connection and returns it as a string.
private static String readResponse(HttpsURLConnection connection) {
try {
StringBuilder str = new StringBuilder();
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = "";
while((line = br.readLine()) != null) {
str.append(line + System.getProperty("line.separator"));
}
return str.toString();
}
catch (IOException e) { return new String(); }
}
}
A few pointers:
Domino has the Apache HTTP client classes. They tend to be more robust than raw HTTP connections
Define a new class as a bean that contains all values that you want to see per row. You only need the getters public
add a method to your managed bean Collection getAllData()
bind that to a repeat control
you then can use repeatvar.someProperty in column values in EL
use better names than I just used

HTTP POST does not return expected JSON response

I have pasted a code snippet for HTTP Post where I am POSTING a multipart message to the server which needs Authentication. I am expecting a JSON response, but when I run this I always get the login page in HTML.
public final class MyScreen extends MainScreen {
private RichTextField _Output;
public MyScreen() {
// Set the displayed title of the screen
setTitle("MyTitle");
_Output = new RichTextField();
add(_Output);
addMenuItem(_GetDataAction);
}
protected MenuItem _GetDataAction = new MenuItem("GetData", 100000, 10) {
public void run() {
String URL = "<Sample URL Goes Here>";
ServiceRequestThread svc = new ServiceRequestThread(URL,
(MyScreen) UiApplication.getUiApplication()
.getActiveScreen());
svc.start();
}
};
public void updateDestination(final String text) {
UiApplication.getUiApplication().invokeLater(new Runnable() {
public void run() {
_Output.setText(text);
}
});
}
}
class ServiceRequestThread extends Thread {
protected String _URL;
protected MyScreen _Dest = null;
protected URLEncodedPostData _PostData = null;
StringBuffer writer = new StringBuffer();
public void setPOSTData(URLEncodedPostData data) {
_PostData = data;
}
public ServiceRequestThread(String URL, MyScreen screen) {
super();
_Dest = screen;
_URL = URL;
}
public void run() {
try
{
String boundary = "SATBA";
String twoHyphens = "--";
String data1 = "{\"IMPORTING\":{ \"IN_COUNTRY_CODE\":\"US\"}}";
String CRLF = "\r\n";
byte[] encoded = Base64OutputStream.encode
("User:password".getBytes(), 0, "User:password".length(), false,false);
"Prepare the data for post"
writer.append("--" + boundary).append(CRLF);
writer.append("Content-Disposition: form-data; name=\"param\"").append(
CRLF);
writer.append("Content-Type: text/json; charset=" + "UTF-8").append(CRLF);
writer.append("Content-Transfer-Encoding: 8bit").append(CRLF);
writer.append("Request-Id:Abcd123456" ).append(CRLF);
writer.append("Request-Type:rfc_json").append(CRLF);
writer.append("function:00163E0136C01EE0AE8B059433A71727")
.append(CRLF);
writer.append(CRLF);
writer.append(data1).append(CRLF);
writer.append("--" + boundary + "--").append(CRLF);
String string = new String(writer);
HttpConnection conn1 = (HttpConnection)Connector.open(_URL,Connector.READ_WRITE);
conn1.setRequestMethod(HttpConnection.POST);
conn1.setRequestProperty("Authorization", "Basic "+ new String(encoded));
conn1.setRequestProperty("Content-Type","multipart/mixed; boundary=" + boundary);
OutputStreamWriter osw = new OutputStreamWriter(conn1.openOutputStream(), "UTF-8");
osw.write(string);
osw.flush();
osw.close();
int responseCode = conn1.getResponseCode();
if (responseCode == HttpConnection.HTTP_OK) {
InputStream data = conn1.openInputStream();
StringBuffer raw = new StringBuffer();
byte[] buf = new byte[4096];
int nRead = data.read(buf);
while (nRead > 0) {
raw.append(new String(buf, 0, nRead));
nRead = data.read(buf);
}
_Dest.updateDestination(raw.toString());
} else {
_Dest.updateDestination("responseCode="
+ Integer.toString(responseCode));
}
}
catch( IOException e)
{
e.printStackTrace();
_Dest.updateDestination("Exception:"+e.toString());
}
}
}
Turns out the code was perfectly alright and the issue was on the rim.public property file where the application.handler.http.AuthenticationSupport was set to true and because of this it was not loggging in.
Now I set it to false and get the correct response.