How can I implement compression of a WCF RESTful (JSON) Service in C# with interoperabiility? - json

I have a WCF RESTful (i.e. JSON) service I'm building in C#. One of the DataContract methods can return a response that is very large, minimum 10 MB and maximum could be over 30 MB. It's all text and it returns it as JSON data to the client. When I test this method in the browser, I see it timing out. I understand there is a way to compress WCF RESTful service response data. Since interoperability is absolutely critical for my purposes, is it still possible to compress WCF RESTful service response data? Right now, I'm still testing the project on a local machine. I will be deploying it to IIS, however.
If there is a way to compress with interoperability, how can this be done?
Thank you.
This is not actually the set of files I'm using, but it's just a sample to show how I'm constructing my services. I realize this sample would not need compression at all.
IService1.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;
namespace WcfService4
{
// NOTE: You can use the "Rename" command on the "Refactor" menu to change the interface name "IService1" in both code and config file together.
[ServiceContract]
public interface IService1
{
[OperationContract]
[WebInvoke(
Method = "GET",
UriTemplate = "employees",
RequestFormat = WebMessageFormat.Json,
ResponseFormat = WebMessageFormat.Json,
BodyStyle = WebMessageBodyStyle.Bare)]
List<Employee> GetEmployees();
}
// Use a data contract as illustrated in the sample below to add composite types to service operations.
[DataContract]
public class Employee
{
[DataMember]
public string FirstName { get; set; }
[DataMember]
public string LastName { get; set; }
[DataMember]
public int Age { get; set; }
public Employee(string firstName, string lastName, int age)
{
this.FirstName = firstName;
this.LastName = lastName;
this.Age = age;
}
}
}
Service1.svc.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;
using System.Net;
namespace WcfService4
{
// NOTE: You can use the "Rename" command on the "Refactor" menu to change the class name "Service1" in code, svc and config file together.
public class Service1 : IService1
{
public List<Employee> GetEmployees()
{
// In reality, I'm calling the data from an external datasource, returning data to the client that exceeds 10 MB and can reach an upper limit of at least 30 MB.
List<Employee> employee = new List<Employee>();
employee.Add(new Employee("John", "Smith", 28));
employee.Add(new Employee("Jane", "Fonda", 42));
employee.Add(new Employee("Brett", "Hume", 56));
return employee;
}
}
}

you can change your web.config file to solve this problem.
change httpRuntime
<httpRuntime maxRequestLength="10240" executionTimeout="1000" />
here,
maxRequestLength: Indicates the maximum file upload size supported by ASP.NET. This limit can be used to prevent denial of service attacks caused by users posting large files to the server. The size specified is in kilobytes. The default is 4096 KB (4 MB).
executionTimeout: Indicates the maximum number of seconds that a request is allowed to execute before being automatically shut down by ASP.NET.

Related

.NET CORE 3.1 Razor Page calling API returns nothing on IIS Server returns data locally

I have a C# .Net Core 3.1 Razor Page where I am calling a 3rd Party API. When I run it locally in debug it returns a full JSON and is parsed into a strongly typed class successfully, but on the server hosting within IIS in process it fails with a 500 error that looks like an empty JSON object
Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0
I am using an AJAX call to a Razor handler method. The website is using oAuth2 to authenticate the user (this is an intranet application) and this part works as expected both locally and on the server. My suspicion is that the user is "authenticated" locally (through windows or some other method and that data is being sent to the API) but on the server the Claims are validated but not passed to the API. I have used Postman on the server to validate the API can be reached. I have also contacted the API host and verified the request is not failing from the API, its not even reaching it. It requires HTTPS on port 443 which I confirmed is open between the server and the API.
Here is the Razor page method (called from AJAX).
public async Task<JsonResult> OnGetPerson()
{
var userInfo = await whitePages.GetPersonByIDAsync(ID);
return new JsonResult(userInfo);
}
This calls a function within an Interface
public async Task<Models.PeopleAPI> GetPersonByIDAsync(string id)
{
string uriString = createGetPersonByidUriString(id); //This just builds the https string
var response = await _httpClient.GetAsync(uriString);
List<Models.PeopleAPI> people=JsonConvert.DeserializeObject<List<Models.PeopleAPI(await response.Content.ReadAsStringAsync());
return people.Single(); // use Single to guarantee only one result was receieved, otherwise throw an exception
}
My peopleAPI model is as follows.
public class PeopleAPI
{
public int PersonID { get; set; }
public string Domain { get; set; }
public string NTID { get; set; }
public string EmployeeID { get; set; }
public string UnixID { get; set; }
etc
etc
}
Please help, I have tried adding in user credentials to make the API call, various other Stack suggestions (5 to 10 different approaches, sync, async, etc) and just cannot seem to come to a solution (each one works locally, but not on the IIS server).
HELP!!!
I was able to solve my own issue using the visual studio remote debugging on my server. It turns out the local debug on my machine was not using the proxy server (System.Net.Http.HttpWindowsProxy was showing empty) but on the server it was using a different version (system.Net.Http.HttpEnvironmentProxy) and using the default proxy specified within netsh winhttp proxy. Once I set the startup.cs to specify the AddHttpClient I also included the useProxy = false. This was within an intranet and hope maybe someone else can use this in the future.
services.AddHttpClient("coolapi", c =>
{
c.BaseAddress = new Uri("https://something.com/");
c.DefaultRequestHeaders.Add("User-Agent", "HttpClientFactory-Sample");
}).ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler
{
UseProxy = false
}) ;

How to Bind Delta<T> in web api2 odata 3 controller

So I'm building out new dataservices, and figured I'd use web api odata. So I added a controller to my project using the scaffolding to generate actions for my entity framework model classes. Everything worked great until I tried the generated put or patch methods. The guid Id from the url binds, but no matter what I try I can't bind the Delta variable. It's always null. After a day of googling i can't find anything newer than about 2011 and those solutions don't work. Does anybody know how to get these to bind?
method signature
[AcceptVerbs("PATCH", "MERGE")]
public async Task<IHttpActionResult> Patch([FromODataUri] Guid key, Delta<AttachmentProposal> patch)
my web api config
public static void Register(HttpConfiguration config)
{
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
builder.EntitySet<AttachmentProposal>("AttachmentProposals");
builder.EntitySet<AttachmentAction>("AttachmentActions");
config.Routes.MapODataServiceRoute("odata", "odata", builder.GetEdmModel());
config.MapHttpAttributeRoutes();
my model is something like
public Guid Id { get; set; }
public string name { get; set; }
public DateTime createDate { get;set; }
public virtual HashSet<AttachmentActions> {get; set;}
if it makes any difference i always try to send up json. typical request body's i've tried are like
{ name: 'some name' }
or
{ every: 'value', single: 'value', property: 'value', on: 'value', my: 'value' model: 'value' }
figured out the answer by making a console app with a reference to the service and watching the traffic. If anybody else is having this problem try adding odata.type: "what ever the type of your object is" to the json in the request

How to validate double values in .Net using a custom validator?

I am developing an ASP.Net web api project and I want to validate my Server data model according to the JSON request I get from the Client side. In my Server Model Class, I have a double value and I am sending value from the Client Side as "12,14". I have written a custom validation class which is implemented by ValidationAttribute class of .Net and I am using IsValid(Object value) method to validate this user input.
So when I send my input as "12,14", .Net automatically converts this "12,14" to "1214" by thinking that "," is a group separator. But in this case, "," is not a group separator since this is a valid Double number for Norwaygian culture format ("no" culture).
public class Client : IClient
{
public string ClientId { get; set; }
public int EngagementId { get; set; }
[MyCustomDoubleType]
public double MyValue{ get; set; } //Notice that this is my double value to be validated.
}
This is the custom validator which I have written to validate "MyValue"
public class MyCustomDoubleTypeAttribute : ValidationAttribute
{
public override bool IsValid(object value) //When I send "12,14" from client, the value gets assigned to this value is "1214". .Net thinks "," is a group separator and removes it.
{
string doubleValue = value.ToString();
try
{
Thread.CurrentThread.CurrentCulture = new CultureInfo("no");
double convertedDouble = double.Parse(doubleValue);
string convertedString = convertedDouble.ToString(Thread.CurrentThread.CurrentCulture);
if (convertedString.Equals(doubleValue))
{
return true;
}
return false;
}
catch (FormatException formatException)
{
return false;
}
}
public override string FormatErrorMessage(string name)
{
return string.Format(CultureInfo.CurrentCulture,
ErrorMessageString,
name);
}
}
So this is my problem. What I want is to get the value as I enter in the client side to the input parameter of IsValid(Object value) method.
Thanks in advance.
You might be able to use a custom model binder. I know that you can use this on a regular MVC site, so I would assume that the same or similar code could be leveraged on Web API. There is a good example of using a custom binder for parsing double values at Phil Haack's site.
The culprit is probably Json.NET that's translating 12,14 into 1214 for you. You should look into writing a custom converter that is more careful about that.
You should be able to find instructions to write your own converter on the web. Here's an example:
How to implement custom JsonConverter in JSON.NET to deserialize a List of base class objects?

WCF RESTful web service with JSON.Net: avoid double serialization

Long time lurcher, first time poster here.
I've been grappling with this problem for a few days now and would appreciate any tips.
I break down the issue here below.
What I'm trying to achieve:
I want to set up a JSON WCF web service but I want to use the JSON.net serializer instead of the one that comes with WCF. Why? Because I found that serializing using the one that comes with WCF bloats up collections (I will show an example of what I mean below). The end consumer of the service is going to be a JavaScript client, so I don't want to have the client do extra messing around to navigate through the JSON.
I came up with a simple example of what I wanted and set out to try to make it work in C#. I must admit to not having used .NET in a long time, which perhaps contributes to my slowness. Anyway, I've got the patience in stock so on with it.
So, I want a service that accepts a username and returns a JSON object with some info on the user.
I envisaged the service to be called something like this:
http://someHostName/whoareyou?username=paul
And have the service respond with this:
{
"errorCode": 0,
"payLoad" : {
"userFullName": "Paul The Octopus",
"userLevel" : "Administrator"
}
}
I could use the above response's JSON object in JavaScript or PHP easily and feel really good about myself.
After some googling around I came across some posts suggesting this would be easy to do with WCF in .NET Framework 4. I had done some fiddling with WCF at a course a while ago but had long forgotten 99% of it so I found and followed this post to adapt to my goal:
http://www.codeproject.com/Articles/105273/Create-RESTful-WCF-Service-API-Step-By-Step-Guide#
Attempt take 1:
Following the above post I was able to set up a WCF Service (code set out below) that did what I wanted but the resulting JSON output was a bit bloated, as you can see below.
RestServiceImpl.svc.cs file
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
using Newtonsoft.Json;
using System.Collections.Specialized;
//Based on this psot: http://www.codeproject.com/Articles/105273/Create-RESTful-WCF-Service-API-Step-By-Step-Guide
namespace RestService
{
public class RestServiceImpl : IRestServiceImpl
{
#region IRestService Members
public WhoAreYouResponse whoareyou(string username)
{
var payLoad = new Dictionary<string, string>
{
{"userFullName", "Paul The Octopus"},
{"userLevel", "Administrator"}
};
WhoAreYouResponse whoAreYouResponse = new WhoAreYouResponse
{
errorCode = 0,
payLoad = payLoad
};
return whoAreYouResponse;
}
#endregion
}
//Helper bits to be used in service implementation
public class WhoAreYouResponse
{
public int errorCode { get; set; }
public Dictionary<string,string> payLoad { get; set; }
}
}
IRestServiceImpl.cs file
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;
namespace RestService
{
[ServiceContract]
public interface IRestServiceImpl
{
[OperationContract]
[WebInvoke(Method = "GET",
ResponseFormat = WebMessageFormat.Json,
BodyStyle = WebMessageBodyStyle.Bare,
UriTemplate = "json/whoareyou?username={username}")]
WhoAreYouResponse whoareyou(string username);
}
}
Web.config file
<?xml version="1.0"?>
<configuration>
<system.web>
<compilation debug="true" targetFramework="4.0" />
</system.web>
<system.serviceModel>
<services>
<service name="RestService.RestServiceImpl" behaviorConfiguration="ServiceBehavior">
<endpoint address="" binding="webHttpBinding" contract="RestService.IRestServiceImpl" behaviorConfiguration="web">
</endpoint>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="ServiceBehavior">
<!-- To avoid disclosing metadata information, set the value below to false and remove the metadata endpoint above before deployment -->
<serviceMetadata httpGetEnabled="true"/>
<!-- To receive exception details in faults for debugging purposes, set the value below to true. Set to false before deployment to avoid disclosing exception information -->
<serviceDebug includeExceptionDetailInFaults="false"/>
</behavior>
</serviceBehaviors>
<endpointBehaviors>
<behavior name="web">
<webHttp/>
</behavior>
</endpointBehaviors>
</behaviors>
<serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
</system.serviceModel>
<system.webServer>
<modules runAllManagedModulesForAllRequests="true"/>
</system.webServer>
</configuration>
*Result of calling the service through a browser like this http://192.168.0.84/TestRestService/RestServiceImpl.svc/json/whoareyou?username=paul*
{
"errorCode":0,
"payLoad" : [
{"Key":"userFullName", "Value":"Paul The Octopus"},
{"Key":"userLevel","Value":"Administrator"}
]
}
The hair in the soup in the JSON response is the way the Dictionary object has been mapped out in the JSON response to "payLoad". It's an array of objects, whereas I was expecting a JSON object sans this "Key", "Value" business. Not quite what I wanted. Close, but pretty finicky to handle at the client end. No one likes bloat and unnecessary work, so thumbs down to this.
Doing some more trolling around for solutions I read some SO posts suggesting what's happening here is that the service is using the built-in serializer
and that doesn't serialize "dictionaries in any other way". Some extra work would be needed to get it in the format I was expecting.
So, I thought, how about using another serializer?
Following this thought I found out about this bad boy here: http://james.newtonking.com/projects/json-net.aspx
This lead to my second attempt below.
Attempt take 2:
Downloading and importing the JSON.NET serializer into my little project, I altered the following files to look like this (changing the return type of the whoareyou method to string):
RestServiceImpl.svc.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
using Newtonsoft.Json;
using System.Collections.Specialized;
//Based on this psot: http://www.codeproject.com/Articles/105273/Create-RESTful-WCF-Service-API-Step-By-Step-Guide
namespace RestService
{
public class RestServiceImpl : IRestServiceImpl
{
#region IRestService Members
public string whoareyou(string username)
{
var payLoad = new Dictionary<string, string>
{
{"userFullName", "Paul The Octopus"},
{"userLevel", "Administrator"}
};
WhoAreYouResponse whoAreYouResponse = new WhoAreYouResponse
{
errorCode = 0,
payLoad = payLoad
};
return JsonConvert.SerializeObject(whoAreYouResponse);
}
#endregion
}
//Helper bits to be used in service implementation
public class WhoAreYouResponse
{
public int errorCode { get; set; }
public Dictionary<string,string> payLoad { get; set; }
}
}
IRestServiceImpl.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;
namespace RestService
{
[ServiceContract]
public interface IRestServiceImpl
{
[OperationContract]
[WebInvoke(Method = "GET",
ResponseFormat = WebMessageFormat.Json,
BodyStyle = WebMessageBodyStyle.Bare,
UriTemplate = "json/whoareyou?username={username}")]
string whoareyou(string username);
}
}
And when the service is called I get this response:
"{
\"errorCode\":0,
\"payLoad\":{
\"userFullName\":\"Paul The Octopus\",
\"userLevel\":\"Administrator\"}
}"
Winner! That's what I wanted and expected….BUT. Back the truck up. The entire JSON object is enclosed in double quotes!?
Hmmm. This is a string containing a JSON object. I took the hair out of the soup and now a fly's flown into it!
After a moment's head scratching it became obvious (I think) that what's happening is that the JSON.NET serializer is working like a charm, spitting out a JSON object in a string, but that is then put through the WCF serializer and is essentially stringified. So what I see in the return is a JSON string. So close! In fact, my method whoareyou says that it returns a string, so pretty much my fault.
So, my question is, how do I get this problem child to stop this double-serialization business? I can't find a return type for my whoareyou method to be something like JSON object.
Is there a way of telling WCF to use the JSON.NET serializer instead, or some such solution?
Pointers much appreciated.
As I understand you are creating service from scratch and there are no limitations on how REST service will be implemented. Then I suggest you to use ASP.NET WebApi
http://www.asp.net/web-api
Do not use legacy web services technologies because in newers a lot of boilerplate code is already done for you.
With web api you can easily replace or add any serializer/formatter. How to do it is described in following articles:
http://www.asp.net/web-api/overview/formats-and-model-binding/media-formatters
http://www.asp.net/web-api/overview/formats-and-model-binding/json-and-xml-serialization
I've experienced problems with serialization that you've described and solved them with this approach. From my experience on older web services technologies ( WCF 3.5 Rest Starter Kit,
Wcf 4 REST) it could be done in much harder way.

Share Json data between Asp.Net MVC 2 and Asp.Net server side C# code?

I created and love my Asp.Net MVC2 application. It's a very nice DDD app with Domain Model classes, View Model classes, a repository, and Json action methods to expose data.
My coworker wants to share my data with his Asp.Net Forms based C# code. He wants to pull through the Internet a class definition (like a Data Contract), then fill it with my Json results, effectively using something like a remote repository.
Any links or ideas on how to provide him with data contracts and data?
Darin Dimitrov had an excellent idea of consuming JSON data using data contracts here. Just wondering if it's possible to use MVC as the source for these items, then let him create the objects on his side, filled with data from my side.
The key to this question is how to send him my data classes, then send him my data.
class Program
{
[DataContract]
class Person
{
[DataMember(Name = "name")]
public string Name { get; set; }
[DataMember(Name = "surname")]
public string Surname { get; set; }
[DataMember(Name="age")]
public int Age { get; set; }
}
static void Main(string[] args)
{
var json = #"{""name"" : ""michael"", ""surname"" : ""brown"", ""age"" : ""35""}";
var serializer = new DataContractJsonSerializer(typeof(Person));
using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(json)))
{
var person = (Person)serializer.ReadObject(stream);
Console.WriteLine("Name : {0}, Surname : {1}, Age : {2}",
person.Name, person.Surname, person.Age);
}
}
}
Write an OData service. The format is JSON, but the tools to consume it easily -- from many languages -- are already written for you.
The nice thing about this is that your data is now not only consumable by your JS and your friend's ASP.NET app, it's consumable by Excel, PHP, etc.