Web Api, howto improve performance with ProtoBuf - json

It takes nearly 50 seconds to load a big chunk of 35 MB Json when accessing the Api. So to improve performance I added the WebApiContrib.Formatting.ProtoBuf to my project. The data is displayed in a Kendo UI Grid.
What am I missing here? A dataType or type in the View, or anything like that? And are there other or better ways to improve the performance?
Here some snippets->
POCO-Class:
[ProtoContract]
public partial class KDAuftraege
{
[ProtoMember(1)]
public int AngebotsNummer { get; set; }
[ProtoMember(2)]
public Nullable<int> BesuchsNummer { get; set; }
[ProtoMember(3)]
public Nullable<int> Kennummer { get; set; }
[ProtoMember(4)]
View:
var remoteDataSource = new kendo.data.DataSource({
transport: {
read: {
url: '/api/WebApiAuftraege'
},
},
Controller:
public IQueryable<KDAuftraege> GetKDAuftraeges()
{
//return db.KDAuftraeges.Take(500);
return db.KDAuftraeges;
}
WebApi:
config.Formatters.Add(new ProtoBufFormatter());
Headers:
Cache-Control no-cache
Content-Length 36227588
Content-Type application/json; charset=utf-8
Date Sat, 07 Jun 2014 09:23:54 GMT
Expires -1
Pragma no-cache
Server Microsoft-IIS/8.0
X-AspNet-Version 4.0.30319
X-Powered-By ASP.NET
X-SourceFiles =?UTF-8?B?YzpcdXNlcnNcb2xkc3BvcnRcZG9jdW1lbnRzXHZpc3VhbCBzdHVkaW8gMjAxM1xQcm9qZWN0c1xWaXM0XFZpczRcYXBpXFdlYkFwaUF1ZnRyYWVnZQ==?=
Anfrage-HeaderQuelltext anzeigen
Accept */*
Accept-Encoding gzip, deflate
Accept-Language de,en-US;q=0.7,en;q=0.3
Connection keep-alive
Cookie __RequestVerificationToken=wMVQWPOkXsB2XDIFN_07RJDtKqN_90dLRYaBYJGsFSGEHTcQ1S6e15mPiWrvkMHS8HrAlHYAI0OVSkqtPQHFVMP5DxoyccijSktL_KsoEFU1; .AspNet.ApplicationCookie=RTQ61CfArDWHlWN06eOpZiZY6NmFGp0SwCCuR8bQCtnItSz6S8YTasQu4-uoRQCc-XqWDCZmtOpEb-b0SyIioQPomkm1BrKywMcVwt3bF_JBxORKGg-UNSHyPvFyBohiS1sJ354LpRHIjrPIA8rUexvZih4VrK9lvHu_sm21ncNXXV7jATKAjTdX7J3XvfxRsF11fhgDNtpXPEWxQPjD7Rkj5yvdqI-vbfr9tfQbszUR1O3oOjYcRxUvvVrJ7xnt-caxt-o_Kut1dixLEA241pMGPCHfetWK73Yp148K3X9By6ylHFOTEjjDwHZyHLIrBwwOZ-ujnaOf20jQzeZXaF16bHxeadLYuKK-Z2DpdzaJXPzZd2pBbzHJMFX7USfZmp7OZzLpOitLCMovGHwdRiLD0F2NR1a0iTHCgiZLvA8
Host localhost:19275
Referer http://localhost:19275/MvcAuftraege
User-Agent Mozilla/5.0 (Windows NT 6.3; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
X-Requested-With XMLHttpRequest

Options 1: Change your view DataSource code (I think this is right)
var remoteDataSource = new kendo.data.DataSource({
transport: {
read: {
type: 'GET',
headers: { Accept: 'application/x-protobuf' },
url: '/api/WebApiAuftraege'
},
},
Option 2: Change your Web API Formatters so that ProtoBuf is the default:
config.Formatters.Insert(0, new ProtoBufFormatter());

Related

Save File And Json Data Via Angular and Spring boot

i have had this bug for a long peroid now, was trying to figure it out myself but am not getting it, it's an angular/spring boot app that am trying to upload different files and also save a json data at the same time, i have gone through previous questions concerning it but it hasn't worked for me, but when i tried saving the data via postman it works perfectly but not the same with angular, please i will appreciate the assistance...
this is my component.ts file containing the function
addBeat(artWork: File,untaggedmp3:File, taggedmp3:File){
const formData:FormData = new FormData();
if(artWork){
formData.append('artWork', artWork)
}
if(untaggedmp3){
formData.append('untaggedmp3', untaggedmp3)
}
if(taggedmp3){
formData.append('taggedmp3', taggedmp3)
}
this.beat.name = this.addBeatForm.get('name').value;
this.beat.price = this.addBeatForm.get('price').value;
this.beat.tempo = this.addBeatForm.get('tempo').value;
this.beat.description = this.addBeatForm.get('description').value;
this.beat.beatKey = this.addBeatForm.get('beatKey').value;
this.beat.mood = this.addBeatForm.get('mood').value;
this.beat.genre = this.addBeatForm.get('genre').value;
formData.append('beatDto', JSON.stringify(this.beat));
this.beatService.uploadBeat(formData)
.subscribe(
(data)=>{
console.log(data);
}
);
}
And my service.ts
public uploadBeat(formData: FormData): Observable<BeatDto>{
return this.httpClient.post<BeatDto>(`${this.apiServerUrl.admin}/api/beat/single/upload`, formData,
{headers: {
'Accept':'multipart/form-data',
'mimeType': 'multipart/form-data'}}
));
}
and my springboot rest controller is this
#RestController
#RequestMapping("/api/beat")
#AllArgsConstructor
#Slf4j
public class BeatController {
private final BeatService beatService;
private final FileStorageService fileStorageService;
#PostMapping(path = "/single/upload", consumes = {MediaType.APPLICATION_JSON_VALUE, MediaType.MULTIPART_FORM_DATA_VALUE})
public ResponseEntity<BeatDto> uploadBeat(#RequestParam("beatDto") String beatDto,
#RequestParam("artWork") MultipartFile artWork,
#RequestParam("taggedmp3") MultipartFile taggedmp3,
#RequestParam("untaggedmp3") MultipartFile untaggedmp3){
BeatDto beatJson = beatService.getJson(beatDto,artWork,taggedmp3,untaggedmp3);
return ResponseEntity.status(HttpStatus.CREATED).body(beatService.uploadBeat(beatJson,artWork,taggedmp3,untaggedmp3));
}
}
so when i test in on the browser i get a error code 400 on the browser console...
{
"headers": {
"normalizedNames": {},
"lazyUpdate": null
},
"status": 400,
"statusText": "OK",
"url": "http://localhost:8081/api/beat/single/upload",
"ok": false,
"name": "HttpErrorResponse",
"message": "Http failure response for http://localhost:8081/api/beat/single/upload: 400 OK",
"error": null
}
And also from my network tab
Request URL: http://localhost:8081/api/beat/single/upload
Request Method: POST
Status Code: 400
Remote Address: [::1]:8081
Referrer Policy: strict-origin-when-cross-origin
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost:4200
Access-Control-Expose-Headers: Origin, Content-Type, Accept, Authorization
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Connection: close
Content-Length: 0
Date: Thu, 06 Jan 2022 02:02:41 GMT
Expires: 0
Pragma: no-cache
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Accept: multipart/form-data
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Content-Length: 638
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryFUAjEynxdkQfpcfA
Host: localhost:8081
mimeType: multipart/form-data
Origin: http://localhost:4200
Referer: http://localhost:4200/
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
main.app
#SpringBootApplication
#EnableAsync
#Import(SwaggerConfiguration.class)
public class BasicApplication {
public static void main(String[] args) {
SpringApplication.run(BasicApplication.class, args);
}
#Bean
public CorsFilter corsFilter(){
CorsConfiguration corsConfiguration = new CorsConfiguration();
corsConfiguration.setAllowCredentials(true);
corsConfiguration.setAllowedOrigins(Arrays.asList("http://localhost:4200"));
corsConfiguration.setAllowedHeaders(Arrays.asList("Origin", "Access-Control-Allow-Origin", "Content-Type",
"Accept", "Authorization", "Origin, Accept", "X-Requested-With",
"Access-Control-Request-Method", "Access-Control-Request-Headers"));
corsConfiguration.setExposedHeaders(Arrays.asList("Origin", "Content-Type", "Accept", "Authorization",
"Access-Control-Allow-Origin", "Access-Control-Allow-Origin", "Access-Control-Allow-Credentials"));
corsConfiguration.setAllowedMethods(Arrays.asList("GET", "POST", "PUT", "DELETE", "OPTIONS"));
corsConfiguration.setMaxAge(3600L);
UrlBasedCorsConfigurationSource urlBasedCorsConfigurationSource = new UrlBasedCorsConfigurationSource();
urlBasedCorsConfigurationSource.registerCorsConfiguration("/**", corsConfiguration);
return new CorsFilter(urlBasedCorsConfigurationSource);
}
}
security config
#Override
public void configure(HttpSecurity httpSecurity) throws Exception {
httpSecurity.cors().and()
.csrf().disable()
.authorizeRequests()
.antMatchers("/resources/**", "/webjars/**", "/assests/**").permitAll()
.antMatchers("/v2/api-docs",
"/configuration/ui",
"/swagger-resources/**",
"/configuration/security",
"/swagger-ui.html")
.permitAll()
.antMatchers("/api/auth/**").permitAll()
.antMatchers(HttpMethod.GET,"/api/beat/**").permitAll()
.antMatchers(HttpMethod.POST,"/api/beat/**").permitAll()
.anyRequest()
.authenticated();
httpSecurity.addFilterBefore(jwtAuthFilter,
UsernamePasswordAuthenticationFilter.class);
}
Thanks In Advance

signin-oidc flow throws 500 in chrome but not IE

I have a netcoreapp2.1 webapp which is presenting an issue in Google Chrome, but not Edge or IE, during the OIDC flow redirect. It simply stops at http://localhost:5000/signin-oidc with this response received:
General:
Request URL: http://localhost:5000/signin-oidc
Request Method: POST
Status Code: 500 Internal Server Error
Remote Address: [::1]:5000
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Content-Length: 0
Date: Tue, 04 Aug 2020 09:49:32 GMT
Server: Kestrel
Request Headers:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 2148
Content-Type: application/x-www-form-urlencoded
Host: localhost:5000
Origin: null
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: cross-site
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
Form Data:
code: Ry9Pec...rdG1U0SB
scopes: resource.WRITE+openid+resource.READ
state: CfDJ8JZ...oESbUvQ
id_token: eyJhbG...9sXMEhs
When I visit localhost:5000 in chrome (incognito or not) the code never reaches the OnTicketReceived event, whereas when fired from IE, Edge etc it does, and proceeds just fine.
This is the startup class:
public class Startup
{
private AppSettings _appSettings;
private IConfiguration _config;
public Startup(IConfiguration configuration)
{
_config = configuration;
_appSettings = _config.Get<AppSettings>();
}
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
services.Configure<AppSettings>(_config);
services.AddSingleton<IAPIRepository, APIRepository>();
services.AddSingleton<IUserRepository, UserRepository>();
services.AddSingleton<INavigationRepository, NavigationRepository>();
services.UseOpenIDConnectMiddleware(new OpenIDConnectMiddlewareOptions
{
BaseUrl = _appSettings.API.BaseUrl,
AppName = _appSettings.AppName,
ClientId = _appSettings.API.ClientId,
ClientSecret = _appSettings.API.ClientSecret,
Secure = !_appSettings.Local
});
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
app.UseAuthentication();
if (!_appSettings.Local)
{
app.UseGlobalLoginMiddleware();
}
if (_appSettings.FeatureFlags["ProcessRedirectionRules"])
{
app.UseRedirectUserMiddleware(_appSettings.RedirectionRules);
}
app.UseMvc();
app.UseProtectHTMLRouteMiddleware();
if (_appSettings.Local)
{
app.UseDeveloperExceptionPage();
app.UseWebpackDevMiddleware(new WebpackDevMiddlewareOptions {
ProjectPath = Path.Join(Directory.GetCurrentDirectory(), "../UI"),
ConfigFile = "node_modules/#vue/cli-service/webpack.config.js",
HotModuleReplacement = true
});
}
else
{
app.UseDefaultFiles();
app.UseStaticFiles();
}
app.UseJavascriptVarMiddleware(new JavascriptSettingsMiddlewareOptions
{
FileName = "env.js",
ObjectName = "__env",
Settings = new Dictionary<string, string>
{
{ "insightsKey", _appSettings.ApplicationInsights.InstrumentationKey },
{ "environment", _appSettings.Environment },
{ "gatekeeperBaseUrl", _appSettings.Gatekeeper.BaseUrl }
}
});
app.UseGlobalSignoutMiddleware(new GlobalSignoutMiddlewareOptions
{
GatekeeperLogoutUrl = _appSettings.API.LogoutUrl
});
app.UseHTMLRouteFallback(new HTMLRouteFallbackMiddlewareOptions
{
Local = _appSettings.Local,
Path = Path.Join(Directory.GetCurrentDirectory(), "../UI/dist")
});
}
}
I have cleaned the localhost and localhost:5000 cookies from the browser.
A quick guess would be that it is a SameSite cookie issue that different browsers handle differently due to various bugs.
This article might give you a starting point:
How To Prepare Your IdentityServer For Chrome's SameSite Cookie Changes - And How To Deal With Safari, Nevertheless
Also, your Configure method looks very odd. Because the ordering of the App.Use statements matter and usually you would put the app.UseMvc(); last in that method. Each incoming request will pass through those middlewares that the App.UseXXX adds and if the ordering is not correct then you will have issues.

Cannot get data with AngularJS + Azure WCF

I'm trying to get data from my website using AngularJS to my wcf in Azure but i'm getting error.
In my js:
$http({
url: dataService.serverPath + 'Services.svc/Login',
method: 'GET',
headers: { 'Accept': 'application/json, text/plain',
'Content-Transfer-Encoding': 'application/json;charset=utf-8',
'Accept-Charset': "charset=utf-8",
},
params: { key: key }
}).success(function (data, status, headers, config) {
//some code here
}
}).error(function (data, status, headers, config) {
$scope.hasError = true;
$scope.errorMsg = "The request failed: " + data;
});
fiddler response:
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/json; charset=utf-8
Vary: Accept-Encoding
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 11 Aug 2015 22:17:09 GMT
Transfer-Encoding: chunked
49
{"ID":0,"NomeUsuario":"Admin","IsAdmin":true,"Error":null}
0
Note, i'm getting HTTP 200 and i'm getting error.
The return data is 49 { Json } 0, what is it?
I tried to remove the Transfer-Encoding: chunked from my WCF but no success.
I spend this hole day trying getting data from Azure and i didnt. Running on my dev machine, it's fine and does not show Transfer-Encoding : Chuncked nor the 49 and 0 enclosure my JSON.
first thing that came to my mind was the MIME type as it needs to be changed by FTPing to your Website in Azure and updating the WebConfig file. But this usually returns 404 as in your case it returns 200, still no harm to check.
My other suggestion would be about the WebInvoke inside your WCF (if you are using it) following piece should set your WCF for Json usage.
[WebInvoke(UriTemplate = "Your template here",
Method = "GET",
RequestFormat = WebMessageFormat.Json,
ResponseFormat = WebmessageFormat.Json,
BodyStyle = WebMessageBodyStyle.Wrapped)]
Hope this helps

Web api return null value

I have an Web Api controller for access data from server:
public class ServicesController : ApiController
{
[AcceptVerbs("POST")]
public IList<TreeMenuItem> LoadMetadata()
{
List<TreeMenuItem> itemsMenu = new List<TreeMenuItem>();
TreeMenuItem dataSource = new TreeMenuItem("1", "DataSources", null);
itemsMenu.Add(dataSource);
return itemsMenu;
}
}
which is call by an angularJS controler:
angular.module('App').
controller('TreeMenuController', ["$scope", "$http", treeMenuController]);
function treeMenuController($scope, $http) {
var baseUrl = "api/Services/LoadMetadata";
$http.post(baseUrl)
.then(function (result) {
$scope.roleList = result.data;
});
};
In browser network I have:
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate
Content-Length:2
Content-Type:application/json;charset=UTF-8
Request Payload
{}
Response Headers:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Content-Length: 4
in Response tab: [{}].
What is wrong?
I make it work:
The big help was the response message when I put in browser the address for accessing api services (api/Services/LoadMetadata):
The error answer was in an xml file where I found that was problem with serialization of the object TreeMenuItem. The advice was to decorate with DataContract the class and DataMember the class properties - like in WCF. After I did that (was need to add reference in project to System.Runtime.Serialization), everything was perfect.

Posting to a WCF Data Service 5

I am trying to POST to a EntityFramework backed WCF Data Service but I am getting:
415 Unsupported Media Type
I followed this guide to put the headers into my jQuery POST http://blogs.msdn.com/b/astoriateam/archive/2012/04/11/what-happened-to-application-json-in-wcf-ds-5-0.aspx
These are my headers:
POST /webservices/service/service.svc/Activities HTTP/1.1
Host: www.url.com
Connection: keep-alive
Content-Length: 138
Origin: http://www.url.com
X-Requested-With: XMLHttpRequest
MaxDataServiceVersion: 3.0
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.56 Safari/536.5
Content-Type: application/json; charset=UTF-8
Accept: application/json;odata=verbose
Referer: http://www.url.com/sites/site/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,en-GB;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
This site helped with using EntityFramework with WCF Data Services - http://blogs.msdn.com/b/writingdata_services/archive/2011/06/15/entity-framework-4-1-code-first-and-wcf-data-services.aspx
function AddActivity() {
var activity = {
activity:
{
"Title": "Test From Code",
"Detail": "Code Example",
"Started": "2012-06-21T09:00:00",
"UserId": 17
}
};
var url = 'http://www.url.com/webservices/service/service.svc/Activities';
$.ajax({
type: "POST",
url: url,
data: activity,
dataType: "json",
contentType: "application/json; charset=utf-8",
beforeSend: function (xhr) {
xhr.setRequestHeader("Accept", "application/json;odata=verbose");
xhr.setRequestHeader("MaxDataServiceVersion", "3.0");
},
success: function (data) {
alert('Success');
},
error: function (err) {
alert('Fail\n' + err.statusText);
}
});
}
I can read data JSON data OK
WCF Data Services 5.0 does not support application/json content type for request (or response) payloads if the payload has version V3 (DataServiceVersion). In the above case you didn't specify the DataServiceVersion (which you always should by the way). The server in such case has to infer the version somehow, the WCF Data Service server will assume the maximum version it and the client understands. Since the server itself understands V3 and you specified MaxDataServiceVersion: 3.0 which means the client also understands V3 it assumes the payload is a V3 payload.
V3 payloads currently don't support application/json, they only support application/json;odata=verbose.
See this blog post for more explanation: http://blogs.msdn.com/b/astoriateam/archive/2012/04/11/what-happened-to-application-json-in-wcf-ds-5-0.aspx. It talks about GET but the same things applies to POST and such as well.
So to fix your problem, modify your client to either send DataServiceVersion: 2.0 (or 1.0) if that would be correct, or even better modify it to send Content-Type: application/json;odata=verbose (which will work no matter the payload version).
Just did this with minimal hacking, using WCF without any WCF Data Services components.
Posting a minimal type back as JSON will automap the values, for example, this JSON
{
"name": "1234567890",
"amount": "2500.00"
}
can post back to this service
[WebInvoke(Method = "POST", ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare, RequestFormat = WebMessageFormat.Json)]
public string MethodName(Message message)
where Message is C# class:
public class Message
{
public string name { get; set; }
public string amount { get; set; }
}
Pretty simple. Then, if you want to process it as an OData request, you have more complex JSON being posted, example (ODataV2):
{
"d": [
{
"__metadata": {
"id": "http://host:54470/ods.svc/MethodName('1234567890')",
"uri": "http://host:54470/ods.svc/MethodName('1234567890')",
"type": "SomeCompany.Helper.Services.RequestMessage.Message"
},
"name": "1234567890",
"amount": "2500.00",
}
]
}
If you create corresponding C# classes, this OData will map to those classes. These classes work with the OData.
public class Message
{
public ODataResult[] d;
}
public class ODataResult
{
public ODataMetaData __metadata;
public string name { get; set; }
public string amount { get; set; }
}
public class ODataMetaData
{
public string id;
public string uri;
public string type;
}
The json posted OData will map to this Message class. This gives you a way to 'shape' the incoming message as needed, which might be required in some cases.
Not saying this is better than using WCF Data Services, just saying it will work.