While developing an Angular 7 web app, I am using the proxy.conf.json file and --proxy-config option on the ng serve command to proxy my requests from https://localhost:4200/api to a remote url which hosts the actual REST API.
When doing an HTTP GET request directly to the remote url through my browser (Chrome) or Postman, I always get the full JSON response like I expect it to be.
When requesting this on the angular proxy url, I get only a part of the JSON response (most of the time - not always).
I am aware that the API returns response header Transfer-encoding 'chunked' and from what I understand, this means having to process the response as a stream.
One way to avoid this, is to let the API return a Content-Length header. But I'd rather want to leave the API unchanged.
I read that when proxying on an nginx server, there is an option 'proxy_buffering' to configure how to handle large HTTP responses. But I'm using the built-in angular proxy (which is a webpack dev server) and I haven't found a way to configure such property.
proxy.conf.json
...
"/api/messages": {
"target": "http://myRemoteHost:myRemotePort/myRemoteContext/v1/",
"secure": false,
"changeOrigin": true,
"logLevel": "debug",
"pathRewrite": {
"^/api": ""
}
},
...
messages.service.ts
return this.http.get<Message[]>(this.endpoint, {params: httpParams});
Error in console
Error Code: 200
Message: Http failure during parsing for http://...
HttpErrorResponse:
error: SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>)
at XMLHttpRequest.onLoad (https://localhost:4200/hal/vendor.js:32570:51)
at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invokeTask (https://localhost:4200/hal/polyfills.js:2768:31)
at Object.onInvokeTask (https://localhost:4200/hal/vendor.js:78088:33) at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invokeTask (https://localhost:4200/hal/polyfills.js:2767:60)
at Zone.push../node_modules/zone.js/dist/zone.js.Zone.runTask (https://localhost:4200/hal/polyfills.js:2540:47)
at ZoneTask.push../node_modules/zone.js/dist/zone.js.ZoneTask.invokeTask [as invoke] (https://localhost:4200/hal/polyfills.js:2843:34)
at invokeTask (https://localhost:4200/hal/polyfills.js:4089:14)
at XMLHttpRequest.globalZoneAwareCallback (https://localhost:4200/hal/polyfills.js:4126:21)
message: "Unexpected end of JSON input"
I was expecting that when using the standard HttpClient from Angular, I always would get the full response (also for large HTTP responses). This does not seem to be the case. My gut feeling says the proxy is causing this.
So do I need to specifically change my code in order to process chunked http responses?
Or is this (hopefully configurable) behaviour from the webpack server?
Does anyone know a possible solution?
One way to avoid this, is to let the API return a Content-Length header. But I'd rather want to leave the API unchanged.
This worked for us, our customers had proxy servers for security, and 25% of our JSON requests were broken because Transfer-Encoding was chunked. We had to remove compression for the time being.
Thank you kind stranger for posting this question.
I had a similar problem using angular 6 and running the local server (with ng --serve) with node.js 8.10.
I resolved it by upgrading node.js to 12.13.0
Got around this by adding
"headers": {
"Connection": "keep-alive"
}
inside the proxy item declaration in proxy.conf.json, as follows:
"/api/messages": {
"target": "http://myRemoteHost:myRemotePort/myRemoteContext/v1/",
"secure": false,
"changeOrigin": true,
"logLevel": "debug",
"headers": {
"Connection": "keep-alive"
}
}
Adding this, I managed to retrieve quite large datasets without the JSON response getting cut off.
Hope this can help someone, this has been haunting me for the past few days.
Related
I'm having trouble getting the results for [GET projects/:projectId/diffs/:diffId/properties] from the new Model Properties API provided by Autodesk Forge.
I've been able to get all the Model Properties API calls (both Index and Diff) to work on Postman. However, on a NodeJS server (using axios) the properties call doesn't work, and returns a 502 error.
Below is an example code:
const result = await axios.get(
`https://developer.api.autodesk.com/construction/index/v2/projects/${projectId}/diffs/${diffId}/properties`,
{
headers: {
Authorization: `Bearer ${internalToken.access_token}`,
},
},
);
If I replace properties with either manifest or fields, the results are returned normally. This applies to both Index and Diff.
I'm not sure if this is actually an internal server error (which is the error returned), or is there something wrong in the request that is being sent?
Note: if it makes any difference, it is a BIM360 project.
Update: After some further usage, it seems that the error 502 with status of 'Bad Gateway' occurs when the response is large. Responses above ~14MB return this error. This is confirmed because the results return normally on Postman.
Could anyone from Autodesk help me with this? And if there are any alternatives to fixing this issue.
In case somebody runs into the same problem, it was solved by adding 'Accept-Encoding': 'gzip, deflate, br' to the header in the request.
I'm sending two post requests using the REST API: http://localhost:8111/app/rest/buildQueue but my second request fails with
403 Forbidden: Responding with 403 status code due to failed CSRF check: no "Origin" header is present and no authentication provided with the request, consider adding "Origin: http://localhost:8111" header.
I'm wondering why is this happening since if I run the build in the UI and change the params ex. build1 has %version=2% and build2 has %version=3% it will run parallel with each other running on different available agents.
Here's my json request:
REST API endpoint: http://localhost:8111/app/rest/buildQueue
JSON body:
{
"branchName": "master",
"buildType": {
"id": "DockerBuild",
"projectId": "Test"
},
"properties": {
"property": [
{
"name": "DOCKER_IMAGE_NAME",
"value": "test-3"
},
{
"name": "SNAPSHOT_DEPENDENCY_VERSION",
"value": "0.6"
}
]
}
}
Am I missing a parameter to be able to run builds in parallel with each other?
When you face problems regarding CSRF protection in TeamCity (for example, you get the "Responding with 403 status code due to failed CSRF check" response from the server), you can follow these steps:
If you use a reverse proxy, make sure you correctly configure Host/Origin headers, as described above. In the meantime, you may want to add the public URL of your server to CORS-enabled origins.
You can temporary disable CSRF protection at all by setting the teamcity.csrf.origin.check.enabled=logOnly internal property.
Information about failed CSRF attempts are logged into TeamCity/logs/teamcity-auth.log files. For more detailed diagnostics of the requests, enable debug-auth logging preset.
Try pass in the request header -H 'Origin: http://localhost:8111'
Maybe this can be useful for someone, I got the same error with a single POST using Postman:
403 Forbidden: Responding with 403 status code due to failed CSRF check: no "Origin" header is present and no authentication provided with the request, consider adding "Origin: http://teamcity:20011" header.
So I followed the recommendation of the error message, and in Header I added "Origin" with the value "http://teamcity:20011" and that fixed the issue. BTW, in Authorization I selected "Bearer Token" and I pasted the token generated previously through TeamCity. This is the call:
http://teamcity:20011/app/rest/buildQueue
I was just testing how to trigger a build using the API and it worked successfully. Now the following step is to implement this call using JavaScript.
Request a CSRF header with the appropriate request:
https://teamcity/authenticationTest.html?csrf
and set it in the "X-TC-CSRF-TOKEN" header of your POST request
If you specify an Access Token to the request header like Authorization: Bearer ..., you don't need to specify a CSRF token, and what you should actually check is if you're not sending Cookies.
This is from the developer in JetBrains:
If you're using a token-based authentication, there should be no need to provide CSRF token header and obtain it with authenticationTest.html call.
In this scenario, it is expected that there are no session Cookies in the HTTP request (otherwise, TeamCity will try to find a token).
I.e. basically, you should be able to do the HTTP call in no-session way by providing the Authorization: Bearer {{token}} header only.
https://youtrack.jetbrains.com/issue/TW-69566/Flaky-builds-with-CSRF-Header-X-TC-CSRF-Token-does-not-match-CSRF-session-value#focus=Comments-27-4644138.0-0
Well, the error and the documentation don't seem to explain this, though...
Need to fetch the build values from apache.org. So i am using their api
https://builds.apache.org/api/json
I tried angularjs $http.jsonp but not able to fetch the data.
In chrome console under network json api is getting loaded but the data is not getting returned instead it is throwing the response as error.
app.controller("jsoncontroller",function($scope,$http){
var url='https://builds.apache.org/api/json';
$http.jsonp(url).success(function(data){
console.log('success');
})
.error(function () {
console.log('error')
});
});
Getting the error as
Uncaught SyntaxError: Unexpected token :
error
As per the jsonp angular docs, you must append JSON_CALLBACK to the URL: https://builds.apache.org/api/json?jsonp=JSON_CALLBACK
However, that URL doesn't work because even when the callback parameter is specified, the server still sends back a content-type of application/json, instead of the expected application/javascript. This causes it to be parsed (evidently) by the json parser instead of the javascript callback needed for JSONP to work. I'm not versed enough in JSONP or Angular to know who is it fault here.
I've made a fiddle with this working with another URL.
[Update]: The apache build server appears to use Jenkins, which has disable JSONP from the remote API. You can verify this yourself by trying to hit their jsonp endpoint, which returns a 403. You'll have to use another endpoint, no way I can see around this.
In my Ruby on Rails application I tried to upload an image through the POSTMAN REST client in Base64 format. When I POST the image I am getting a 406 Not Acceptable Response. When I checked my database, the image was there and was successfully saved.
What is the reason for this error, is there anything I need to specify in my header?
My request:
URL --- http://localhost:3000/exercises.json
Header:
Content-Type - application/json
Raw data:
{
"exercise": {
"subbodypart_ids": [
"1",
"2"
],
"name": "Exercise14"
},
"image_file_name": "Pressurebar Above.jpg",
"image":"******base64 Format*******"
}
Your operation did not fail.
Your backend service is saying that the response type it is returning is not provided in the Accept HTTP header in your Client request.
Ref: http://en.wikipedia.org/wiki/List_of_HTTP_header_fields
Find out the response (content type) returned by Service.
Provide this (content type) in your request Accept header.
http://en.wikipedia.org/wiki/HTTP_status_code -> 406
406 Not Acceptable
The resource identified by the request is only capable of generating response entities which have content characteristics not
acceptable according to the accept headers sent in the request.
406 happens when the server cannot respond with the accept-header specified in the request.
In your case it seems application/json for the response may not be acceptable to the server.
You mentioned you're using Ruby on Rails as a backend. You didn't post the code for the relevant method, but my guess is that it looks something like this:
def create
post = Post.create params[:post]
respond_to do |format|
format.json { render :json => post }
end
end
Change it to:
def create
post = Post.create params[:post])
render :json => post
end
And it will solve your problem. It worked for me :)
"Sometimes" this can mean that the server had an internal error, and wanted to respond with an error message (ex: 500 with JSON payload) but since the request headers didn't say it accepted JSON, it returns a 406 instead. Go figure. (in this case: spring boot webapp).
In which case, your operation did fail. But the failure message was obscured by another.
You can also receive a 406 response when invalid cookies are stored or referenced in the browser - for example, when running a Rails server in Dev mode locally.
If you happened to run two different projects on the same port, the browser might reference a cookie from a different localhost session.
This has happened to me...tripped me up for a minute. Looking in browser > Developer Mode > Network showed it.
const request = require('request');
const headers = {
'Accept': '*/*',
'User-Agent': 'request',
};
const options = {
url: "https://example.com/users/6",
headers: headers
};
request.get(options, (error, response, body) => {
console.log(response.body);
});
Changing header to Accept: */* resolved my issue and make sure you don't have any other Accept Header
In my case, I added:
Content-Type: application/x-www-form-urlencoded
solved my problem completely.
If you are using 'request.js' you might use the following:
var options = {
url: 'localhost',
method: 'GET',
headers:{
Accept: '*/*'
}
}
request(options, function (error, response, body) {
...
})
In my case for a API in .NET-Core, the api is set to work with XML (by default is set to response with JSON), so I add this annotation in my Controller :
[Produces("application/xml")]
public class MyController : ControllerBase {...}
Thank you for putting me on the path !
It could also be due to a firewall blocking the request. In my case the request payload contained string properties - "like %abc%" and ampersand symbol "&" - which caused the firewall to think it is a security risk (eg. a sql injection attack) and it blocked the request. Note here the request does not actually go to the server but is returned at the firewall level itself.
In my case, there were no application server logs generated so I knew that the request did not actually reach the server and was blocked before that. The logs that helped me were Web application firewall (WAF) logs.
I'm using this plugin:
http://loopj.com/jquery-tokeninput/
Initialized using
$("#actors").tokenInput("/Services/AutoComplete.asmx/AdminNames", {
method: "post",
theme: "facebook"
});
On typing in the input, I get this error in the NET tab of Firebug
500 Internal Server Error
Post - q=a
Response - {"Message":"Invalid JSON primitive: q.","StackTrace":" ...
JSON - "Invalid JSON primitive: q."
If I use a variable instead of Web Service, it works fine. I don't even get the debug control of the web service.
Thanks,
Dev