how to fetch large json from a post in angular6/7 - json

I have migrated a piece of code to be able to export data as excel file in angular.
I assume the fact that the json is well formed and send from the server to the angular side. I can see it in the network frame in th browser.
For small json, it's ok but when the size of the json starts to be large, the answer still failed.
This following code corresponding to the service call
exportSynthesis(recordId: number, moduleId: number) {
const body = null;
return this.http.post(this.apiUrl + `/data`
+ `${recordId}/module/${moduleId}`, body,
{
headers: new HttpHeaders({ 'Content-Type': 'application/json' }),
observe: 'response', responseType: 'json' }).pipe(
map((resp: any) => {
return resp.body;
}));
}
and here, its the method which manages the return.
exportSynthesis() {
this.service.exportSynthesis(this.recordId, this.moduleId)
.subscribe(
(exportResult) => { this.exportResult = exportResult; },
err => {
console.log('err:', err);
this.errorHandlerService.handleError('failed', err);
},
() => {
console.log('json:', this.exportResult);
const worksheet: XLSX.WorkSheet = XLSX.utils.json_to_sheet(this.exportResult);
const workbook: XLSX.WorkBook = { Sheets: { 'data': worksheet }, SheetNames: ['data'] };
const excelBuffer: any = XLSX.write(workbook, { bookType: 'xlsx', type: 'array' });
const blob = new Blob([excelBuffer], { type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;charset=UTF-8' });
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = '(GEO) ' + this.record.label + ' - name.xlsx';
a.click();
window.URL.revokeObjectURL(url);
a.remove();
});
}
Currently, i do not manage to understand why it still finish in error and I get only "ok" in the console log.
Any idea?
regards

Angular's HttpClientModule default response is a json.
Your problem is that you try to access the body property of the HTTP response, but Angular interprets that as you trying to access the body property in the body of the response.
Remove observe and responseType from your post request and treat the response as a json. It should work.

find:
just need to use text as json
return this.http.post(this.apiUrl + `/geo/v1/synthesis/xls/record/`
+ `${recordId}/module/${moduleId}`, body,
{
headers: headers,
observe: 'response',
responseType: 'text' as 'json'}).
map((resp: any) => {
return resp.body;
});
}

Related

API returns index.ejs HTML template instead of JSON

When trying to bring a news/posts list from a common WordPress RSS Feed (Rest API), it ends up returning my index.ejs webpack template in my app instead of the JSON response.
Using an endpoint like: example.com/wp-json/wp/v2/posts, the expected response would be a JSON "string" with a certain number of posts,
and instead it's returning a text/html response, which happens to be the html template for my app, nothing unkown.
The code in JS, React being used is:
import axios from 'axios';
const getPosts = async () => {
const postsEndpoint = '/wp-json/wp/v2/posts';
const config = {
headers: {
'accept': 'text/html, application/json',
'Content-Type': 'application/json; charset=UTF-8',
},
params: {
_limit: 5,
}
};
let response;
try {
response = await axios.get(postsEndpoint, config);
} catch (err) {
console.log(err);
} finally {
// eslint-disable-next-line no-unsafe-finally
return response.json;
}
};
export default getPosts;
And the code to show this in display:
const blogNews = () => {
const [isLoading, setIsLoading] = useState(false);
const [posts, setPosts] = useState(null);
const getPostsData = async () => {
try {
setIsLoading(true);
const data = await getPosts();
console.log(data);
if (data.status === 200) {
setPosts(data);
setIsLoading(false);
}
} catch (err) {
console.log(err);
} finally {
setIsLoading(false);
}
};
useEffect(() => {
getPostsData();
}, []);
return ...
And the nginx configuration lines looks like this:
location /wp-json/wp/v2/ {
proxy_pass http://example.com/;
}
I'm using 'text/html' in the Accept header just to see what's in the response, if I only include 'application/json' it just returns a 404 error.
The fact that I'm not including Cors nor CSP headers it's because it's because the policy criteria is being met with the nginx configuration.
Do you guys have any recommendations regarding this? Thanks in advance.

How can I read http errors when responseType is blob in Axios with VueJs? [duplicate]

This question already has answers here:
How can I get the status code from an HTTP error in Axios?
(15 answers)
Closed 7 days ago.
I'm using blob responseType with Axios in my VueJS app for downloading a document from the server. When the response code is 200 it works fine and download the file but when there is any http error, I'm not able to read the status code when I catch the error because the error is a JSON response.
Has anyone had a similar issue and worked out a way to convert the blob response type to json and thrown an error based on the status code?
I have tried sending the response as a plain text from Laravel backend and tried converting the response to JSON or text in the front-end but no luck.
I have tried reading error response headers but no luck.
Axios({
url: 'xxxx',
method: 'GET',
responseType: 'blob',
})
.then((response) => {
//code to read the response and create object url with the blob and download the document
})
.catch((error) => {
console.log('Error', error.message); //nothing
console.log('Error', error.error); //undefined
console.log('Error', error.data); //undefined
const blb = new Blob([error], {type: "text/plain"});
const reader = new FileReader();
// This fires after the blob has been read/loaded.
reader.addEventListener('loadend', (e) => {
const text = e.srcElement.result;
console.log(text);
});
// Start reading the blob as text.
reader.readAsText(blb);
});
I just want to throw the error message based on the status code. If it's 401 just want it to be unauthorized and anything else throw it on to the component.
The reason is that the response type is blob.
In case of error, the status code is available directly in your exception object. However, the response is a promise.
What you need to do is:
.catch((error) => {
let statusCode = error.response.status
let responseObj = await error.response.data.text();
:
:
For more details you can read documentation.
You can do this, it cast the error in JSON if it's a blob or let the response data as it received and you can do work with it.
let errorString = error.response.data;
if (
error.request.responseType === 'blob' &&
error.response.data instanceof Blob &&
error.response.data.type &&
error.response.data.type.toLowerCase().indexOf('json') != -1
) {
errorString = JSON.parse(await error.response.data.text());
}
alert(errorString);
You need to convert the response blob to json:
Axios({
url: 'xxxx',
method: 'GET',
responseType: 'blob',
})
.then((response) => {
//code to read the response and create object url with the blob and download the document
})
.catch((error) => {
if (
error.request.responseType === 'blob' &&
error.response.data instanceof Blob &&
error.response.data.type &&
error.response.data.type.toLowerCase().indexOf('json') != -1
) {
new Promise((resolve, reject) => {
let reader = new FileReader();
reader.onload = () => {
error.response.data = JSON.parse(reader.result);
resolve(Promise.reject(error));
};
reader.onerror = () => {
reject(error);
};
reader.readAsText(error.response.data);
})
.then(err => {
// here your response comes
console.log(err.response.data)
})
};
});
You can use this inside interceptor.
For more info
I believe you might be using the error variable in your catch() incorrectly.
Axios passes an error object that has a response property that is where you will want to look for your error or message.
https://github.com/axios/axios#handling-errors
On a side note if you can catch the error server side you could try setting the Content-type header to text/plain. Using either header('Content-Type: plain/text') or Laravel's Response Methods
you can do the following way
axios.post("URL", data, {
validateStatus: (s) => s <= 500,
responseType: 'blob',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/pdf'
}
})
.then(async (response) => {
if (response.status !== 200) {
// error handling
const error = JSON.parse(await response.data.text());
console.log('error: ', error);
alert(error.message);
} else {
console.log('res', response)
const url = window.URL.createObjectURL(new Blob([response.data]));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', 'file.pdf'); //or any other extension
document.body.appendChild(link);
link.click();
}
})
You can convert blob response globaly:
$axios.onError(async ({request, response}) => {
const status = response.status
let data = response.data
if (request.responseType === 'blob' && data instanceof Blob && data.type) {
const text = await data.text()
data = data.type.toLowerCase().includes('json') ? JSON.parse(text) : text
}
throw {status, data} // error model
})
catch(async(error) => {
const errorJson = JSON.parse(await error.response.data.text());
console.log('error: ', errorJson.error_message);
})

Angular 6 - get csv response from httpClient

Can any one please give an example of fetching application/octet-stream response from angular 6 httpClient. I am using the below code and it doesn't work ( I get unknown error - 401 response) -
import { saveAs } from 'file-saver';
getJobOutput() {
this.workflowService.fetchOutput(this.jobId,this.outputId).subscribe((response : any) => { // download file
var blob = new Blob([response.blob()], {type: 'application/octet-stream'});
var filename = 'file.csv';
saveAs(blob, filename);
});
}
Service is as below -
fetchOutput(jobId : string, outputId) {
var jobOutputURL = "myEnpoint";
var params = this.createHttpAuthorization(jobOutputURL,"GET");
params["format"] = "csv";
const options = {
headers: new HttpHeaders( { 'Content-Type': 'application/octet-stream',
'Accept' : 'application/octet-stream',
'Access-Control-Allow-Origin' : '*'}
)};
var endpoint = `${jobOutputURL}?oauth_consumer_key=${params["oauth_consumer_key"]}&oauth_signature_method=${params["oauth_signature_method"]}&oauth_nonce=${params["oauth_nonce"]}&oauth_timestamp=${params["oauth_timestamp"]}&oauth_version=1.0&format=${params["format"]}&oauth_signature=${params["oauth_signature"]}`;
return this.httpClient.get(endpoint, {...options, responseType: 'blob'});
}
To fetch an application/octet-stream, you have to set arraybuffer as the response type in the Angular HttpHeaders.
This is the service method:
fetchOutput(): Observable<ArrayBuffer> {
let headers = new HttpHeaders();
const options: {
headers?: HttpHeaders;
observe?: 'body';
params?: HttpParams;
reportProgress?: boolean;
responseType: 'arraybuffer';
withCredentials?: boolean;
} = {
responseType: 'arraybuffer'
};
return this.httpClient
.get('https://your-service-url.com/api/v1/your-resource', options)
.pipe(
map((file: ArrayBuffer) => {
return file;
})
);
}
This is the call to the service method and to the saveAs function:
this.yourService
.fetchOutput()
.subscribe((data: any) => {
const blob = new Blob([data], { type: 'application/octet-stream' });
const fileName = 'Your File Name.csv';
saveAs(blob, fileName);
})
As other users are suggestion: 401 Unauthorized is usually a client side error due to missing credentials.

keeps fetching the old json data?

I'm trying to fetch a json file from a https link however, no matter what link a give the result does not change!?
I validated all the json files. in case they had an error.
the responseData stays the same, and even when I force the data to change by instead returning responseData returning a json manually written; it changes right back to the old json data that just doesnt change when I return responseData back.
And the responseData that I requested to be be posted on the console gives the wrong information
The url given is correct.
but the output doesnt correspond to the data when I fill the link in the internetbrowser.
constructor(props){
super(props);
this.state = {
connected: false,
}
this.init = this.init.bind(this);
this.getJson = this.getJson.bind(this);
this.updateVisited = this.updateVisited.bind(this);
}
init = async ({json})=>{
if(json==null){
await AsyncStorage.setItem('database', "");
alert('error occured');
} else {
await AsyncStorage.setItem('database', JSON.stringify(json));
this.setState({
connected: true
});
}
}
getJson = async ()=>{
var url = await AsyncStorage.getItem("database_url");
console.log(url);
return fetch(url,
{
method: "GET",
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json',
},
})
.then(response => response.json())
.then(responseData => {
this.updateVisited(url);
console.log(responseData);
return responseData;
})
.catch(error => {
alert('Could not connect!');
return null;
})
}
connect = async ({url})=>{
await AsyncStorage.setItem("database_url", url);
this.getJson().then(json => this.init({json}));
}
"a_json": [{"name": "greg"}]
"test": [{"name": "sheldon"}]
"temp": [{"name": "bob"}]
when the url points to the json test it gives bob expecting sheldon
when the url points to the json temp it gives bob expecting bob
when the url points to the json a_json it gives bob expecting greg
when returning a json without trying to fetch it from the internet at the place of responseData; it gives the expecting value
If you need more information, feel free to ask.
Thank you for your time reading my question.
The problem was the Cache-Control.
I added 'Cache-Control': 'no-cache' to the header of the fetch, which fixed the problem!
This was pointed out by #Pritish Vaidya in the comments

cors unexpected end of JSON input

I am parsing my json on end but I am still receiving this error.
'use strict';
const http = require('http');
const tools = require('./tools.js');
const server = http.createServer(function(request, response) {
console.log("received " + request.method + " request from " + request.headers.referer)
var body = "";
request.on('error', function(err) {
console.log(err);
}).on('data', function(chunk) {
body += chunk;
}).on('end', function() {
console.log("body " + body);
var data = JSON.parse(body); // trying to parse the json
handleData(data);
});
tools.setHeaders(response);
response.write('message for me');
response.end();
});
server.listen(8569, "192.168.0.14");
console.log('Server running at 192.168.0.14 on port ' + 8569);
Data being sent from the client:
var data = JSON.stringify({
operation: "shutdown",
timeout: 120
});
I successfully receive the json but I am unable to parse it.
Update:
I've updated the code to include the server code in its entirety.
To be perfectly clear, using the following code:
....
}).on('end', function() {
console.log("body " + body);
var json = JSON.parse(body); // trying to parse the json
handleData(json);
});
I get this:
However, this:
....
}).on('end', function() {
console.log("body " + body);
//var json = JSON.parse(body); // trying to parse the json
//handleData(json);
});
produces this
Can we see the server code, please?
Here is a working end-to-end example which is (more or less) what you are attempting, I believe.
"use strict";
const http = require('http');
/********************
Server Code
********************/
let data = {
operation: 'shutdown',
timeout: 120
};
const server = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.write(JSON.stringify(data));
res.end();
});
server.listen(8888);
/********************
Client Code
********************/
let options = {
hostname: 'localhost',
port: 8888,
path: '/',
method: 'POST',
headers: {
'Accept': 'application/json'
}
};
let req = http.request(options, res => {
let buffer = '';
res.on('data', chunk => {
buffer += chunk;
});
res.on('end', () => {
let obj = JSON.parse(buffer);
console.log(obj);
// do whatever else with obj
});
});
req.on('error', err => {
console.error('Error with request:', err);
});
req.end(); // send the request.
It turns out that as this is a cross-origin(cors) request, it was trying to parse the data sent in the preflighted request.
I simply had to add an if to catch this
....
}).on('end', function() {
if (request.method !== 'OPTIONS') {
var data = JSON.parse(body);
handleData(data);
}
});
Further reading if you're interested: HTTP access control (CORS)
Put the identifiers in quotes.
{
"operation": "shutdown",
"timeout": 120
}
http://jsonlint.com/ Is a helpful resource.