I want to parse an url to html then to use it. For this, I use axios but I get the following error:
Request failed with status code 500
My code:
import axios from 'axios';
export default async function parseHtml() {
const url = 'http://www.ameli.fr/accueil-de-la-ccam/telechargement/index.php';
let response;
try {
response = await axios({
method: 'get',
url: url,
});
} catch (error) {
throw error.message;
}
console.log(response.data);
}
Any idea please, I try also to use jsdom, htmlparse but I don't get any result
That url seems to be sending back a 500 error incorrectly (or it appears so).
You can ignore this error (something you wouldn't normally do!) using the validateStatus property like so:
async function parseHtml() {
const url = 'http://www.ameli.fr/accueil-de-la-ccam/telechargement/index.php';
let response;
try {
response = await axios({
url,
method: 'GET',
// Ignore failing HTTP status codes
validateStatus: status => true
});
console.log("Response status code:", response.status);
console.log("Response data:", response.data);
} catch (error) {
console.log(error.message);
}
}
This will give you the html data, which you can then parse.
Related
When trying to bring a news/posts list from a common WordPress RSS Feed (Rest API), it ends up returning my index.ejs webpack template in my app instead of the JSON response.
Using an endpoint like: example.com/wp-json/wp/v2/posts, the expected response would be a JSON "string" with a certain number of posts,
and instead it's returning a text/html response, which happens to be the html template for my app, nothing unkown.
The code in JS, React being used is:
import axios from 'axios';
const getPosts = async () => {
const postsEndpoint = '/wp-json/wp/v2/posts';
const config = {
headers: {
'accept': 'text/html, application/json',
'Content-Type': 'application/json; charset=UTF-8',
},
params: {
_limit: 5,
}
};
let response;
try {
response = await axios.get(postsEndpoint, config);
} catch (err) {
console.log(err);
} finally {
// eslint-disable-next-line no-unsafe-finally
return response.json;
}
};
export default getPosts;
And the code to show this in display:
const blogNews = () => {
const [isLoading, setIsLoading] = useState(false);
const [posts, setPosts] = useState(null);
const getPostsData = async () => {
try {
setIsLoading(true);
const data = await getPosts();
console.log(data);
if (data.status === 200) {
setPosts(data);
setIsLoading(false);
}
} catch (err) {
console.log(err);
} finally {
setIsLoading(false);
}
};
useEffect(() => {
getPostsData();
}, []);
return ...
And the nginx configuration lines looks like this:
location /wp-json/wp/v2/ {
proxy_pass http://example.com/;
}
I'm using 'text/html' in the Accept header just to see what's in the response, if I only include 'application/json' it just returns a 404 error.
The fact that I'm not including Cors nor CSP headers it's because it's because the policy criteria is being met with the nginx configuration.
Do you guys have any recommendations regarding this? Thanks in advance.
I'm trying to fetch a json file from a https link however, no matter what link a give the result does not change!?
I validated all the json files. in case they had an error.
the responseData stays the same, and even when I force the data to change by instead returning responseData returning a json manually written; it changes right back to the old json data that just doesnt change when I return responseData back.
And the responseData that I requested to be be posted on the console gives the wrong information
The url given is correct.
but the output doesnt correspond to the data when I fill the link in the internetbrowser.
constructor(props){
super(props);
this.state = {
connected: false,
}
this.init = this.init.bind(this);
this.getJson = this.getJson.bind(this);
this.updateVisited = this.updateVisited.bind(this);
}
init = async ({json})=>{
if(json==null){
await AsyncStorage.setItem('database', "");
alert('error occured');
} else {
await AsyncStorage.setItem('database', JSON.stringify(json));
this.setState({
connected: true
});
}
}
getJson = async ()=>{
var url = await AsyncStorage.getItem("database_url");
console.log(url);
return fetch(url,
{
method: "GET",
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json',
},
})
.then(response => response.json())
.then(responseData => {
this.updateVisited(url);
console.log(responseData);
return responseData;
})
.catch(error => {
alert('Could not connect!');
return null;
})
}
connect = async ({url})=>{
await AsyncStorage.setItem("database_url", url);
this.getJson().then(json => this.init({json}));
}
"a_json": [{"name": "greg"}]
"test": [{"name": "sheldon"}]
"temp": [{"name": "bob"}]
when the url points to the json test it gives bob expecting sheldon
when the url points to the json temp it gives bob expecting bob
when the url points to the json a_json it gives bob expecting greg
when returning a json without trying to fetch it from the internet at the place of responseData; it gives the expecting value
If you need more information, feel free to ask.
Thank you for your time reading my question.
The problem was the Cache-Control.
I added 'Cache-Control': 'no-cache' to the header of the fetch, which fixed the problem!
This was pointed out by #Pritish Vaidya in the comments
For the following created code in react, after I search similar question, I get always an error (seems that the error is return of a promise).
I am using webpack version 3.1.9
In web-pack configuration I did (don't know whether it is necessary):
module.exports = {
...
devServer: {
headers: {
'Access-Control-Allow-Origin': '*',
"Access-Control-Allow-Headers": "*"
}
},
...
Here is my code:
var options = {
method: 'get',
mode: 'no-cors',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'Access-Control-Allow-Headers': '*',
'Access-Control-Allow-Origin': '*'
}
};
let _url = 'my url ... web api returns json';
fetch(_url, options)
.then(response =>
response
.json()
.then(data => ({
data: data,
status: response.status
}))
.catch(err =>
/******ERROR: always catch the error *****/
({ error_data: err })
)
)
.then(res => {
console.log(res);
// console.log(res.status, res.data.title)
})
.catch(err => {
console.log(err);
});
The error in the line with the asterisks, as code above
SyntaxError: Unexpected end of input at eval
The code was checked for restful api in C#:
I did in the controller code:
public ActionResult Index()
{
ActionResult x = Json(db.Trainees.ToList(),
JsonRequestBehavior.AllowGet);
//return Content(db.Trainees.ToList().ToString(),
"application/json");
return Json(db.Trainees.ToList(), JsonRequestBehavior.AllowGet);
// return View(db.Trainees.ToList());
}
I assume it is related to fact that json returns a Promise, as described in: json returns promise
I see that json is problematic. When I change response.json() to response.text() there is no error, but I realize that even I send the options with 'no-cors', I see an information message:
Cross-Origin Read Blocking (CORB) blocked cross-origin response ... with MIME type application/json
Seems that the fetch ignore the options with 'no-cors'.
Any clues, why the code encounters an error?!
Thanks.
Problem had been fixed.
In react I did the changes:
var options = {
method: 'GET',
}
...
In C# restfull api controller I did the changes:
Response.AddHeader("Access-Control-Allow-Origin", "*");
return Json(db.Trainees.ToList(), "application/json",
JsonRequestBehavior.AllowGet);
I'm trying to execute a callback function in nodejs, using expressjs and angular 2 (i don't know if the angular2 part it's relevant).
What I do is:
I have a formular in angular2, with that I send a get request to my API route, then I send the text field in the formular to the URL via get, then I do an MYSQL query to look into a phonebook database, and I'm expecting to get a complete user with his details, from the phonebook.
Formular:
<div class="container">
<div class="col-md-4">
<h1>Addressbook</h1>
<form [formGroup]="searchForm" (ngSubmit)="doSearch($event)">
<input formControlName="searchString" type="text" placeholder="Name">
<button type="submit">Search</button>
</form>
</div>
</div>
First function, doSearch:
doSearch(event) {
let formData = this.searchForm.value;
var searchString = this.searchForm.value.searchString;
this.http.get('/phonebook/search/'+searchString, function(req, res){}).subscribe(
function(response) {
console.log("Success Response");
},
function(error) { console.log("Error happened" + error)},
function() { console.log("the subscription is completed")}
);
}
This calls to the route sending a parameter, so not so hard.
Now the create router gets into the game:
public static create(router: Router, basePath: string) {
console.log("[SearchRoute::create] Creating routes for /search.");
// call the function for retrieving the address book results
router.get(basePath + "/search/:searchString", (req: Request, res: Response, next: NextFunction) => {
console.log("## [SearchRoute] Called GET /search.");
var object = searchUser(req);
console.log(object);
});
}
And finally, the function searchUser gets called:
function searchUser(req: Request) {
console.log("searchUser Function executed.");
var searchString = req.params.searchString;
var query = p_query('SELECT XXXX')
.then(function (results) {
console.log("query executed and all okay");
return (results);
})
.catch(function (error) {
console.error("Wooopsi", error);
});
console.log("query result: "+query);
}
Additionally, I post here the new query function that I build to be able to handle promises (which I don't know if it was the best choice):
function p_query(q) {
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
myMYSQL.db.query(
q,
function (error, results) {
if (error)
reject(error);
resolve(results);
});
})
};
So, what I actually want to do, what's my issue?
I want to send the result of the query back to the client (the angular2 formular), and I was not being able to do it...
So after this really long post, I really appreciate if you read til here, and sorry for the complicated question!
PS: I know i explain myself really bad :(
Regards,
Daniel
In this official angular 2 documentation on the http client they propose to put the http logic into a separate service. I've setup it similar to the example just for a search.service.ts:
import { Injectable } from '#angular/core';
import { Http, Response,Headers, RequestOptions,URLSearchParams }
from '#angular/http';
import { Observable } from 'rxjs/Observable';
import 'rxjs/add/operator/catch';
import 'rxjs/add/operator/map';
#Injectable()
export class SearchService {
constructor(private http: Http) {
}
getSearchResult(searchString) : Observable<any> {
return this.http.get('/phonebook/search/'+searchString)
.map(this.extractData)
.catch(this.handleError);
}
private extractData(res: Response) {
let body = res.json();
return body;
}
private handleError (error: Response | any) {
let errMsg: string;
if (error instanceof Response) {
const body = error.json() || '';
const err = body.error || JSON.stringify(body);
errMsg = `${error.status} - ${error.statusText || ''} ${err}`;
} else {
errMsg = error.message ? error.message : error.toString();
}
return Observable.throw(errMsg);
}
}
In your component import the service and do the code snippets:
// don't forget to put the service in the app.modul or the component providers!
constructur(public mySearchService : SearchService) {}
// in your doSearch of your component:
doSearch(event) {
let formData = this.searchForm.value;
var searchString = this.searchForm.value.searchString;
mySearchService.getSearchResult(searchString).subscribe(
data => mylist.data, // or which datastructure I want to write to.
error => console.error(error) // or how I log the errors..
);
}
EDIT: The search_user in your database model:
function searchUser(searchString) {
console.log("searchUser Function executed.");
return myMYSQL.db.select('phonebookentry', {
pbe_lastname: searchString, pbe_syncstate: 'new'
}) // returning the promise/observable to the main function...
} // Currently I don't know, how the returned data looks like.
On the node/express side in the router send it with res.json EDIT: use asynchronous call to searchUser:
router.get(basePath + "/search/:searchString",
(req: Request, res: Response, next: NextFunction) => {
console.log("## [SearchRoute] Called GET /search.");
searchUser(req)
.then( data => res.json(data);console.log(data) )
.catch (error => console.log(error));
});
You should go with recursive callback with each query results try to enjoy the beauty of async platform.
Send data to client via
res.send(data);
Your answer it's completely perfect, i understand everything! The only problem that i'm facing now, its this one:
I'm calling the function searchUser, and it doesn't return anything, just an undefined object, so i quess i'm not doing the return correctly.
That's my searchUser function:
function searchUser(searchString) {
console.log("searchUser Function executed.");
myMYSQL.db.select('phonebookentry', {
pbe_lastname: searchString,
pbe_syncstate: 'new'
}).then(function (user) {
console.log("user before: "+user);
return (user);
}).catch(function (err) {
console.log(err);
})}
Thank you so much for your useful answer! I'm almost finished here
I have error meantime angular2 post rest data to NodeJS backend.
I see POST is done, server is LOG correct data, but error is showing up on browser.
An error occurred:
SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data
My NG2 call and service:
onSubmit(form:FormGroup) {
let userform: FormGroup = form.value;
console.log("userform: ", userform);
if (form.valid) {
console.log(form.value);
this.appService.signIn(userform)
.subscribe(form => console.log('subscribe: ', form))
} else {
console.log("Form is not VALID!");
}
}
SERVICE:
signIn(dataUser: Object): Observable<User> {
dataUser = JSON.stringify(dataUser);
debugger;
let headers = new Headers({
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': 'http://127.0.0.1:3005'
});
let options = new RequestOptions({ headers: headers });
console.log("data: ", dataUser, "\nHeaders: ", headers);
return this.http
.post( this.signInUrl, dataUser, options)
.map( (res:Response) => res.json().data || { } as User )
.catch(this.handleError);
}
and nodeJS:
app.post('/login', function (req, res) {
console.log("Recived login request!");
console.log("Request: ", req.body);
res.header({
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'GET,PUT,POST,DELETE',
'Accept': 'q=0.8;application/json;q=0.9'
})
res.end();
});
In post we have: "{"username":"username","password":"password"}".
What I'am making wrong? Please for help or solution.
Awwwww. That was my bad, Take care of your NodeJS Server response. After get POST, should be sended any res.json({status: "OK"}) or sommething similar, to get response. This error was not because of Angular2, but because of NodeJS. Browser get empty response from nodeJS, or it was not JSON format.