I'm using Node. On server side, I got requests from clients. Before sending response to the client, I'm fetching data from other web sites so I'm creating a request object using request npm module.
Basically I have something like this :
request(url, (req,resp) => {...});
The problem is with some pages it's dynamic. Think about something like facebook, when you scroll down the page with your mouse => new data is coming and the HTML code is becoming bigger and bigger.
The problem with the HTML data I got back from resp object in mycallback function is that it only has very little data.
With a browser I can get more data by scrolling down the page but is it possible to get all data programatically ?
Thank you ;)
You can try:
request(url, (req,resp) => {...})
.on('complete', (data) => {
console.log(data.body);
});
Or
let bufferedData = '';
on('data', (data) => {
bufferedData += data.toString();
})
Related
Working on an Angular app (golf handicap generator) and following along with a Pluralsight video to implement a form and using Http and he is using PutsReq as a way to just test the post and response. When I follow everything I am getting the above syntax error but unsure why. All I am trying to do is send 2 different numbers. Here is my code
Service Component
postUserRoundsForm(userRounds: UserHandicapRounds) : Observable<any> {
return this.http.post("https://putsreq.com/ZmOvG1coGYa174oig3r1", userRounds);
}
Template Component
onSubmit(form: NgForm) {
console.log('in onSubmit: ', form.valid)
this.dataService.postUserRoundsForm(this.userHandicapRounds).subscribe({
next: (v) => console.log('Donzo: ', v),
error: (e) => console.log('Errorrr: ', e),
complete: () => console.log('complete: sends NOTHING')
})
}
HTML
<form #handicapCalcForm="ngForm" (ngSubmit)="onSubmit(handicapCalcForm)">
I also checked the network tab in chrome and saw the payload is coming back with the correct data. I also tried re-serving the app and a different browser and that didn't help either.
I'm fairly new to HTTP and trying to learn more about it but haven't been able to find a reason as to why this isn't working, I would think it should
Im writing an PWA in Svelte with Routify and im trying to save notes (containing id, title and a body) in a local json file.
Ive been following this code from Svelte REPL (Svelte POST example), but they use an web URL. When trying to use a direct link i get a 404, even tho the path is correct.
<script>
let foo = 'title'
let bar = 'body'
let result = null
async function doPost () {
const res = await fetch('https://httpbin.org/post', {
method: 'POST',
body: JSON.stringify({
foo,
bar
})
})
const json = await res.json()
result = JSON.stringify(json)
}
</script>
<input bind:value={foo} />
<input bind:value={bar} />
<button type="button" on:click={doPost}>
<p>Post it.</p>
</button>
<p>Result:</p>
<pre>
{result}
</pre>
I installed a json server plugin, which kinda worked, but i want to store the data as a local file.
Is it possible to write, using POST to a local json file without using any server?
Is it possible to use relative path when using fetch? Or should i use something else?
Generally, you don't POST data anywhere else but to a server. Having said that, if you absolutely want to save your data using POST, you can add a serviceworker to your app that intercepts the fetch() request and then saves the data in cache, indexeddb, localstorage or something like this. But having that serviceworker in between just for that is a bit silly, you should rather store the data directly in cache, indexeddb or localstorage.
Example for localstorage:
const data = { someKey: { someOtherKey: 'some value' } };
localStorage.setItem('myData', JSON.stringify(data));
Be aware though that, no matter which kind of storage you're using, they all might be wiped out if the user decides to clear browser data or if the browser cleans up by itself due to storage shortage.
I have two issues but I believe the second issue will be fixed once the first is fixed (question title).
I am receiving JSON from woocommerce. I can call for this data by using fetch of course on client side and it looks as such in the code:
async componentDidMount() {
const response = await fetch('/products');
const json = await response.json();
this.setState({
data: json,
})
// other code ....
}
When I go on the browser I get this error regarding my json data:
Unhandled Rejection (SyntaxError): Unexpected token < in JSON at position 0
With the following error in the console.log:
index.js:6 GET http://localhost:3000/products 500 (Internal Server Error)
index.js:6 Uncaught (in promise) SyntaxError: Unexpected token < in JSON at position 0
Once the webpage is refreshed...this all disappears, everything becomes A-OKAY, why? and how do I go about rectifying this?
My JSON data when consoled and the page refreshed returns an object - no '<' is there. Also I don't know why I get the 500 error shown above? I am learning node.js - so I think this is a server side issue as I had no issues before I split my code to client and server.
help?
What is happening is the data call you do takes time to load the data.
Till then the this.state.data is null. The error
Unexpected token < in JSON at position 0
is because you are trying to process the this.state.data but finds null. You need to make sure you handle what needs to be displayed when data is null.
Also, I think you don't need the await before response.json()
The 500 server error is a server side issue.
To stop this whole refreshing the page issue; I had to fix my server file (node.js of course)
My get request had originally looked like this:
let response;
app.get('/products', (req, res, err) => {
WooCommerce.get('products', function(err, data, res) {
response = res;
});
res.status(200).json(JSON.parse(response));
});
The issue here was that I was calling /products with fetch which url didn't point to anything but another API call, which only got called once I forced it pass the first url call I guess using a page refresh. Forgive my understanding here.
The correct code was calling Woocommerce api first then passing its response to the /product url so I can fetch it in the front end, like so;
let response;
WooCommerce.get('products', function(err, data, res) {
response = res;
app.get('/products', (req, res, err) => {
if (res.status(200)) {
res.status(200).json(JSON.parse(response));
} else {
console.log('theres an error', err);
}
})
});
And Tarrrrdaaa no refresh issue/SyntaxError error!!!
As one of the answers says, this error happens when you try to parse a nullish value. You can fix this issue by making sure the data is defined before trying to parse it:
if(data){
JSON.parse(data);
}
Hey Everyone i have a router that basically pulls in a JSON file and outputs various instances of the array. When i console.log the title in my for loop it loops through and outputs and each instance is outputted, works great. When i put my res.render inside of the for loop and pass it the same variable being looped in the console.log it will only output the first instance. Does anyone know why this is? and is there a way to loop through the res.render to output all instances in the JSON same as the console.log.
Thank you for your time,
router.get('/fuelTypeFilter', function(req, res, next) {
var url = "Example.JSON"
request({
url: url,
json: true
}, function (error, response, obj) {
if (!error && response.statusCode === 200) {
// console.log(obj) // Print the json response
for(key in obj.categories){
var img = obj.categories[key];
var title = obj.categories[key].product_category_title;
console.log(title);
// console.log(obj.categories[key]);
}
res.render('fuelTypeFilter', { title: 'Fuel Type', item: title });
} //end of if
}); // end of request
});
What res.render does is basically serve up a plain HTML file with some template inserted, so with data like the object you pass into the render call. Therefore, res.render acts like a return statement, as in when your code reaches that mark, it signals to the router that we need to render this page (present in the browser to user), therefore we must be done with our router handling logic.
As to whether it's possible to 'loop through' render calls, the answer is no since that would be implying to just, even if succeeded, present however many pages you want instantaneously one after the other, so in that perspective, you wouldn't really want to keep making render calls as your view in the browser would effectively keep refreshing moment after moment.
One way to do this with render is just package your array-like data into the object that you pass into res.render and then, assuming you are using some sort of templating engine, do something analogous to a 'for loop' there, looping and printing the elements as, say HTML divs.
Hope this is a bit helpful.
Hi I am trying to make a http request from this rss feed. I tried this:
makeRequest() {
this.http.get('http://www.gazetaexpress.com/rss/auto-tech/?xml=1')
.subscribe(data => {
this.posts = data;
console.log("request funktioniert");
}, error => {
console.log(JSON.stringify(error.json()));
});
}
When I use {{posts}} in the html page it returns:
Response with status: 200 OK for URL: http://m.gazetaexpress.com/mobile/rss/auto-tech/?xml=1
When using {{post | json}} it return the whole html page. I tried using:
this.http.get('http://www.gazetaexpress.com/rss/ballina/?xml=1').map(res => res.json())
instead for the first line but it returned an error:
TypeError: error.json is not a function
Please help me :D
EDIT
I imported the xml2js library and typings and imported in the ts file:
import * as xml2js from "xml2js"
And changed the request to:
makeRequest() {
this.http.get('http://www.gazetaexpress.com/rss/auto-tech/?xml=1').subscribe(data => {
this.posts = data;
xml2js.parseString(this.posts, function (err, result) {
console.log(result);
});
}, error => {
console.log(JSON.stringify(error.json()));
});
}
The console returns undefined...
EDIT 2
Okay I hotfixed the issue. I just took the hole string and cut it down to pieces using the node structure and using this.posts.replace(/\\t|\\n|\\r/gi, ""); to get rid of escaped chars. Surely there must be a good library working on this issue my code is pretty badass ;). I will overlook this code later on for now it`s working and thats okay. If you have some better ideas feel free to tell me :D
If you get a response of Status 200 OK you should be able to show the results as they are ordered on the website you are making the http request.
Try to fetch the data each when subscribing and use an asnyc pipe in your html code this should do the trick