How should I go about parsing and using website data? I am using node js [closed] - html

Question

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 2 years ago.
Improve this question
I am using node js to create a bot that gets stats from r6.tracker.network, but when I load it in the data won't show up unlike when I have 'www.google.com' as the host. I don't really know where to go from here. I'm trying to use several different debugging methods, but I haven't found anything yet. There is no output.
function getWebpage(parameter){
const pathuser = '/profile/pc/' + parameter;
var http = require('http');
var options = {
host: 'r6.tracker.network',
path: '/'
}
var request = http.request(options, function (res) {
var data = '';
res.on('data', function (chunk) {
data += chunk;
});
res.on('end', function () {
console.log(data);
});
});
request.on('error', function (e) {
console.log(e.message);
});
request.end();
}

Answer 1

They are redirecting you to the https version of their site.
The key here is to add this logging:
console.log(res.statusCode);
console.log(res.headers.location);
When you do that, you will see this:
301
https://r6.tracker.network/
In other words, they want you to use the https version of their site and they are redirecting you to do that. You won't get the content of the web page from the http URL. You have to use https.
Coding takeaways here:
Always look at the http status and only proceed normally if you get a 2xx response.
Be prepared for 3xx redirect responses.
FYI, if you use a more modern library such as got() or any of the other libraries listed here, they will all follow redirects for you automatically and they will gather the full response for you automatically too. And, they use the more modern promise-based method of asynchronous programming. I'd really suggest you pick one of those libraries for your outbound http requests.