I would like to scrape this page: calendar events
for specific data, like formattedDate and description. How do I go about that in a module in Node.JS. I am having a hard time understanding the process in Node.JS.
Any help would go a long way, thanks in advance.
it's pretty simple, you can import the request module and use it. For example, see code below.
const request = require("request");
request("MY_URL", (error, response, body) => {
console.log('body:', body);
});
Also, you can try this here, on Repl.it
First of all, you need to parse your JSON, this allows you to access fields from received json.
const data = JSON.parse(body);
Now, if you want to access some information about an event you need to loop events and access what you need, something like:
const events = data.bwEventList.events;
events.map((data, index) => console.log(data.calendar))
Final code also on Repl.it
from nodeJS docs here
const http = require('http');
http.get('http://umd.bwcs-hosting.com/feeder/main/eventsFeed.do?f=y&sort=dtstart.utc:asc&fexpr=(categories.href!=%22/public/.bedework/categories/sys/Ongoing%22%20and%20categories.href!=%22/public/.bedework/categories/Campus%20Bulletin%20Board%22)%20and%20(entity_type=%22event%22%7Centity_type=%22todo%22)&skinName=list-json&count=30', (res) => {
const { statusCode } = res;
const contentType = res.headers['content-type'];
let error;
if (statusCode !== 200) {
error = new Error('Request Failed.\n' +
`Status Code: ${statusCode}`);
}
if (error) {
console.error(error.message);
// consume response data to free up memory
res.resume();
return;
}
res.setEncoding('utf8');
let rawData = '';
res.on('data', (chunk) => { rawData += chunk; });
res.on('end', () => {
try {
const parsedData = JSON.parse(rawData);
console.log(parsedData["bwEventList"]["resultSize"]);
} catch (e) {
console.error(e.message);
}
});
}).on('error', (e) => {
console.error(`Got error: ${e.message}`);
});
see console.log(parsedData["bwEventList"]["resultSize"]);
slice parsedData as an array until you get what you want
Related
I sent to asyncStorage all the info as stringify,
i tried to parse it.
this is what i get from console log:
"{\"metadata\":{\"lastSignInTime\":1610728860334,\"creationTime\":1610728860334},\"phoneNumber\":null,\"displayName\":null,\"isAnonymous\":false,\"providerData\":[{\"email\":\"ad#ad.com\",\"phoneNumber\":null,\"uid\":\"ad#ad.com\",\"photoURL\":null,\"displayName\":null,\"providerId\":\"password\"}],\"email\":\"ad#ad.com\",\"emailVerified\":false,\"providerId\":\"firebase\",\"photoURL\":null,\"uid\":\"3lkoKoMxQSMKeSxFOyysESt3oKh1\"}"
and i need to get email and uid seperate.
how do I get in that object? i tried user.email or user.providerData.email non of them work.
any suggestion?
edited:
here is the object I get from firebase
let res = await auth().createUserWithEmailAndPassword(Email, Password)
if (res) {
console.log( "?", res)
this.setState({ userData: JSON.stringify( res.user) });
this.storeToken(JSON.stringify(res.user));
then I store the token in async:
async storeToken(user) {
console.log('set user register: ', user)
try {
await AsyncStorage.setItem("userData", JSON.stringify(user));
} catch (error) {
console.log("Something went wrong", error);
}
}
and I get the object from above.
const readData = async () => {
console.log('data === ')
try {
const data = await AsyncStorage.getItem('userData')
let _data = JSON.parse(data);
console.log('data === ', data)
If you share code block it'll be easy for us.
Here is general answer.
Console log shows its still in string format. I use this separate file to read and write json to AsyncStorage. You can either use this OR match to see your mistake.
import AsyncStorage from '#react-native-community/async-storage';
const Api = {
storeData: async function (name, value) {
try {
await AsyncStorage.setItem(name, value);
return true;
} catch (error) {
return false;
}
},
readData: async function (name) {
let value = null;
try {
value = await AsyncStorage.getItem(name)
return JSON.parse(value);
} catch (e) {
return [];
}
},
}
export default Api;
after few console log I was able to get it by double parsing the object.
let _data = JSON.parse(JSON.parse(data));
console.log('data === ', _data.email)
and seem to work.
I have an API request that returns the image of the product that had been saved in the database in the form of buffer in MONGODB. The API returns the image and I want this image as source to my html element. How do I do that?
The API request is:
router.get('/product/avatar/:id', async (req, res) => {
try{
const product = await Product.findById(req.params.id)
if(!product || !product.avatar){
throw new Error()
}
// console.log('product avatar')
res.set('Content-Type', 'image/png')
res.send(product.avatar)
} catch(e) {
res.status(404).send()
}
})
And this is how I was trying to assign that image to my HTML
the code snippet given below is a part of the javascript used to render information on webpage from database.
var picId = data[i].name + 'Image'
// console.log(data[i]._id)
fetch('/product/avatar/'+data[i]._id, {
method: 'GET'
}).then((res) => {
return res.json()
}).then((data) => {
// picId.value = data
// document.getElementById(picId).src = data
console.log(data)
})
everybody! I'm trying to build a Google Pagespeed client in nodejs.
I get a json file with syntax errors for any url. For example: url: http://www.bbc.com/ , error: enter image description here
json file:enter image description here
I need only the property "ruleGroups". I tried to extract it so jsonpath.query(d, '$.ruleGroups') -
did not work out.
Help please understand, sorry if the issue is dilettian.
let key = 'key';
let url = 'http://www.bbc.com/';
let strategy = 'desktop';
https.get({
host: 'www.googleapis.com',
path: '/pagespeedonline/v4/runPagespeed?url=' + encodeURIComponent(url) +
'&key='+key+'&strategy='+strategy
}, function (res) {
console.log ("statusCode:", res.statusCode);
console.log ("headers: ", res.headers);
res.on ('data', function (d) {
let json = JSON.parse(d);
fs.appendFile('log.txt', json);
});
}). on ('error', function (e) {
console.error (e);
});
You may need to accumulate all the data, then parse in the “end” event:
let rawData = '';
res.on('data', (chunk) => { rawData += chunk; });
res.on('end', () => {
try {
const parsedData = JSON.parse(rawData);
console.log(parsedData);
} catch (e) {
console.error(e.message);
}
});
More information: https://nodejs.org/api/http.html#http_http_get_options_callback
I'm still very new to node.js so please bear with me.
I'm trying to extract POST data then sent a json as response, but I don't seem to be able to extract the data from the POST request and even worse I can't find the syntax for people who are NOT using Express to send the json. It keeps telling me res.json is not a function.
EDIT: I found out the problem for the json part, I was a dump. I finally remember what I was told, json are sent like strings.
var http = require('http');
var qs = require("querystring");
server = http.createServer(function (req, res) {
try {
var body = "";
var post = qs.parse("");
if (req.method == "POST") {
res.writeHeader(200, {"Content-Type": "application/json"});
req.on("data", function (data) {
body += data;
console.log(data); //It gives something like <Buffer xx xx xx ...>
if (body.length > 1e6)
req.connection.destroy();
});
req.on("end", function () {
post = qs.parse(body);
console.log(post.test); //It gives "undefined"
});
res.end(JSON.stringify({ a: 1 }));
} catch(err) {
console.dir(err);
res.writeHeader(200, {"Content-Type": "text/plain"});
res.end("Hi hi");
}
});
server.listen(8000);
console.log("http start #8000");
Any help? Thanks in advance.
below solves the date to string (i.e. converting buffer to string
res.on('data', function(chunk) {
var textChunk = chunk.toString('utf8');
// console.log(textChunk); // will give you a stream of text from data
});
you could store textChunk out of the ondata handler, to then use that if required (say returning relevant data to the user back again)
For example, when the path is
/json/users/4
meteor app must return json something like
{
id: 4,
name: 'Alex'
}
I'm using reactrouter:react-router for client routing. I know about reactrouter:react-router-ssr, but how to use it to response raw json? And make it not conflicting with existing client routing?
I found the answer. Meteor's default Webapp package will help (doc):
WebApp.connectHandlers.use("/hello", function(req, res, next) {
res.writeHead(200);
res.end("Hello world from: " + Meteor.release);
});
I put this in server folder. Other routes will be rendered as they was.
So, there is more useful example (es6):
WebApp.connectHandlers.use("/payme", function(req, res, next) {
res.writeHead(200, {'Content-Type': 'application/json'});
if (req.method === 'POST') {
req.on('data', (chunk) => {
const body = chunk.toString();
if (body.length < 1e6) {
const params = body.split('&').reduce((result, item) => {
const [key, val] = item.split('=');
//do it for utf-8 values (I use it for cyrillic strings)
result[key] = unescape(decodeURI(val)).replace(/\+/g, ' ');
return result;
}, {}); //post method params
//do something and get resulting json
res.end(JSON.stringify(result));
} else
res.end(JSON.stringify({error: 'too big query'}));
});
} else
res.end(JSON.stringify({error: 'isnt post req'}));
});
req.query can be used to get GET params.