I am trying to scrape a website for some content using NodeJs, it all works good but then the scraped text is desplayed in the console only however, i want to pass my scraped data to my html page (index.html), but didn't know how to do that.
here is my nodejs file (scrape.js)
const request = require('request');
const cheerio = require('cheerio');
request('https://store.steampowered.com/search/?filter=weeklongdeals', (error, response, html) =>{
if(!error && response.statusCode == 200){
const $ = cheerio.load(html);
$('.title').each((i,ele) => {
const title = $(ele).text();
console.log(title);
});
}
})
and here is my html file where the data should displayed in(index.html)
<!DOCTYPE html>
<html>
<title>Real Time Data</title>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css" />
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Raleway" />
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.6.9/angular.min.js"></script>
<body ng-app="myApp" ng-controller="myCtrl">
<div class="bgimg w3-display-container w3-animate-opacity w3-text-white">
<div class="w3-display-topleft w3-padding-large w3-xlarge">
Real Time Data
</div>
<div class="w3-display-middle">
<h1 class="w3-jumbo w3-animate-top">Real Time Data</h1>
<center>
<div id="getDDta">
<table>
<tr>
<th>Title</th>
</tr>
<tr>
<td>Data...</td>
</tr>
</table>
</div>
</center>
</div>
<div class="w3-display-bottomleft w3-padding-large">
Powered by
Real Time Data</a>
</div>
</div>
</body>
</html>
my request is very simple, i want to display the scraped data in nodejs (scrape.js) in (index.html)
You should be able to use EJS (or perhaps any other templating engine) for this purpose.
We download the titles to an array, then render using ejs.render.
Make sure you install ejs using
npm install ejs
In your project you'll need to create the following structure:
/index.js
/views/index.ejs
index.js
const request = require('request');
const cheerio = require('cheerio');
const express = require('express');
var app = express();
app.set('view engine', 'ejs');
app.get('/', function(req, res) {
request('https://store.steampowered.com/search/?filter=weeklongdeals', (error, response, html) => {
if(!error && response.statusCode == 200) {
const $ = cheerio.load(html);
let titles = [];
$('.title').each((i,ele) => {
const title = $(ele).text();
console.log(title);
titles.push(title);
});
res.render('index', { titles });
}
})
});
app.listen(8080);
console.log('Express listening on port 8080');
index.ejs
<!DOCTYPE html>
<html>
<title>Real Time Data</title>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
<body style="padding: 2em">
<table class="table table-striped table-bordered">
<tr>
<th>Title</th>
</tr>
<% for (let title of titles) { %>
<tr>
<td><%= title %></td>
</tr>
<% } %>
</table>
</body>
</html>
Then navigate to localhost:8080/ to see your rendered page.
I've created an online example here
What #Terry said, but use get and map rather than pushing elements in an each loop:
let titles = $('.title').get().map(e => $(e).text())
Related
Hello dear community, I have the following problem:
I use nodejs with mysql, express and handlebars.
I am trying to send my MySQL data by an express query (router.get(/chat) from page.js ({data: results})
router.get('/tchat', authController.isLoggedIn, (req, res) => {
console.log(req.user);
if( req.user ) {
db.query("SELECT email from users WHERE name = ?", ["Fada"], function (err, results) {
if(err) throw err;
res.render('tchat', {
user: req.user,
data: results
});
})
} else {
res.redirect('/login');
}
})
The mysql data is sent, but when I retrieve it on my tchat.hbs on the client side, it shows me [Object object] instead of my email. How can I fix this?
I retrieve the data with
<p>{{data}}</p>
tchat.hbs :
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="tchat.css">
<title>Document</title>
</head>
<body>
<p id="pp">{{user.name}}</p>
<div>
<ul id="messages"></ul>
<input id="m" /> <button onclick="send()" id="btn">Send</button>
</div>
<p>unordered list</p>
<p>{{data}}</p>
<script src="https://cdn.socket.io/4.4.1/socket.io.min.js" integrity="sha384-fKnu0iswBIqkjxrhQCTZ7qlLHOFEgNkRmK2vaO/LbTZSXdJfAu6ewRBdwHPhBo/H" crossorigin="anonymous"></script>
<script src="/test.js"></script>
</body>
</html>
Thank you for your answers.
I search for "RowDataPacket" and found this StackOverflow post.
It looks like results is an array of RowDataPacket objects.
Therefore, in your /tchat GET handler, you should be able to access the email like:
data: results[0].email
And then the {{data}} in your template should work.
Additionally, it would be wise to add some protection for the case when your database query returns no rows.
data: results.length ? results[0].email : ''
I'm trying to redirect using nodejs and expressjs, but when I click on button nothing happens only url changes.
I'm using a form and within it has a button, this form has an action to "/failure"
const express = require("express")
const bodyparser = require("body-parser")
const request = require("request")
const app = express()
app.use(express.static("public"))
app.use(bodyparser.urlencoded({
extended: true
}))
app.get("/", function (req, res) {
res.sendFile(__dirname + "/signup.html")
})
app.post("/failure", function(req, res){
res.redirect("/")
})
app.listen(3000, function () {
console.log("Server is running on port 3000")
})
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Failure</title>
<!-- Bootstrap core CSS -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css"
integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
</head>
<body>
<div class="jumbotron jumbotron-fluid">
<div class="container">
<h1 class="display-4">Uh oh!</h1>
<p class="lead">There was a problem signip you up Please try again or contact the developer!.</p>
<form action="/failure" method="POST">
<button class="btn btn-lg btn-warning" type="submit" name="button">Try again</button>
</form>
</div>
</div>
</body>
</html>
Have you tried
res.redirect(307, '/test');
});
This will preserve the send method, for more info you can check http://www.alanflavell.org.uk/www/post-redirect.html
This behavior is correct. You are posting to the '/failure' route and within the handler of that route, it is redirecting to get '/' route handler which will return signup.html - which was your starting point.
You are posting to this:
app.post("/failure", function(req, res){
res.redirect("/") // this is redirecting your route handler '/' which serves 'signup.html
})
I'm trying to use nodejs to read the IP address and display on a html page, here is what I've done so far:
app.js
const express =require('express')
const app = express();
var os = require( 'os' );
var path = require('path')
app.get('/',function(req,res){
res.sendFile(path.join(__dirname+'/index.html'))
})
var networkInterfaces = Object.values(os.networkInterfaces())
.reduce((r,a)=>{
r = r.concat(a)
return r;
}, [])
.filter(({family, address}) => {
return family.toLowerCase().indexOf('v4') >= 0 &&
address !== '127.0.0.1'
})
.map(({address}) => address);
var ipAddresses = networkInterfaces.join(', ')
console.log(ipAddresses);
app.get('/DHCP',(req,res)=>{
return networkInterfaces[1];
});
app.listen(1000)
and the index.html:
<!DOCTYPE html>
<html>
<head>
<meta charset='utf-8'>
<meta http-equiv='X-UA-Compatible' content='IE=edge'>
<title>Page Title</title>
<meta name='viewport' content='width=device-width, initial-scale=1'>
<link rel='stylesheet' type='text/css' media='screen' href='main.css'>
<script src='main.js'></script>
</head>
<body>
<p id="DHCP" align="middle"> DHCP:</p>
</body>
</html>
I'm new to web dev world so I just don't get how can I do it !
thanks in advance !
You need to pick a template engine (e.g. pug), then call res.render() with an html template modified according your template engine's syntax.
use something like this:
<p id="DHCP" align="middle">{{DHCP:}}</p>
This works if you previously set your application to use HTML instead of any View Engine.
In my application i have declared ng-app in Master.html and added all script, stylesheet references in it
this is my master page
<html ng-app="mainApp">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>AdminLTE 2 | Dashboard</title>
<script src="../Angular/angular.min.js"></script>
<script src="../Angular/angular-route.js"></script>
<script src="../Scripts/AngularServices/App.js"></script>
</head>
<body class="hold-transition skin-blue sidebar-mini">
<li><i class="fa fa-circle-o"></i>Group</li>
<li><i class="fa fa-circle-o"></i>Member</li>
<section class="content">
<div ng-view></div>
</section>
</body>
</html>
App.js
var mainApp = angular.module("mainApp", ['ngRoute'])
mainApp.config(function ($routeProvider, $locationProvider) {
$routeProvider.when('/main/Group', { templateUrl: '/Portal/Group.html', controller: 'GroupController' }),
$routeProvider.when('/main/Member', { templateUrl: '/Portal/Member.html', controller: 'MemberController' });
$locationProvider.html5Mode(true);
});
// group
mainApp.controller('GroupController', function ($scope, $http) {
$http.get('/api/APIGroup/GetGroupDetails').then(function (result) {
$scope.group = result.data;
});
});
Group.html
<div ng-controller="GroupController">
<div class="row">
<h1>Welcome to group</h1>
my content here
</div>
</div>
when i execute master page and if click group link group.html form opening inside master page my url like this
http://localhost:50810/main/chitGroup
but if reload page here am getting error as
Server Error in '/' Application.
The resource cannot be found.
Master page not applying to how to fix this
In your angular.app you have ngRoute to handle your states for create Single Page Application.
ngRoute need to pass the state names correctly as ng-href="#!/main/Group", that because when you use ngRoute the url changed automaticly to http://localhost:50810/#!/
Server Error in '/' Application.
The resource cannot be found.
This error because you redirect to http://localhost:50810 to find /main/Group which not exist.
I have small JSON data from server and wanted to show in html file. unfortunately not working. Any idea?
I have invested couple of hours to find these. but still not found.. :(
Thanks for your feedback.
JS File:
(function() {
var app = angular.module('myreddit', ['ionic']);
app.controller('RedditCtrl', function($scope, $http) {
$scope.conditions = [];
$http.get('http://50wave.com/deesh.json')
.success(function(response) {
angular.forEach(response.ht, function(child) {
$scope.stories.push(child);
});
});
});
app.run(function($ionicPlatform) {
$ionicPlatform.ready(function() {
if(window.cordova && window.cordova.plugins.Keyboard) {
cordova.plugins.Keyboard.hideKeyboardAccessoryBar(true);
}
if(window.StatusBar) {
StatusBar.styleDefault();
}
});
});
}());
html file:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="initial-scale=1, maximum-scale=1, user-scalable=no, width=device-width">
<title></title>
<link href="lib/ionic/css/ionic.css" rel="stylesheet">
<link href="css/style.css" rel="stylesheet">
<script src="lib/ionic/js/ionic.bundle.js"></script>
<script src="cordova.js"></script>
<script src="js/app.js"></script>
</head>
<body ng-app="myreddit" ng-controller="RedditCtrl">
<ion-pane>
<ion-header-bar class="bar-positive">
<h1 class="title">FootBall</h1>
</ion-header-bar>
<ion-content>
<div class="list">
<div ng-repeat="condition in conditions">
{{conditions.name}}
</div>
</ion-content>
</ion-pane>
</body>
</html>
Thanks for your feedback.
There might be more issues with your application, you did not provide enough data to troubleshoot, but from the code you have posted I see at least these 2:
1 you are pushing to stories array, which doesn't seem to be read in your application. I think it should be:
$http.get('http://50wave.com/deesh.json')
.success(function(response) {
angular.forEach(response.ht, function(child) {
$scope.conditions.push(child);
});
});
2 when inside ng-repeat cycle, you should be addressing the item, not the collection, so not {{conditions.name}} but {{condition.name}}, so the code should be:
<div ng-repeat="condition in conditions">
{{condition.name}}
</div>