I would like to be able to scrape some data on a website with login. I use this to get login cookie :
var options =
{
"method" : "post",
"muteHttpExceptions" : true,
"payload":{"login[email]" : email,
"login[password]" : password
}
};
var response = UrlFetchApp.fetch(adresse,options);
var sessionDetails = response.getAllHeaders();
Logger.log(response.getContentText());
But I always have error (I try different things but always the same error..) here the log:
Infos <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /</pre>
</body>
</html>
It would be nice if someone could help! I found some post where people say: I m able to login but... and when I try the code they give its not working!
Thanks!
As I said in my comment, this is code you can add to your current code for curl, I added it into an answer because it has better formatting
curl --include \
--request POST \
'***your url here***'
Related
I'll make the succeeding programs which transfers URL defined by GAS to html.
In this time, I set URL as fix letters for simplification, however,
in the actual situations, URL is generated uniquely in every trial.
function getUrl() {
var url = "https://www.google.co.jp/"
var html = HtmlService.createTemplateFromFile("dialog2").evaluate();
SpreadsheetApp.getUi().showModalDialog(html, "Download File");
}
// error ReferenceError: url is not defined # test.gs:3
html side (dialog2.html)
<!DOCTYPE html>
<html>
<head>
<base target="_top">
</head>
<body>
Click here
<!--↑wanting to get url as a variable letters from the GAS program-->
</body>
</html>
I'm sure that the way of transfering the parameter (this time,url) is the crucial fautor to compile these codes.
If we cannnot get this plan into the practice, please tell me sub plans.
Thank you for checking this question.
I solved this problem by defining the paramerter url as property.
correct code (GAS)
function getUrl() {
var url = "https://www.google.co.jp/"
var html = HtmlService.createTemplateFromFile("dialog2");
html.url = url;
SpreadsheetApp.getUi().showModalDialog(html.evaluate(), "Download File");
}
I have a minimalist socket.io with running on Node.js with a HTML client.
This is my minimalist server code (server.js):
const io = require('socket.io')(3000);
io.on('connection', (socket) => {
console.log(`new connection : ${socket.id}`);
});
and my minimalist client code (client.html):
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/3.1.3/socket.io.js"></script>
<script>
var socket = io('http://localhost:3000/');
</script>
<title>Document</title>
</head>
<body>
</body>
</html>
When I open client.html I get the message:
new connection : DedBsiEyAHLIu86oAAB0
which is fine, however after 7-8 seconds I get other new connection messages with differents socket ID :
new connection : K3DUMN7SJ0tOU7TsAACO
new connection : 3kCOVCz9fwRWtIypAACP
new connection : HgMVgpeP7raq1c4YAACQ
new connection : U6-lD-3lT1vT39_oAACR
new connection : aQCKYpBUVXD7t8WIAACS
On Chrome devtools-network tab, the same repeating message is displayed (which populate the new connection) :
Request URL: http://localhost:3000/socket.io/?EIO=4&transport=polling&t=NYCWok4
Request Method: GET
Status Code: 200 OK
Remote Address: [::1]:3000
Referrer Policy: strict-origin-when-cross-origin
Your help is greatly appreciated.
Regards
Pierre.
I realized that I was using a CORS enable plug-in with Chrome, I disable it which solve my problem
I am trying to create a very basic static web server that records the IP addresses of each client machine served.
By and large, it is working... but the web site as served has a strange header... b8:
This header doesn't show up in my html at all, and I'm rather confused.
index.html code:
<!DOCTYPE html>
<html>
<head>
<title>Science Treasure Hunt</title>
</head>
<body>
<p>Eventual Home of Science Treasure Hunt Webpage</p>
</body>
</html>
My guess is that it is somewhere on the node.js side of things, but I don't know what could be causing it from that end, either...
index.js code:
let http = require('http');
let requestIp = require('request-ip');
let winston = require('winston');
let static = require('node-static');
http.createServer(onRequest).listen(80);
let logger = winston.createLogger({
level: 'info',
format: winston.format.combine(
winston.format.timestamp(),
winston.format.printf(info => {
return `${info.timestamp} ${info.level}: ${info.message}`;
})
),
transports: [
new winston.transports.Console(),
new winston.transports.File({filename: 'firstlog.log'})
]
});
var file = new(static.Server)('./public');
function onRequest(request, response) {
file.serve(request, response);
var ip = request.headers['x-forwarded-for'] || request.connection.remoteAddress;
logger.log('info', ip);
response.writeHead(200);
}
What might be happening here?
I'm trying to use oEmbed to test if a Youtube or Vimeo video exists. My code works fine with Youtube, but it doesn't work with Vimeo, even if I follow Vimeo's official documentation for oEmbed and the example provided there. Why is my code not working for Vimeo? I get a server response 200 for Youtube, but a server response 0 for Vimeo. My code is:
<?php
//Problematic case: I get 0 as a server reponse
$cURL = curl_init("https://vimeo.com/api/oembed.json?url=https%3A//vimeo.com/76979871");
// Youtube case - works fine: server response = 200
// $cURL = curl_init("http://www.youtube.com/oembed?url=https://www.youtube.com/watch?v=ebXbLfLACGM");
// Set option 1: return the result as a string
curl_setopt($cURL, CURLOPT_RETURNTRANSFER, true);
// Set option 2: Follow any redirect
curl_setopt($cURL, CURLOPT_FOLLOWLOCATION, true);
// Execute the query
$cURLresult = curl_exec($cURL);
// Get the HTTP response code
$response = curl_getinfo($cURL, CURLINFO_HTTP_CODE);
?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Server response</title>
</head>
<body>
<h1><?php print "Server response: " . $response; ?></h1>
</body>
</html>
The problem came from using localhost to connect to Vimeo. Localhost works fine with Youtube, but not with Vimeo. I've test the above script in a real server, as suggested by another member, and there everything works fine.
Moral of the history: always test problematic external connections in a real server, not on localhost.
Following is the content of my JSON File -
{
"tags": [
"Red",
"Green",
"Blue",
"Yellow"
]
}
I checked this with jsonlint but still I am getting the following error in firefox.
Timestamp: Wednesday 18 June 2014 10:39:41 IST Error: not well-formed
Source File:
file:///home/trialcode/trialcode/Projects/ang/18-06-2014/ang/content.json
Line: 1, Column: 1 Source Code: {
I am not sure what I am doing wrong if any.
FYI -
OS - Linux Ubuntu 12.04
Firefox - 24.0
EDIT
I am using the content of content.json file in angular controller via $http.get method.
I explored more about this kind of error and found it is related with the Content-Type setting.
Following is the full code -
<!doctype html>
<html lang="en" ng-app="app">
<head>
<title>JSON Read In Angularjs</title>
<meta charset="UTF-8" />
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.10/angular.min.js"></script>
<script>
var app = angular.module('app', []);
app.controller('myCtrl', function($scope, $http){
$scope.data = {};
$http.get('content.json').success(function(data) {
$scope.data = data;
});
});
</script>
</head>
<body ng-controller="myCtrl">
<ul>
<li ng-repeat="em in data.tags">{{em}}</li>
</ul>
</body>
</html>
How do I set the content type if that is a problem. I searched HERE but unable to fix it. Help me Please if any.
After few hours of searching I came across that -
Chrome and other modern browsers have implemented security
restrictions for Cross Origin Requests, which means that you cannot
load anything through file:/// , you need to use http:// protocol at
all times, even locally -due Same Origin policies.
Source -
Cross Origin Script Stackoverflow Answer
Simple Solution For Local Cross Origin Requests