I can't run whatsapp on my electron browser even after setting useragent to the latest chrome version
if someone have solution please tell
It is necessary to remove:
ResponseHeader => "X-Frame-Options"
RequestHeader => "Sec-Fetch-Dest"
I think Whatsapp doesn't allow iframes, so you have to remove that information for it to work properly
In Main process
const mainWindow = new BrowserWindow({
webPreferences: {
nodeIntegration: true,
contextIsolation: false,
},
});
mainWindow.webContents.session.webRequest.onHeadersReceived(
{ urls: ['https://web.whatsapp.com/'] },
(details: any, callback) => {
if (details && details.responseHeaders['X-Frame-Options']) {
delete details.responseHeaders['X-Frame-Options'];
} else if (details.responseHeaders['x-frame-options']) {
delete details.responseHeaders['x-frame-options'];
}
callback({ cancel: false, responseHeaders: details.responseHeaders });
});
mainWindow.webContents.session.webRequest.onBeforeSendHeaders(
{ urls: ['https://web.whatsapp.com/'] },
(details, callback) => {
details.requestHeaders['User-Agent'] = userAgent;
details.requestHeaders['Access-Control-Allow-Origin'] = '*';
if (details.requestHeaders['Sec-Fetch-Dest']) {
delete details.requestHeaders['Sec-Fetch-Dest'];
}
callback({ cancel: false, requestHeaders: details.requestHeaders });
});
In Remote process
<iframe src="https://web.whatsapp.com/" />
You can use different user agent like this:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) old-airport-include/1.0.0 Chrome Electron/7.1.7 Safari/537.36
Related
This is code
const browser = await puppeteer.launch({
headless: false,
timeout: 0,
defaultViewport: null,
args: [
"--no-sandbox",
"--disable-setuid-sandbox",
"--start-maximized",
"--ignore-certificate-errors",
],
ignoreDefaultArgs: ["--enable-automation"],
});
const page = await browser.newPage();
await page.setUserAgent(
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
);
// set download path
const client = await page.target().createCDPSession();
await client.send("Page.setDownloadBehavior", {
behavior: "allow",
downloadPath: "D:\\Download",
});
// open uri
await page.goto(
"https://translate.google.com.hk/?hl=zh-CN&sourceid=cnhp&sl=en&tl=zh-CN&op=docs",
{
waitUntil: "networkidle2",
}
);
// upload pdf docuemnt
const [fileChooser] = await Promise.all([
page.waitForFileChooser(),
page.click("label"),
]);
await fileChooser.accept(["D:\\test.pdf"]);
// click translate button
const button = await page.waitForSelector(
"div[jsname='itaskb'] > div > button"
);
await button.evaluate((b) => b.click());
// click download button
const button2 = await page.waitForSelector(
"div[jsname='itaskb'] > button",
{
visible: true,
timeout: 0,
}
);
await button2.evaluate((b) => b.click());
The whole process is the same as my manual operation. But the translated document after download is not zh-CN, but the same as the uploaded document, which is en.
What happened? How do I proceed to get the translation I want.
I've trying to deploy puppeteer on Render.com,
I can do other requests but puppeteer does not seem to work,
did I do something wrong
async function startBrowser() {
let browser;
try {
console.log("Opening the browser......");
browser = await puppeteer.launch({
headless: false,
ignoreDefaultArgs: ["--disable-extensions"],
args: [
"--no-sandbox",
"--use-gl=egl",
"--disable-setuid-sandbox",
],
ignoreHTTPSErrors: true,
});
} catch (err) {
console.log("Could not create a browser instance => : ", err);
}
return browser;
}
I realized that I forgot to switch back to headless : true, and in headless : true I have to set user agent for that page. Example:
await page.setUserAgent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36")
I've been trying to log in to this website using my credentials in order to scrape my profile name using google apps script. The status code is 200 and I can see that the script is able to get cookies. However, I get Undefined as result instead of profile name.
This is how I'm trying:
function loginAndParseProfile() {
var link = 'https://stackoverflow.com/users/login?ssrc=head&returnurl=https%3a%2f%2fstackoverflow.com%2f';
var options = {
"method": "get",
"headers": {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36"
}
};
var res = UrlFetchApp.fetch(link, options);
var $ = Cheerio.load(res.getContentText());
var fkey = $("input[name='fkey']").first().attr('value');
var payload = {
'fkey': fkey,
'ssrc': 'head',
'email': 'emailaddress',
'password': 'password',
'oauth_version': '',
'oauth_server': ''
};
var options = {
"method" : "post",
'payload': payload,
'muteHttpExceptions': true,
"headers": {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36",
}
};
var loginURL = "https://stackoverflow.com/users/login?ssrc=head&returnurl=https%3a%2f%2fstackoverflow.com%2f";
var resp = UrlFetchApp.fetch(loginURL,options);
console.log(resp.getResponseCode());
console.log(resp.getAllHeaders()['Set-Cookie']);
var $ = Cheerio.load(resp.getContentText());
var item = $('a.my-profile > [class^="gravatar-wrapper"]').first().attr('title');
console.log(item);
}
How can I make the script work?
Disable redirects by setting followRedirects to false:
var options = {
"method" : "post",
'payload': payload,
'muteHttpExceptions': true,
"headers": {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36",
},
'followRedirects': false
};
Grab the acct cookie from the response to the POST /users/login request:
const acct = resp.getAllHeaders()['Set-Cookie']
.find(cookie => cookie.includes('acct=t='))
.match(/(acct=t=.*?)\s/)[1];
Make a GET / request supplying the acct cookie and grab your profile name:
const profileRequest = UrlFetchApp.fetch('https://stackoverflow.com', {
method: 'get',
headers: {
Cookie: acct
}
});
const $main = Cheerio.load(profileRequest.getContentText());
const myName = $main('a.my-profile > [class^="gravatar-wrapper"]').first().attr('title');
console.log(myName);
If your credentials are correct, this should output robots.txt.
I want to get html from a web. But it show like that.
meta http-equiv=refresh content="0;url=http://www.skku.edu/errSkkuPage.jsp">
But when I use https://www.naver.com/ instead of https://www.skku.edu/skku/index.do, it works well.
I want to know the reason.
Here's my code.
var request = require('request');
const url = "https://www.skku.edu/skku/index.do";
request(url, function(error, response, body){
if (error) throw error;
console.log(body);
});
The website blocks the request that is coming from programmatic script checking User-Agent in the request header.
Pass the user-Agent that web-browser(eg: Google chrome) sends and it should work.
var request = require('request');
var options = {
'method': 'GET',
'url': 'https://www.skku.edu/skku/index.do',
'headers': {
'User-Agent': ' Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36'
}
};
request(options, function (error, response) {
if (error) throw new Error(error);
console.log(response.body);
});
I wouldn't recommend request module as it is not maintained for changes anymore. see it here - https://github.com/request/request/issues/3142
You could look for alternatives in form of got, axios etc which makes code much more readable and clear. And most important thing - Native support for promises and async/await The above code will look like
var got = require('got');
const url = "https://www.skku.edu/skku/index.do";
(async () => {
const response = await got(url);
console.log(response.body);
})();
I have the following route:
Loads.TestRoute = Ember.Route.extend({
model: function() {
return this.store.find('load');
}
});
This to the best of my knowledge will return all instances of load in the data store, which in this case can be anywhere from 1 to 100. For this application, I am using a local storage adapter.
My controller looks like this:
Loads.TestController = Ember.ArrayController.extend({
actions: {
test: function () {
var loads = this.get('model');
var driverId = getCookie("id");
this.store.find("driver", driverId).then(function (driver,loads) {
$.ajax({
type: "POST",
data: JSON.stringify({ Driver: driver, Loads: loads }),
url: "api/build",
contentType: "application/json",
success: function (message) {
alert(message);
}
});
});
}
}
});
What I'm trying to accomplish is to send all instances of the model 'load' along with a specific instance of the model driver as a JSON object to an API on my server that builds spreadsheets.
When I run this, I can see in the request payload that the Driver model object is turned into JSON, but the Loads are not. Here is what is in the payload:
Remote Address:[::1]:49438
Request URL:http://localhost:49438/api/build
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:66
Content-Type:application/json
Cookie:id=atcn4
Host:localhost:49438
Origin:http://localhost:49438
Referer:http://localhost:49438/
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
X-Requested-With:XMLHttpRequest
Request Payloadview source
{Driver: {firstName: "Ron", lastName: "Burgandy", truck: "12"}}
How can I update this to make it so both the driver and loads models are sent in the Payload?
Thanks for any help in advance!
You need to make sure both promises from your store are resolved before you send off your ajax request. Use Ember.RSVP.hash
Loads.TestController = Ember.ArrayController.extend({
actions: {
test: function () {
var driverId = getCookie("id");
Ember.RSVP.hash({
loads: this.store.find('load'),
driver: this.store.find('driver', driverId)
}).then(function(data) {
$.ajax({
type: "POST",
data: JSON.stringify({ Driver: data.driver, Loads: data.loads }),
url: "api/build",
contentType: "application/json",
success: function (message) {
alert(message);
}
});
});
}
}
});