I'm working on an extension that intercepts network requests using the chrome.devtools.network API.
The extension is written with React.
Currently, the interception is done on the devTools panel, so the interception started only when the user enter my extension.
I can move the interception logic to the devTools but I wonder how to pass the data to the panel dynamically, so the React component on the panel will be updated with the new data and can show it.
Any suggestion?
I've solved it by doing something similar to this answer like this:
devTools.tsx:
chrome.devtools.panels.create('Apex Network', "", 'panel.html', function (extensionPanel) {
var _window; // Going to hold the reference to panel.html's `window`
let reqs = [];
const requestHandler = request => {
reqs.push(request);
};
chrome.devtools.network.onRequestFinished.addListener(requestHandler)
extensionPanel.onShown.addListener(function tmp(panelWindow) {
extensionPanel.onShown.removeListener(tmp); // Run once only
chrome.devtools.network.onRequestFinished.removeListener(requestHandler)
_window = panelWindow;
_window.INITIAL_DATA = reqs;
}) })
Panel.tsx
declare global {
interface Window { INITIAL_DATA: any; }
}
window.INITIAL_DATA = window.INITIAL_DATA || {};
const Panel = () => {
const requestHandler = (request) => {...}
useEffect(()=>{
window.INITIAL_DATA?.forEach(requestHandler)
window.INITIAL_DATA = [];
chrome.devtools.network.onRequestFinished.addListener(requestHandler)
});
}
So in the end, it's done by doing that:
When the devtools is open, the devtools register to the network requests and store them in an array.
When the devtools extension is open, the devtools unregister for the network requests and the requests is passed to the panel and the devtools stop capturing requests
The panel handle the request that captured at the devtools
The panel register to the network requests
Thanks
Related
I'm scraping data from Google using Puppeteer. But before Puppeteer gets to the google page, an annoying popup appears (screenshot).
I want to prevent this from happening so that I don't have to click the "reject" / "allow" button in Puppeteer every time. What are the necessary cookies to achieve it?
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto("https://google.com/")
const cookies = [
{
domain: '.google.com',
expirationDate: 9999999999.298648,
name: 'CONSENT',
...
},
]
await page.setCookie(...cookies)
Instead of trying to hard code the cookies, just save them.
Come up with a file path where the cookies file will be saved, e.g. cookiesFilePath.
After launching puppeteer, check to see if a file exists at the path. If it exists load it.
Keep your code for navigating the consent form, as you will use it at least once. At the end of the accepting/rejecting, save page.cookies to cookiesFilePath
const cookiesPath = './cookies.json';
const browser = await puppeteer.launch()
const page = await browser.newPage()
if(await fs.existsSync(cookiesPath)){
let cookies = require(cookiesPath)
for(let cookie of cookies){
await page.setCookie(cookie)
}
}
await page.goto("https://google.com/")
const hasConsentForm = async ()=>{
// create query that will return consent form
const consentFormQuerySelector=''
// I dont remember if page.$eval querySelector fails
// throws errors or not
try{
return await page.$eval(consentFormQuerySelector,el=>Boolean(el))
}catch(err){
console.log(err)
return false
}
}
const navigateConsentForm = async ()=>{
// add your existing code that navigates the consent form
// ...
// after the consent form is cleared, save cookies
let cookies = await page.cookies()
fs.writeFileSync(cookiesPath,JSON.stringify(cookies,null,2))
}
if(await hasConsentForm()){
await navigateConsentForm()
}
Assuming that the lack of cookies or the lack of a certain cookie property is the reason that the consent form pops up, this is will make it so that after running once, the form doesnt show again.
Looking into a way of sharing data via Google App Scripts's Cache Services from one web app to another.
Users load up the first webpage and filled out their information. Once submitted a function is run on this data and stored via the cache.
CacheService.getUserCache().put('FirstName','David')
CacheService.getUserCache().put('Surname','Armstrong')
Console log shows reports back that these two elements have been saved to cache.
However in the second web app when cache is called upon the console log returns null
var cache = CacheService.getUserCache().get('Firstname');
var cache2 = CacheService.getUserCache().get('Surname');
console.log(cache)
console.log(cache2)
Any ideas?
A possible solution would be to implement a service to synchronize the cache between web apps.
This can be achieved by creating a WebApp that via POST allows to add to the ScriptCache of the "Cache Synchronizer" the UserCache of the individual Web Apps.
The operation would be very simple:
From the web app that we want to synchronize, we check if we have cache of the user.
If it exists, we send it to the server so that it stores it.
If it does not exist, we check if the server has stored the user's cache.
Here is a sketch of how it could work.
CacheSync.gs
const cacheService = CacheService.getScriptCache()
const CACHE_SAVED_RES = ContentService
.createTextOutput(JSON.stringify({ "msg": "Cache saved" }))
.setMimeType(ContentService.MimeType.JSON)
const doPost = (e) => {
const { user, cache } = JSON.parse(e.postData.contents)
const localCache = cacheService.get(user)
if (!localCache) {
/* If no local data, we save it */
cacheService.put(user, JSON.stringify(cache))
return CACHE_SAVED_RES
} else {
/* If data we send it */
return ContentService
.createTextOutput(JSON.stringify(localCache))
.setMimeType(ContentService.MimeType.JSON)
}
}
ExampleWebApp.gs
const SYNC_SERVICE = "<SYNC_SERVICE_URL>"
const CACHE_TO_SYNC = ["firstName", "lastName"]
const cacheService = CacheService.getUserCache()
const syncCache = () => {
const cache = cacheService.getAll(CACHE_TO_SYNC)
const options = {
method: "POST",
payload: JSON.stringify({
user: Session.getUser().getEmail(),
cache
})
}
if (Object.keys(cache).length === 0) {
/* If no cache try to fetch it from the cache service */
const res = UrlFetchApp.fetch(SYNC_SERVICE, options)
const parsedResponse = JSON.parse(JSON.parse(res.toString()))
Object.keys(parsedResponse).forEach((k)=>{
console.log(k, parsedResponse[k])
cacheService.put(k, parsedResponse[k])
})
} else {
/* If cache send it to the sync service */
const res = UrlFetchApp.fetch(SYNC_SERVICE, options)
console.log(res.toString())
}
}
const createCache = () => {
cacheService.put('firstName', "Super")
cacheService.put('lastName', "Seagull")
}
const clearCache = () => {
cacheService.removeAll(CACHE_TO_SYNC)
}
Additional information
The synchronization service must be deployed with ANYONE access. You can control the access via an API_KEY.
This is just an example, and is not fully functional, you should adapt it to your needs.
The syncCache function of the web App is reusable, and would be the function you should use in all Web Apps.
There is a disadvantage when retrieving the cache, since you must provide the necessary keys, which forces you to write them manually (ex CACHE_TO_SYNC).
It could be considered to replace ScriptCache with ScriptProperties.
Documentation
Cache
Properties
Session
The doc says:
Gets the cache instance scoped to the current user and script.
As it is scoped to the script, accessing from another script is not possible. This is also the case with PropertiesService:
Properties cannot be shared between scripts.
To share, you can use a common file shared between them, like a drive text file or a spreadsheet.
Is there any way to port a complete Web App (which is already responsive and fully compatible with small screens, already has touch UI controls, etc.) to Android/iOS?
My Web App is barebone HTML/JS/CSS, so is super vanilla (I don't even use jQuery).
I thought I could just smash my web app into an empty Ionic-Cordova project and be good with it, but I was wondering is there is a faster/better way to do this?
Maybe a tool or service i don't know about that takes as input a folder and pops out an android/IOS executable?
You can make a PWA (Progressive Web App).
Progressive Web Apps (PWAs) are modern, high quality applications built using web technology. PWAs offer similar capabilities to iOS/Android/desktop apps, they are reliable even in unstable network conditions, and are installable making it easier for users to find and use them.
Basically you have to add a manifest file in .json to your project root where you'll inform many things about your App like icon, name, main color, display mode (choose standalone if you want it to be like an real app) and etc...
(see it here: https://web.dev/add-manifest/) and link to your project pages:
<link rel="manifest" href="/manifest.json">
After that you have to make it installable (https://web.dev/codelab-make-installable/), to do that you will need a service-worker script in your project, you can get one here (https://glitch.com/edit/#!/make-it-installable?path=service-worker.js%3A1%3A0)
const CACHE_NAME = 'offline';
const OFFLINE_URL = 'offline.html';
self.addEventListener('install', function(event) {
console.log('[ServiceWorker] Install');
event.waitUntil((async () => {
const cache = await caches.open(CACHE_NAME);
// Setting {cache: 'reload'} in the new request will ensure that the response
// isn't fulfilled from the HTTP cache; i.e., it will be from the network.
await cache.add(new Request(OFFLINE_URL, {cache: 'reload'}));
})());
self.skipWaiting();
});
self.addEventListener('activate', (event) => {
console.log('[ServiceWorker] Activate');
event.waitUntil((async () => {
// Enable navigation preload if it's supported.
// See https://developers.google.com/web/updates/2017/02/navigation-preload
if ('navigationPreload' in self.registration) {
await self.registration.navigationPreload.enable();
}
})());
// Tell the active service worker to take control of the page immediately.
self.clients.claim();
});
self.addEventListener('fetch', function(event) {
// console.log('[Service Worker] Fetch', event.request.url);
if (event.request.mode === 'navigate') {
event.respondWith((async () => {
try {
const preloadResponse = await event.preloadResponse;
if (preloadResponse) {
return preloadResponse;
}
const networkResponse = await fetch(event.request);
return networkResponse;
} catch (error) {
console.log('[Service Worker] Fetch failed; returning offline page instead.', error);
const cache = await caches.open(CACHE_NAME);
const cachedResponse = await cache.match(OFFLINE_URL);
return cachedResponse;
}
})());
}
});
Just add and save it in .js file in your project.
After that make sure you register the service worker using that code in your project:
/* Only register a service worker if it's supported */
// Service Worker
window.addEventListener('load', () => {
if ('serviceWorker' in navigator) {
navigator.serviceWorker.register('/service-worker.js');
}
});
Now you can make your site installable via some <button> element for example:
window.addEventListener('beforeinstallprompt', (event) => {
// Get the event first
window.deferredPrompt = event;
});
document.querySelector('#buttonInstall').addEventListener('click', () => {
const promptEvent = window.deferredPrompt;
if (! promptEvent) {
return;
}
promptEvent.prompt();
promptEvent.userChoice.then((result) => {
window.deferredPrompt = null;
});
}
});
You can hide the install button when people are in your PWA this way:
if (! window.matchMedia('(display-mode: standalone)').matches) {
// hide your install button
}
Here is some important things:
Your app have to meets certain criteria to be installable, you can
see it here: https://web.dev/install-criteria/
If the install pop-up doesn't appear, it means you made something
wrong, or your manifest is broken or your script.
You can see if your manifest.json is ok in browser developer tools open it (F12), go to Application tab and go to Manifest, this will show all your manifest parameters and it will show if something is wrong too.
I recommend you to read all the links above, there is a lot more
details an explanation about PWAs
simple way to port web app to mobile app is to make a WebView app in android. then give it your web app link address
I built a website that uses the Chrome web midi interface (based on navigator.requestMidiAccess) that works fine in a local development server, but when pushed to a cloud server fails, saying that navigator.requestMidiAccess is not a function. The same code, the same browser. I'll try to include the relevant code:
function initializeMidi() {
navigator.requestMIDIAccess()
.then(
(midi) => midiReady(midi),
(err) => console.log('Something went wrong', err));
}
window.onload = (event) => {
initializeMidi();
};
// this next function builds a list of radio buttons to select the MIDI device
function midiReady(midi) {
globalMidi = midi.outputs
parentElement = document.getElementById('midi-devices-div')
parentElement.innerHTML = ''
var lastMidiPortName = null
midi.outputs.forEach(function (port, key) {
addRadioButton(parentElement, port)
lastMidiPortName = port.name
})
var n = window.localStorage.getItem('selectedMidiPortName')
if (n)
{
var e = document.getElementById(n)
e.checked = true
}
}
The Web MIDI interface is only exposed to SecureContexts, you must serve your document using https://.
I need to execute a script in every Window object created in Chrome – that is:
tabs opened through puppeteer
links opened by click()ing links in puppeteer
all the popups (e.g. window.open or "_blank")
all the iframes contained in the above
it must be executed without me evaluating it explicitly for that particular Window object...
I checked Chrome's documentation and what I should be using is Page.addScriptToEvaluateOnNewDocument.
However, it doesn't look to be possible to use through puppeteer.
Any idea? Thanks.
This searches for a target in all browser contexts.
An example of finding a target for a page opened
via window.open() or popups:
await page.evaluate(() => window.open('https://www.example.com/'))
const newWindowTarget = await browser.waitForTarget(async target => {
await page.evaluate(() => {
runTheScriptYouLike()
console.log('Hello StackOverflow!')
})
})
via browser.pages() or tabs
This script run evaluation of a script in the second tab:
const pageTab2 = (await browser.pages())[1]
const runScriptOnTab2 = await pageTab2.evaluate(() => {
runTheScriptYouLike()
console.log('Hello StackOverflow!')
})
via page.frames() or iframes
An example of getting eval from an iframe element:
const frame = page.frames().find(frame => frame.name() === 'myframe')
const result = await frame.evaluate(() => {
return Promise.resolve(8 * 7);
});
console.log(result); // prints "56"
Hope this may help you