How to get "Coverage" data out from the Chrome Dev Tools - google-chrome

I am using the Coverage tab at my Chrome Dev Tools and I have a really big file and after playing a lot with Coverage it's clear enough that only 15% enough of my CSS code is being used (I simulated button presses, hover menus...).
The problem is getting hat 15% of code OUT of the Coverage tab. I cant believe the Devs behind this really nice feature didnt think an easy way for the end user copy only the green part of the code. Check image attached.
Do you have any idea how I could do that? I read something about using Puppeteers but it requires lots of preparation. On latest Canary version it looks like I can export a JSON but it would require some time to code a parser to that JSON in order to extract only the needed part.

Thanks to an article by Phillip Kriegel (https://www.philkrie.me/2018/07/04/extracting-coverage.html) I managed to setup Puppeteer to extract the coverage CSS from a URL and output that CSS into a file.
Here's how to do it:
Step 1: Install node.js globally
Step 2: Create a folder on your desktop
Step 3: Inside the folder install the Node Package Manager (NPM) and the Puppeteer node module
Step 4: Create a JavaScript file inside the folder, name it coverage.js
Step 5: Put this code inside that js file:
const puppeteer = require('puppeteer');
// Include to be able to export files w/ node
const fs = require('fs');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
// Begin collecting CSS coverage data
await Promise.all([
page.coverage.startCSSCoverage()
]);
// Visit desired page
await page.goto('https://www.google.com');
//Stop collection and retrieve the coverage iterator
const cssCoverage = await Promise.all([
page.coverage.stopCSSCoverage(),
]);
//Investigate CSS Coverage and Extract Used CSS
const css_coverage = [...cssCoverage];
let css_used_bytes = 0;
let css_total_bytes = 0;
let covered_css = "";
for (const entry of css_coverage[0]) {
css_total_bytes += entry.text.length;
console.log(`Total Bytes for ${entry.url}: ${entry.text.length}`);
for (const range of entry.ranges){
css_used_bytes += range.end - range.start - 1;
covered_css += entry.text.slice(range.start, range.end) + "\n";
}
}
console.log(`Total Bytes of CSS: ${css_total_bytes}`);
console.log(`Used Bytes of CSS: ${css_used_bytes}`);
fs.writeFile("./exported_css.css", covered_css, function(err) {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
await browser.close();
})();
Step 6: BE SURE TO REPLACE the URL at this point in the code await page.goto('https://www.google.com'); with your desired URL
Step 7: In the command line tool (Git Bash) type node coverage.js
A file called exported_css.css will be created, it will contain all your coverage CSS for the URL you set in the code.
CAVEAT: This will extract the coverage CSS from ALL the CSS assets that are loaded from the URL you set. You will then have to further optimize that CSS (not covered in this example).

Open Chrome Tab --> Inspect Element (F12) --> Press Escape button

I'm in the process of creating a PHP script that parses the Coverage JSON exported file, and outputs the extracted used CSS/JS only. Unfortunately I have come across a snag, at some point the JSON parser loses the correct character number, and I end up with broken or incorrect CSS/JS syntax. It's only off by a few characters, but the amount of characters that it is off by is variable so it's almost impossible to predict it during parsing.
I'm not positive, but I think the issue results from PHP running on the server, and the server may read the characters in the original CSS file differently than a browser would. I'm going to attempt to write a Coverage JSON parser in JavaScript. When I do I'll be sure to post the code here for all to use.
Sorry I couldn't be of more help, I just wanted to warn away people from using PHP to do this as it seems to not read character numbers correctly in large CSS files.

Related

Snapchat download all memories at once

Over the years on snapchat I have saved lots of photos that I would like to retrieve now, The problem is they do not make it easy to export, but luckily if you go online you can request all the data (thats great)
I can see all my photos download link and using the local HTML file if I click download it starts downloading.
Here's where the tricky part is, I have around 15,000 downloads I need to do and manually clicking each individual one will take ages, I've tried extracting all of the links through the download button and this creates lots of Urls (Great) but the problem is, if you past the url into the browser then ("Error: HTTP method GET is not supported by this URL") appears.
I've tried a multitude of different chrome extensions and none of them show the actually download, just the HTML which is on the left-hand side.
The download button is a clickable link that just starts the download in the tab. It belongs under Href A
I'm trying to figure out what the best way of bulk downloading each of these individual files is.
So, I just watched their code by downloading my own memories. They use a custom JavaScript function to download your data (a POST request with ID's in the body).
You can replicate this request, but you can also just use their method.
Open your console and use downloadMemories(<url>)
Or if you don't have the urls you can retrieve them yourself:
var links = document.getElementsByTagName("table")[0].getElementsByTagName("a");
eval(links[0].href);
UPDATE
I made a script for this:
https://github.com/ToTheMax/Snapchat-All-Memories-Downloader
Using the .json file you can download them one by one with python:
req = requests.post(url, allow_redirects=True)
response = req.text
file = requests.get(response)
Then get the correct extension and the date:
day = date.split(" ")[0]
time = date.split(" ")[1].replace(':', '-')
filename = f'memories/{day}_{time}.mp4' if type == 'VIDEO' else f'memories/{day}_{time}.jpg'
And then write it to file:
with open(filename, 'wb') as f:
f.write(file.content)
I've made a bot to download all memories.
You can download it here
It doesn't require any additional installation, just place the memories_history.json file in the same directory and run it. It skips the files that have already been downloaded.
Short answer
Download a desktop application that automates this process.
Visit downloadmysnapchatmemories.com to download the app. You can watch this tutorial guiding you through the entire process.
In short, the app reads the memories_history.json file provided by Snapchat and downloads each of the memories to your computer.
App source code
Long answer (How the app described above works)
We can iterate over each of the memories within the memories_history.json file found in your data download from Snapchat.
For each memory, we make a POST request to the URL stored as the memories Download Link. The response will be a URL to the file itself.
Then, we can make a GET request to the returned URL to retrieve the file.
Example
Here is a simplified example of fetching and downloading a single memory using NodeJS:
Let's say we have the following memory stored in fakeMemory.json:
{
"Date": "2022-01-26 12:00:00 UTC",
"Media Type": "Image",
"Download Link": "https://app.snapchat.com/..."
}
We can do the following:
// import required libraries
const fetch = require('node-fetch'); // Needed for making fetch requests
const fs = require('fs'); // Needed for writing to filesystem
const memory = JSON.parse(fs.readFileSync('fakeMemory.json'));
const response = await fetch(memory['Download Link'], { method: 'POST' });
const url = await response.text(); // returns URL to file
// We can now use the `url` to download the file.
const download = await fetch(url, { method: 'GET' });
const fileName = 'memory.jpg'; // file name we want this saved as
const fileData = download.body; // contents of the file
// Write the contents of the file to this computer using Node's file system
const fileStream = fs.createWriteStream(fileName);
fileData.pipe(fileStream);
fileStream.on('finish', () => {
console.log('memory successfully downloaded as memory.jpg');
});

Why does Chrome DevTools show multiple garbled versions of my source code for my Vue application?

I have a Vue application and I'm trying to debug it in Chrome DevTools. The problem is when I try to find the file I want to debug, I get a list of files with the same name plus some weird hash tacked onto the end:
When I open any one file, I get some garbled minified code:
Sometimes I can find the file I want (with the original source code) but sometimes not.
What are these weird files and how can I find the file I want (with the original source code). Is there a way of getting the DevTools to list only the original source code files?
Thanks.
What tool in dev tools are you using to get that list? Seems like a list of cached files, so it's showing all the old versions of your code.
If you go to the network tab and reload the page. You should see a list of all the resources downloaded by the browser. Choose the js filter and you should see your vue js bundle (made by webpack) somewhere in that list.
To allow chrome to display the source correctly you need to generate the Source Maps in development deployments.
I am not sure what tool you are using to build and bundle, but it is likely that you might have support for this already.
Chrome Details:
https://developer.chrome.com/docs/devtools/javascript/source-maps/
OMG - debugging my debugging environment. It's SO maddening.
I'm working with Vue v2, and I'm using vuetify in my app. Here is a complete vue.config.js configuration that solved this problem for me.
// vue.config.js file
const path = require('path')
const { defineConfig } = require('#vue/cli-service')
module.exports = defineConfig({
transpileDependencies: [
'vuetify'
],
configureWebpack: config => {
if (process.env.NODE_ENV === 'development') {
// See available sourcemaps:
// https://webpack.js.org/configuration/devtool/#devtool
config.devtool = 'eval-source-map'
// console.log(`NOTICE: vue.config.js directive: ${config.devtool}`)
config.output.devtoolModuleFilenameTemplate = info => {
let resPath = path.normalize(info.resourcePath)
let isVue = resPath.match(/\.vue$/)
let isGenerated = info.allLoaders
let generated = `webpack-generated:///${resPath}?${info.hash}`
let vuesource = `vue-source:///${resPath}`
return isVue && isGenerated ? generated : vuesource
}
config.output.devtoolFallbackModuleFilenameTemplate =
'webpack:///[resource-path]?[hash]'
}
},
})
I found a work around for this. While you can not see the source code of your file, just change the code (add console or sth.) of the file you want to see while Vue is hot reloading your changes. It occurs to me that the source code is then reachable when you check the developer console.
There is a surprising number of developers I meet on projects that have no idea there are official browser extensions for debugging Vue, Router, VueX etc.
Stumbling across this question prompted me to post this life saving link for those that land here and have missed the existence of this essential tool:
https://devtools.vuejs.org/guide/installation.html

Is it possible to export React single page website as HTML?

I have a single page web application using React and materialize-css and I would like to export it as static HTML and CSS so that it is possible to easily edit HTML for the purpose of prototyping. Is it possible to export at least a snapshot of current state?
I tried "save page" in Firefox and Chrome, but it does not provide good results.
Follow the following steps :-
1. In brouser, got to the developer tools,
2. select Inspector(firefox)/Elements(chrome),
3. then select the tag HTML, right click on it,
4. then click Edit as HTML.
Now you can copy all the code and save it. While the color and shape of the document remains, you will miss the pictures.
Good luck ! :)
Probably not ideal, but you can store the entire page as a variable and download it. Run this in your browser console after the page has loaded:
var pageHTML = document.documentElement.outerHTML;
var tempEl = document.createElement('a');
tempEl.href = 'data:attachment/text,' + encodeURI(pageHTML);
tempEl.target = '_blank';
tempEl.download = 'thispage.html';
tempEl.click();
The ReactDOMServer module contains a function for rendering a React application to static HTML - it's designed for use on the server, but I don't think there's anything to stop you using it in the browser (don't do this in production though!)
import ReactDOMServer from "react-dom/server";
import App from "./yourComponent";
document.body.innerHTML = ReactDOMServer.renderToStaticMarkup(App);
var pageHTML = window.document.getElementById('divToPDF').innerHTML;
let data = new Blob([pageHTML], {type: 'data:attachment/text,'});
let csvURL = window.URL.createObjectURL(data);
let tempLink = document.createElement('a');
tempLink.href = csvURL;
tempLink.setAttribute('download', 'Graph.html');
tempLink.click();
You can build your code and host it on github.io. The following tutorial will help you achieve that. You can then use the generated code in the gh-pages branch as your exported HTML
This was the first thread I found on SW.. so I think it would be appropriate to copy my own answer from another thread: https://stackoverflow.com/a/72422258/1215913
async function saveToFile() {
const handle = await showSaveFilePicker({
suggestedName: 'index.html',
types: [{
description: 'HTML',
accept: {'text/html': ['.html']},
}]
});
const writable = await handle.createWritable();
await writable.write(document.body.parentNode.innerHTML);
writable.close();
}; saveToFile();
for more info check the source answer
I had done this before but was stuck and couldn't seem to find the documentation anywhere. My scenario was I had a react js SPA and needed to create a static build to run without a server (through an organisations SharePoint using a static doc repository).
It is pretty simple in the end, run
npm run build
in your project directory and it will create the static build in a 'build' folder ready for you to dump wherever needed.
Reference: https://create-react-app.dev/docs/production-build/

NodeJS JSON.parse(...) takes forever to finish (under debugger in WebStorm)

EDIT
The problem seems to be related to WebStorm itself, it seems that it doesn't want to work with objects containing huge amount of nested objects. Neither it wants to show the object contents inside Watches window. The problem is kinda strange because I'm able to inspect the string, it is loaded blazingly fast. Seems like a WebStorm issue
I have a relatively big JSON file 4.9mb that I need to process in NodeJS, the file is stored in file system and is loadded using following lines of code:
var path = require('path');
var filename = path.join(__dirname, 'db_asci.json');
var fs = require('fs');
var content = fs.readFileSync(filename);
debugger;
var decycledObj = JSON.parse(content);
debugger;
The problem is that after the first debugger; breakpoint is hit, the second one is not, I'm waiting for more than 20 minutes and nothing, one processor core is loadded at 100%. I'm unable to debug the function because it's native.
Here is ASCI version of JSON
Here is UTF8 version of JSON
What am I doing wrong?
The problem you are running in to is not JSON parsing taking too long. Indeed, try this:
var start = Date.now();
var obj = JSON.parse(fs.readFileSync(filename));
console.log('Took', Date.now() - start, 'ms');
You'll probably see that it took less than a second or so.
What you are running into is an issue with the debugger itself – the observer effect. The act of observing a system changes that system.
I assume you're using node-inspector. Whenever you have an extremely large, complex object, it is extremely expensive to load the object into the inspector. While it is doing so, your node process will peg the CPU and the event loop is paused.
My guess is that the JSON is parsed and a huge (given that we're dealing with 5MB) object is created. Node then hits the second debugger, and the inspector needs to load locals. The excruciatingly slow process begins, and the inspector won't show that you've hit a breakpoint until it finishes. So to you it just looks frozen.
Try replacing your JSON file with something small (like {a:1}). It should load quickly.
Do you really need to visually inspect the entire object? There are tools better suited for viewing JSON files.
+1 for Pradeep Mahdevu solution, here is another way to the same thing, (edit with the async version)
var fs = require ('fs');
var options = { encoding: 'utf8' };
var jsonData = fs.readFile('db_asci.json', options, function (err, data) {
if (err) throw err;
var object = JSON.parse(data);
});
You can require .json files. So, no need to parse.
var content = require('./db_asci.json');
That should do it!

Importing local json file using d3.json does not work

I try to import a local .json-file using d3.json().
The file filename.json is stored in the same folder as my html file.
Yet the (json)-parameter is null.
d3.json("filename.json", function(json) {
root = json;
root.x0 = h / 2;
root.y0 = 0;});
. . .
}
My code is basically the same as in this d3.js example
If you're running in a browser, you cannot load local files.
But it's fairly easy to run a dev server, on the commandline, simply cd into the directory with your files, then:
python -m SimpleHTTPServer
(or python -m http.server using python 3)
Now in your browser, go to localhost:3000 (or :8000 or whatever is shown on the commandline).
The following used to work in older versions of d3:
var json = {"my": "json"};
d3.json(json, function(json) {
root = json;
root.x0 = h / 2;
root.y0 = 0;
});
In version d3.v5, you should do it as
d3.json("file.json").then(function(data){ console.log(data)});
Similarly, with csv and other file formats.
You can find more details at https://github.com/d3/d3/blob/master/CHANGES.md
Adding to the previous answers it's simpler to use an HTTP server provided by most Linux/ Mac machines (just by having python installed).
Run the following command in the root of your project
python -m SimpleHTTPServer
Then instead of accessing file://.....index.html open your browser on http://localhost:8080 or the port provided by running the server. This way will make the browser fetch all the files in your project without being blocked.
http://bl.ocks.org/eyaler/10586116
Refer to this code, this is reading from a file and creating a graph.
I also had the same problem, but later I figured out that the problem was in the json file I was using(an extra comma). If you are getting null here try printing the error you are getting, like this may be.
d3.json("filename.json", function(error, graph) {
alert(error)
})
This is working in firefox, in chrome somehow its not printing the error.
Loading a local csv or json file with (d3)js is not safe to do. They prevent you from doing it. There are some solutions to get it working though. The following line basically does not work (csv or json) because it is a local import:
d3.csv("path_to_your_csv", function(data) {console.log(data) });
Solution 1:
Disable the security in your browser
Different browsers have different security setting that you can disable. This solution can work and you can load your files. Disabling is however not advisable. It will make you vulnerable for all kind of threads. On the other hand, who is going to use your software if you tell them to manually disable the security?
Disable the security in Chrome:
--disable-web-security
--allow-file-access-from-files
Solution 2:
Load your csv/json file from a website.
This may seem like a weird solution but it will work. It is an easy fix but can be unpractical though. See here for an example. Check out the page-source. This is the idea:
d3.csv("https://path_to_your_csv", function(data) {console.log(data) });
Solution 3:
Start you own browser, with e.g. Python.
Such a browser does not include all kind of security checks. This may be a solution when you experiment with your code on your own machine. In many cases, this may not be the solution when you have users. This example will serve HTTP on port 8888 unless it is already taken:
python -m http.server 8888
python -m SimpleHTTPServer 8888 &
Open the (Chrome) browser address bar and type the underneath. This will open the index.html. In case you have a different name, type the path to that local HTML page.
localhost:8888
Solution 4:
Use local-host and CORS
You may can use local-host and CORS but the approach is not user-friendly coz setting up this, may not be so straightforward.
Solution 5:
Embed your data in the HTML file
I like this solution the most. Instead of loading your csv, you can write a script that embeds your data directly in the html. This will allow users use their favorite browser, and there are no security issues. This solution may not be so elegant because your html file can grow very hard depending on your data but it will work though. See here for an example. Check out the page-source.
Remove this line:
d3.csv("path_to_your_csv", function(data) { })
Replace with this:
var data =
[
$DATA_COMES_HERE$
]
You can't readily read local files, at least not in Chrome, and possibly not in other browsers either.
The simplest workaround is to simply include your JSON data in your script file and then simply get rid of your d3.json call and keep the code in the callback you pass to it.
Your code would then look like this:
json = { ... };
root = json;
root.x0 = h / 2;
root.y0 = 0;
...
I have used this
d3.json("graph.json", function(error, xyz) {
if (error) throw error;
// the rest of my d3 graph code here
}
so you can refer to your json file by using the variable xyz and graph is the name of my local json file
Use resource as local variable
var filename = {x0:0,y0:0};
//you can change different name for the function than json
d3.json = (x,cb)=>cb.call(null,x);
d3.json(filename, function(json) {
root = json;
root.x0 = h / 2;
root.y0 = 0;});
//...
}