Using the "inspect" feature, the network resources that a certain website was using. Here it's just an interactive real-time bus map.
I added just one filter, the string vehicle and filter narrows down to requests to service-status.js being made on 30 second intervals each file is 2.5kB and 102ms which was the time for the request to happen. Finally, it says 32kB / 1.9 MB transferred.
How do *.json files or a transcript of the exchanges listed here? There is a HAR file option, which saves extra information. Using the search filter, I only need a fraction of the network resources being transferred.
You could try webQsee (https://chrome.google.com/webstore/detail/webqsee-web-sniffer-recor/gamdpfnfkjknkimfbboonmgdfnondfme?hl=de), it offers a lot of filtering and export options. So you can not only export to HAR format but also to Excel.
Related
In the network tab within the Chrome Dev Tools, I have a large amount of HTTP requests that I would like to export into a HAR to analyze. I've tried to export the HAR with the export button, and I've tried the "Save all as HAR with content" option as well, but both result in a 0byte HAR file when downloaded. I've exported HARs in the past, but I guess they've never been as large so they have succeeded without issue. For reference, the network tab shows:
9262 requests
8.6 MB transferred
21.8 MB resources
Anyone have any experience in exporting large HAR files? Is there a way to build a script that could export this?
You could try webQsee (https://chrome.google.com/webstore/detail/webqsee-web-sniffer-recor/gamdpfnfkjknkimfbboonmgdfnondfme?hl=de), it is able to handle large amounts of data.
In addition to HAR export, you can also export so called "Behavior Reports" that contain not only the network communication but also console output and the screen video for later analysis.
I have a folder with hundreds of files that were saved on a specific format of a given software (in this case it is the Qualisys Track Manager and the file format is .qtm).
This software has the option of exporting the files to another format such as TSV, MAT, C3D,...
My problem: I want to export all my files to TSV format but the only way I know is open the software, go to File->Export->To TSV. And doing this for hundreds of files is time consuming. So I was thinking on writing a script where I could call my files, access the software and it would do the export automatically.
But I have no clue how to do this, I was thinking on writing a script on Notepad++, running on the command window and then I would get all the files on TSV format.
[EDIT] After some research I think maybe a Batch script or a PowerShell script may help me but I have no idea how to run automatically the commands of the software of if it is even possible... (I am using Windows10)
It is highly likely to be a perpetual file format(.qtm) and Powershell/batch would not understand it. Unless this file can be read in a known way (Text XML etc), they would not be able to convert it.
I googled it and seems QTM have a REST API interface. It would be the best chance you have. I'm not sure if the documentation is available publicly, I didn't find it. I'd recommend you contact their support for REST API document/ask if their REST API can handle this task/sample code to get you start.
Then you can make REST API calls with Invoke-RestMethod in a loop from powershell.
I want to get the json file in Azure Storage Blob through the browser.
I used Stream Analysis and comes out a json file in the Blob container. Now i need to get the information inside the json file in order to show the IOT device status in real-time.
I tried to use Jsonp,
but I don't know how to add the callBack method in the Json file without download it. Is there any way to add the callBack method??
or Is there another way to get the information inside the container?
for this particular scenario, I'd recommend PowerBI. Now Stream Analytics have direct output to PowerBI and you can pretty much customize the dashboard for your real time IoT needs.
You can refer to this article for step by step Stream Analytics + PowerBI.
Coming back to your question, you need to download the blob to access the content. Stream Analytics to BLOB is usually for archiving or later predictive analysis scenarios.
Instead if you still prefer not to use PowerBI, I'd either arrange the SA output to an event hub and read the data from there in real time or alternatively save the data into a NO-SQL db like DocumentDB on Azure and then read from there. I can recommend Highcharts if you want to use custom gauges etc to visualize the data.
Hope this helps.
If I use wget, and get an html file from a URL. How can I generate a HAR file from that HTML file.
Any open source implementations for generating har files from html files?
Once the HAR file is generated I can read data from the HAR file using harlib.
If possible please suggest C, C++ or Java implementations.
The primary point of the HAR format is to have a standard HTTP tracing format that many tools can use and analyze. In other words, it's original intent was and primarily is, for performance analysis, not "archiving" webpages per se.
If you fetch a page with wget, you're missing 99% of all the performance data. To capture the necessary data you really need a browser to execute the requests, fetch all the associated resources, save all the timers, etc. This will enable you to build the waterfall charts, etc.
If you need to capture this data on the server, then you can use pcap to capture the TCP trace and then convert that to HAR, although you still need a client which will actually parse the HTML and request all the sub-resources (pcap is just listening in the background). Alternatively, you can route your browser through a proxy and let it spit out a HAR file for you.
Last but not least, you can just drive the browser through its debug interface and export the HAR file that way. Java example for driving Firefox: https://github.com/Filirom1/browsermob-page-perf
Is there an easy way, to save a Google Ngram result
http://books.google.com/ngrams/
as a csv?
So that I get a list like
1900 peace 500000times
1901 peace 540000times
and so on?
I downloaded their raw data but have no idea how to handle it. When I open those csv in OpenOffice, I can't even see a single word.
It can be done, and it's actually quite easy. Generate the graph you want on the Google Ngram viewer, then use your browser's function to show the page source code (this might be hidden under advanced or developer options). Then in the code (probably on line 297), you will find the data simply listed. You can use any word processor and/or spreadsheet software to clean up the data and export them as CSV.
No, you'd have to go to their datasets, which would be daunting to find what you wanted, or you could try Microsoft Research's N-Gram service.