I have loaded up a database with about 7.5M nodes having 33+M relationships - it's about 25 GB in total. So, it's reasonably large is my point. What I am finding since loading it is that periodically my Neo4j client is just falling over, leaving nothing more than Chrome's irritating "Aw snap - something went wrong" behind. I have checked the logs and found nothing significant there. How can I begin to track down what is happening on these failed queries?
The issue might be with your Chrome environment. This Google support page on the "Aw Snap!" error may be of help.
Also, if your queries return a lot of data or take too long to respond, the browser might be running out of memory or exceeding some internal timeouts. So, make sure the queries you make via a browser are tailored accordingly.
Related
Chrome Timing View
The image above show chrome spends most of the time queuing up the request. I am trying to figure out why that is happening to minimize it.
According to chrome developer documents:
A request being queued indicates that:
The request was postponed by the rendering engine because it's
considered lower priority than critical resources (such as
scripts/styles). This often happens with images.
The request was put on hold to wait for an unavailable TCP socket that's about to free up.
The request was put on hold because the browser only allows six TCP connections per origin on HTTP 1.
Time spent making disk cache
entries (typically very quick.)
Number 3 seems to be the most likely problem according to chrome developer documents but I know that only one request is going out at a time so that can't be it. I don't think it is number 1 either because the performance monitor doesn't show a lag from rendering. Maybe it is either 2 or 4 but I don't know how to test that.
Chrome Performance Monitor
I've included a picture of the performance monitor that shows these long tasks where something is happening in the system. These are also a mystery to me and seem related.
Any help is greatly appreciated!
Edit: It seems you can disable the disk cache when you open dev tools and that didn't seem to fix the problem.
We have an air application (native for windows) that after being unused or background for about some hours after you try to use it it got stuck and "not responding" windows message.
No log or error or crash just got stuck.
The best behaviour that we have got is that after some seconds it unstucks but then stucks and unstucks every 30 seconds aprox, so it is anusable.
We have tried some "ugly" tricks like after being out of focus/background for some minutes, goes into front, it goes, but this haven't resolved the above problems.
I understand clearly, that SO (Windows) has removed the cached memory pages associated to this process, however, this is normal for each program, so, it is normal that program (specially visually heavy like ours) come back to use, it takes some seconds to go into memory again.
However after that time of re caching into memory any programs (most) works like a charm.
This is not the case of air app.
is there any trick or recommendation for fixing this?
Maybe we should combine the "to front" trick with some other "dumb processing" to keep the program in memory?
thanks in advance.
I've developed a Chrome Extension that cycles through approximately 1500-2000 pages to collect information from a website and push it onto my own server. I use a Chrome extension, as given the requirements it's much easier to configure than making and parsing from the server side.
The extension is used only for internal purposes and we trigger this job using the chrome.alarm API to run at 3:00 am every day. This alarm when triggers pops open a new tab and runs through the 1500-2000 pages. The problem is when we check in the morning we see the SNAP dialogue after the extension has cycled through about 1500 pages (approx 3/4). I'm presuming this is due to the memory demand placed on Chrome to maintain such an unusually large history?
My question is, what would be the best way to mitigate this? Presumably killing the tab and reopening (after x number of pages) would work but that would slow down the feed and require quite a bit of code re-factoring. Is there any way you can force Chrome to dump the history and would doing so free memory in the immediate session?
Just to add some context, I am running this on a extra small VM with only 1 GB of memory. I appreciate I could upgrade the VM but that really just defers the problem.
It's hard to make a solid recommendation without seeing your code, but here's something a bit more general:
The problem is likely the information that is being collected. It continues to build up in memory until you run out. You might consider saving the results to local storage using Chrome's unlimited storage capability. In manifest file:
"permissions": [
"unlimitedStorage"
],
What I would probably do is save to local storage after every page, or something similar, and then reset the array (or whatever you're using).
UPDATE
If indeed history is the problem, you could consider deleting it on the fly. This function will delete entries for a specific URL:
https://developer.chrome.com/extensions/history.html#method-deleteUrl
Suppose I've got the following parts in my system: Storage (S) and a number of Clients (C). The clients are separate Web Workers and I'm actually trying to emulate something like shared memory for them.
Right now I've got just one Client and it's communicating with the Storage pretty intensively. For the sake of testing it is spinning in a for-loop, requesting some information from the Storage and processing it (processing is really cheap).
It turns out that this is slow. I've checked the process list and noticed chrome --type=renderer eating lots of CPU, so I thought that it might be redrawing the page or doing some kind of DOM processing after each message, since the Storage is running in the page context. Ok, I've decided to try to move the Storage to a separate Worker so that the page is totally idle now and… ended up getting even worse performance—exactly twice slower (I've tried a Shared Worker and a Dedicated Worker with explicit MessageChannels with the same results).
So, here is my question: why sending a message from a Worker to another Worker is exactly twice slower than sending a message from a Worker to the page? Are they sending messages through the page? Is it “by design” or a bug? I was going to check the source code, but I'm afraid it's a bit too complex and, probably, someone is already familiar with this part of Chromium internals…
P.S. I'm testing in Chrome 27.0.1453.93 on Linux and Chrome 28.0.1500.20 on Windows.
I'm the primary developer at a web firm, but often end up doing some sysadmin stuff, and was wondering what resources are available for learning how to troubleshoot slow page load times.
My experience with sysadmin tools is almost none. I'm relatively proficient at the Linux/Unix command line, but have never used any type of packet tracking software and only know the basics of using dig for ip resolves. My experience with apache and mysql is mostly limited to configuring initial setup and then using them.
Are there any good books or web sites that cover the topics needed for accurately diagnosing website performance/bottlenecks and if so what are they, or is the gamut of technologies used to large and experience/time with using the technologies typically how people get good at this stuff?
Overall, there's no substitute for experience. A concept as broad as "slow web page load time" could be hitting a bottleneck in any number of different places:
Client is slow to resolve IP from domain.
Network between client and domain is slow or congested.
Server is slow to respond to request.
Requested page is large.
Requested resources embedded in page are large.
Page contains server-side code that requires significant processing.
Database is slow to respond.
Page manipulates a lot of data before responding.
Rendered page on client contains a lot of code and runs slowly.
etc.
For any given page, it's a matter of know where the bottlenecks could be and determining what that bottleneck is in order to address it. Having a full and complete understanding of everything that goes on from end to end in "loading a page" is essential. Identifying patterns of slow load times across multiple disparate requests will help narrow down the potential bottlenecks. etc.
It's very much a case-by-case scenario.
Have you diagnosed exactly where the problems are? Slow load times could be caused by anything, such as:
too much going on in the dom/js
a) not caching js or other resources on the client
b) not minifying/compressing resources
c) making too many requests with ajax/doing silly things in the browser like redrawing dom that doesn't need it.
too much going on in the server
a) no cache on db tables, no indexes
b) not handling long running tasks asynchronously
c) improperly configured proxies/apache servers
network issues -- wish I knew more about this.
Step 1 is always to figure out where the worst slowdown is. Do some metrics on the server to make sure it is doing easy stuff fast. And hard stuff reasonably fast. Look in the browser to see how long loading resources is taking. Look in the chrome/firebug profilers to see how much time the javascript is taking to run.
You will probably find a bunch of things that could be improved. Prioritize and address the issues...