I am building an application with a simple search panel with few search attributes and a result panel. In the result panel, I am rendering the data in a tabular form using Slickgrid.
After few searches (AJAX call to server), the page gets so much loaded and it eventually crashes after sometime. I have checked the DOM count and the JavaScript heap usage for possible memory leaks. I couldn't find anything wrong there. However, when I ran the experimental native memory profiler, I see that the "JavaScript external resource" section uses 600+ MB memory. On running the garbage collector, it is getting down to few MBs. I have couple of questions here:
What contributes to the "JavaScript external resource" section? I thought it corresponds to the JSON data / JavaScript sources which gets transferred from the server. FYI, the gzipped JSON response from the server is ~1MB.
Why is Chrome not releasing the memory pro-actively instead of crashing the page? Again, when I manually run the garbage collector, it is releasing the memory used by "JavaScript external resources".
How do I fix the original problem?
JS Heap Profiler makes a snapshot of the objects in the javascript but javascript code may use native memory with help of "Int8Array", "Uint8Array", "Uint8ClampedArray", "Int16Array", "Uint16Array", "Int32Array", "Uint32Array", "Float32Array" and "Float64Array".
So when you take a snapshot it will have only small wrappers that point to native memory blocks.
Unfortunately heap snapshot does not provide data about the native memory that was used for these kind of objects.
Native heap snapshot is able to count that memory and now we know that the page uses native memory via an array or via an external string.
I'd like to know how did you check that the page has no memory leaks? Did you use three snapshot technique or just checked particular objects?
Tool to track down JavaScript memory leak
Related
I'm using QEMU to test some software for a personal project and I would like to know whenever the program is writing to memory. The best solution I have come up with is to manually add print statements in the file responsible for writing to memory. Which this would require remaking the object for the file and building QEMU, if I'm correct. But I came across QMP which uses JSON commands to manipulate QEMU, which has an entire list of commands, found here: https://raw.githubusercontent.com/Xilinx/qemu/master/qmp-commands.hx.
But after looking at that I didn't really see anything that would do what I want. I am sort of a new programmer and am not that advanced. And was wondering if anyone had some idea how to go about this a better way.
Recently (9 jun 2016) there were added powerful tracing features to mainline QEMU.
Please see qemu/docs/tracing.txt file as manual.
There are a lot of events that could be traced, see
qemu/trace_events file for list of them.
As i can understand the code, the "guest_mem_before" event is that you need to view guest memory writes.
Details:
There are tracing hooks placed at following functions:
qemu/tcg/tcg-op.c: tcg_gen_qemu_st * All guest stores instructions tcg-generation
qemu/include/exec/cpu_ldst_template.h all non-tcg memory access (fetch/translation time, helpers, devices)
There historically hasn't been any support in QEMU for tracing all guest memory accesses, because there isn't any one place in QEMU where you could easily add print statements to trace them. This is because more guest memory accesses go through the "fast path", where we directly generate native host instructions which look up the host RAM address in a data structure (QEMU's TLB) and perform the load or store. It's only if this fast path doesn't find a hit in the TLB that we fall back to a slow path that's written in C.
The recent trace-events event 'tcg guest_mem_before' can be used to trace virtual memory accesses, but note that it won't tell you:
whether the access succeeded or faulted
what the data being loaded or stored was
the physical address that's accessed
You'll also need to rebuild QEMU to enable it (unlike most trace events which are compiled into QEMU by default and can be enabled at runtime.)
My application connects to a SignalR-hub, and it receives about 15 json-files per minute. For every json-file with a geolocation, a marker gets placed on a sphere in the 3D WebGL-scene, and gets stored in an object {}. When the objectlist reaches 100 items, the application deletes the first item in the objectlist, and deletes the corresponding marker from the scene.
But still, there is a huge memory leak, and when I hit about 200 events, the application crash.
Are the json-files being stored somewhere, and are causing the memory leak?
Though this is not the solution, but I will suggest you to use Ants Profiler to see if there is any memory leak issues in your application.
I will suggest you to derefernce all the listitem in the generic list after it is no use. This is the major cause of memory leaking.
Beware of concatenating strings, use stringbuilder as much as possible.
I am working on the same issue. I had no memory leaks before implementing signalr and now have a fairly significant one. I can disable the signalr in my app and the leak goes away. I'm on version 2.2.0 of signalr (the most current version from Nuget. The only thing that could be hanging up in memory from my code point of view is the event handler. At least that's the only thing I'm creating on the client side.
As I load more data into my application, 'Other' section keeps growing in my web app. What does this mean? I have checked my code for any memory leaks and couldn't find any. Also, the objects I hold in my application are very much needed for the application.
Varunkumar Nagarajan
How do you checking your app for a leak?
Have you seen Tool to track down JavaScript memory leak
Native memory snapshot is an experimental feature and couldn't be used for detecting memory leaks at the moment.
What are in-memory function calls? Could someone please point me to some resource discussing this technique and its advantages. I need to learn more about them and at the moment do not know where to go. Google does not seem to help as it takes me to the domain of cognition and nervous system etc..
Assuming your explanatory comment is correct (I'd have to see the original source of your question to know for sure..) it's probably a matter of either (a) function binding times or (b) demand paging.
Function Binding
When a program starts, the linker/loader finds all function references in the executable file that aren't resolvable within the file. It searches all the linked libraries to find the missing functions, and then iterates. At least the Linux ld.so(8) linker/loader supports two modes of operation: LD_BIND_NOW forces all symbol references to be resolved at program start up. This is excellent for finding errors and it means there's no penalty for the first use of a function vs repeated use of a function. It can drastically increase application load time. Without LD_BIND_NOW, functions are resolved as they are needed. This is great for small programs that link against huge libraries, as it'll only resolve the few functions needed, but for larger programs, this might require re-loading libraries from disk over and over, during the lifetime of the program, and that can drastically influence response time as the application is running.
Demand Paging
Modern operating system kernels juggle more virtual memory than physical memory. Each application thinks it has access to an entire machine of 4 gigabytes of memory (for 32-bit applications) or much much more memory (for 64-bit applications), regardless of the actual amount of physical memory installed in the machine. Each page of memory needs a backing store, a drive space that will be used to store that page if the page must be shoved out of physical memory under memory pressure. If it is purely data, the it gets stored in a swap partition or swap file. If it is executable code, then it is simply dropped, because it can be reloaded from the file in the future if it needs to be. Note that this doesn't happen on a function-by-function basis -- instead, it happens on pages, which are a hardware-dependent feature. Think 4096 bytes on most 32 bit platforms, perhaps more or less on other architectures, and with special frameworks, upwards of 2 megabytes or 4 megabytes. If there is a reference for a missing page, the memory management unit will signal a page fault, and the kernel will load the missing page from disk and restart the process.
I'm writing a Windows CE application, and I want to play a sound (a short wav file) when something happens. Since this sound will be played often, my first instinct was to load the wav file into a memory stream and reuse that stream instead of reading the file every time.
But then it occured to me that these Windows Mobile devices only have one kind of memory, which is used both for data storage (= the file system) as well as for program memory; there's even a nice slider in the control panel which you can use to delegate memory to either storage or program execution. So, theoretically, reading a file from the file system (or some value from a SQL Server CE database) should take (almost) the same amount of time as reading this value from some in-memory object, right?
Is this assumption correct (i.e., in-memory caching on application level doesn't make sense here) or did I miss something? For simplicity, let's assume that only the internal memory of the device is used (no memory card).
The assumption may or may not be valid. Where in storage does it reside? If it's persistent storage (like a storage card folder or anything else that remains when you hard reset) then it's backed by Flash, which is way, way slower than RAM and there will be a difference in load perf, though how much it might impact your app I can't say - only testing will tell you that.
When I want to play a short WAV file on Windows Mobile (e.g. notification sound). I usually add it as a resource to my executable. AFAIK resources are loaded into RAM since they are part of the executable image. You can then conveniently call PlaySound() with the SND_RESOURCE (and probably OR that with SND_ASYNC too so the call isn't going to block while the file is being played) flag.