jupyter lab becomes very lagging after machine learning with a lot of data when using JupyterDash - plotly-dash

I am using JupyterDash and use Jupyter Lab to edit and run the file. When opening it the first time, it is running well, however after doing machine learning with a lot of data, jupyter lab becomes very lagging. Even I closed the browser and closed anaconda prompt and re-logged into jupyter lab, it still is lagging a lot.
so I think it is related to memory in the browser?
I don't use dcc.Store() since my data is more than 10MB. I just pass the data in callback to the function.
Here is what I used. Thanks for your help.
if __name__ == "__main__":
app.run_server(mode='external',debug=True,port=8050)

Related

Reverse Engineering / Log or intercept program instructions

I'm trying to find a way of replicating the action / instruction that a physical button being pushed on a control panel sends to the software of a CNC machine of ours.
Ultimately I would like to integrate this instruction into an executable file I could make using AutoIT, but that is further down the line!
After some googling, resulting in all kinds of weird and wonderful results, I'm at a loss of how to begin this task. I believe I need to either use debugging software to find the instruction as it takes place, or possibly Process Monitor?
The machine runs off of a Windows XP machine.
Unfortunately obtaining this information from the manufacturer is not an option.
If anyone could help point me in the right direction that would be appreciated,
Thanks
Edit: I have since come across Windows Hooks, Detours and Interception, but still haven't made much progress!
Your topic is too broad ... You might as well be asking "How do I reverse engineer?" First thing I would do would be to load up the program in a debugger, put a breakpoint in the callback function and find out what the button is doing. What you will most likely find is that it's pushing some information onto the stack and making a call to an external .DLL such as an API or device driver ( you could probably find out which DLL using Process Monitor too ). Just load that .DLL up into your new program and make the same call.

Building web server using chrome.socket API

I've built a web server using Chrome Packaged Apps. The problem I see repeatedly is that chrome.socket.accept() and chrome.socket.write() don't invoke their callback functions. It usually works more or less reliably if request rate is less than 1 request per seconds. If I go above that, then I start seeing errors or missing callbacks.
I did similar tests with sample "webserver" app build by Google (https://github.com/GoogleChrome/chrome-app-samples/tree/master/webserver). It has the same problem. It usually takes less than 100 requests before web server stops responding. The easiest way to reproduce the problem is to use Chrome browser as a client and hold F5 key for few seconds.
It would be desirable to have a sample app that demonstrates how to build reliable web server using chrome.socket. So far I tried several different workarounds for monitoring the situation from the app itself and restarting socket when socket stops working, but it's not easy because there is no reliable way to check the status of the connection or status of the last operation when callback is not fired. I tried to use getInfo() method, but it always returns "connected=true" regardless of the situation.
I saw this on Windows 7 and Chrome OS (Chromebook).
Just an update on this. According to this the issue is now fixed.
There are still other problems with the sample web server application. I noticed that I could make the sample app lock up by holding down Ctrl-R in the browser. I wrote a more robust one that you can use here: https://github.com/kzahel/web-server-chrome

AS3 workers and large data files not possible?

so I load large amounts of data from services. I just updated our project to the latest 4.7 FB with Flex 4.9 SDK and AIR 3.4. Implemented workers. They seem to work great for one-off tasks EXCEPT when it comes to internet data loading. I haven't found the magic limit yet, but it seems as though if you load an internet request (Loader, URLLoader, HttpService... whatever) that is receiving a large replay, the worker just locks up and quits. The main thread can do this just fine in all cases (but with UI being unresponsive until the load is done). This is why I was so excited about workers is to offload the sometimes large data loads to the background workers.
Has anyone else run into this? I saw comments on the Worker class docs online where a few other people have seen similar problems and suggest putting data loading from the internet back on the main thread. Seems like "what's the point of a worker then"?
Can they only do local calculations? Math is cool... but HTTP Gets are not?
I tried giving the worker app privileges.. no help there. Is there a magic worker.canloadlargefiles = true? (rhetorical).
Any direction or help here would be greatly appreciated.
Well not a solution, but a workaround. Instead of digging internals of VM, use KISS (Keep It Simple and Stupid) principle, divide your data into managable chunks. You do not need to divide the files or data but can tell the server do it by a request format so it sends you manageable portions of data that will not timeout the VM. Retrieve and join the chunks on the client side to form the big file back.
Thats my two cents.

Java SE binary crash

I have a Java swing application that subscribes to a lot of data and displays this data in various ways. Under heavy load I have come to encounter that the JRE simply stops working with message "Java(TM) Platform SE binary has stopped working". This obviously shuts down my application and I need to restart it. I have tried to google for ways to troubleshoot this issue as I do not get a stacktrace in my code or anything that I can work with but I have found very little useful information beyond upgrading/re-installing the JRE and running virus scans. I have done both of these measures and rebooted the server but the problem still persists. I have tried to monitor the process with Java VisualVM (see dump below) but I am no expert on this tool and may not know what to look for. The observation that I have made is that the 'crashes' appear to coincide with Garbage Collections.
The issue is quite easy to reproduce and occurs after about 10 minutes of running the application. I do not run the application with any specific jvm parameters. The Java version is 1.6.0_31 (was _25 before upgrade) and I run on Windows 7 64-bit.
In the pic below from VisualVM the Java binary has just stopped working which appears to coincide with the GC-run.
Any help or ideas so that I can troubleshoot or remedy the problem is greatly appreciated. Thanks.
Three things to check:
If you've implemented the finalize() method anywhere, make sure it doesn't directly or indirectly lock any objects; this can cause a catatrophic deadlock correlated with GC.
If you've got native code, any number of weird things can happen if the code is not using global references correctly, including deadlocks and weird memory corruption, which would again correlate with GC activity.
Finally, GC might just be "stirring the pot" and exposing vanilla deadlocks which exist otherwise in the application; check your synchronization protocols.
Garbage collection pauses the VM's application threads while it happens, which might be exposing a race condition somewhere.

VS2008 partially freezing when switching to HTML design view

This is an odd freeze. When I switch from source view to design view for an HTML or ASPX file, the client area freezes, but I can still click on other tabs and menus.
What am I missing here? Really don't feel like reinstalling VS2008.
I had the same problem, and found one resolution.
In VS 2008, In a page that was using a master page, the either frequency while working in source view or switching to design view, IDE would freeze for 10-20 seconds.
In my master template, I had references to the Google hosted JQuery, Jquery UI, and one or two more scripts off site. These were placed directly in my master page's head section.
I downloaded the js and then by deleting any offsite references, my IDE would be smooth again in both design and source mode.
I also discovered I could put the scripts inside my ToolkitScriptManager (I'm using AjaxControlToolkit) and added the Mode="Release", and could place the http://www.google.com references for the scripts. The IDE is still working fine for me.
This is often due to the Design mode downloading external resources that are timing out. As #JonK mentioned, for him it was jQuery references. I have seen this when the ConnectionString was set to production databases that could not be accessed on my development machine, even though I wasn't debugging (running) the site only editing code, it would still try to connect and because it couldn't it would stall waiting for the timeout.
VS2008 is mostly single-threaded for UI operations like this, so if it is downloading a slow or non-existent network path it hangs like this.
VS2008 can make all kinds of network requests, so these two examples may not solve it for you. The best way I have found to diagnose the problem is to use the Microsoft tool Process Monitor, filter by the Process webdev.exe, and watch for I/O requests that are long running and/or throwing errors. In my case, I could find the place that was having a problem because there would be a 20 second gap in between the hundreds of I/O entries in Process Monitor. Then, just back-tracked from when that gap began and I eventually found the request that was causing the problem.
This may not be possible for you, but if you can, an upgrade to VS2010 would help; it does a much better job of running process on multiple threads in more places so you don't have to worry about this as much.
Have you tried restarting your computer and then reopening your project?