Remote CUDA profiling? - cuda

Is it possible to remotely execute a CUDA profile execution (similar to computeprof) and then bring the profile back for analysis?
The particular remote machine is headless and not-under-my-control, so no X, no Qt libraries, etc.

Yes you can. The CUDA driver has built-in profiling facilities. How to do it is discussed in the Compute_Profiler.txt file you will find in the doc directory of the toolkit, but the basic idea is something like this:
$ COMPUTE_PROFILE=1 COMPUTE_PROFILE_CSV=1 COMPUTE_PROFILE_LOG=log.csv COMPUTE_PROFILE_CONFIG=config.txt ./app
which tells the runtime to turn on profiling, use csv format output written to log.csv, including the profile statistics read from config.txt. After the app has run, the runtime will drop an output file with the raw profiling results in them. You can then use the tool of your choice to look at them. The visual profiler can be convinced open to the output, but a lot of the fancy synchronization it does requires the output to be generated using its own profile configuration files (under the hood it is dynamically doing the same thing you do manually, but on the fly). I have done some digging around and scraped copies of the configuration files so I could regenerate specific application profiling runs without the profiler on headless cluster nodes. Not too much fun, but it can be done.

Related

Can I use Teacup to manage custom offline Tcl code

I'm trying to figure out a good way for my company to have a local repository/package manager (something a little more user friendly than git). I like Teacup and we are using ActiveState Tcl anyways (Tcl 8.5, we have legacy systems using this version).
Can I use Teacup to make my own offline package manager repo? Sort of like how you can do that with Anaconda in Python. It has to be totally offline but I want to be able to upload packages to it when I make them and let Teacup handle the installation of them for other users in my company.
I've read through this page a little bit but it is missing some content.
You are looking for the server-side component to the "teaparty": teapot as the server providing for the teacup client:
https://wiki.tcl-lang.org/page/Teapot
While there is a dedicated teapot (server) implementation available from ActiveState, the client/ server protocol is straight forward: It is about generating markup (HTML) resources delivered via HTTP (containing table DOM structures) and processed by the teacup client. As always, these resources can be generated statically or dynamically, or anything inbetween.
Watch the examples at:
http://teapot.rkeene.org/index.html
Better:
view-source:http://teapot.rkeene.org/index.html
Assuming your Tcl projects are hosted using some SCM repo, you may provide some repository (CD, pipeline) action to produce a static resource structure served by a HTTP server of your choice? The original teacup client can be used against this resource collection.

Octave runs but graph not displayed

num=[1];
den=[1 3 1];
G=tf(num,den);
H=1;
T=feedback(G,H);
step(T);
hold on;
Kp=23;
Ki=0;
Kd=0;
C=pid(Kp,Ki,Kd);
T=feedback(C*G,H);
step(T);
When run this script nothing happen in Octave but works fine in octave-online.net
online octave
Octave Windows
I will put a proper answer here for future users, even though OP has already solved their problem from the comments.
octave-online.net is an excellent cloud service providing an instance of octave on the cloud.
Contrary to a typical installation of octave on linux or windows, the octave-online client autoloads some of the more popular packages, one of which is control.
You can confirm this by typing pkg list in the octave-online console.
In your normal linux / windows installation however, this needs to be loaded explicitly before use, e.g. in the case of the control package, by doing pkg load control.
Your code uses the functions feedback and pid, both of which rely on the control package, therefore in your windows instance, your code failed, because you tried to use these functions without loading the package first.
Presumably there was also an error in your terminal informing you of this fact, that you may have missed.

Export a KNIME workflow as a standalone application or JAR

Is there a way to export or compile a KNIME workflow as a standalone Java application or JAR? I'd like to run the workflow on a platform where KNIME cannot be installed and/or as part of a larger program to simplify a complex but isolated piece of analytics. My options are many, but installing KNIME on the target platform is not one of them.
The only relevant reference I can find on the KNIME site is this ten-year-old(!) forum question. The only answer there links to this project which does seem to be active and says it is 'a KNIME extension that exports KNIME workflows to different workflow engines', though without digging into its code it's not clear what engines those are.
Other than that, I guess your options are:
ask on the KNIME forum again
since KNIME is open source and is based on Eclipse, look into the more general question of how to build and run a minimal standalone version of Eclipse - there seem to be some relevant-looking answers on here if you search, but I have no further knowledge on how to do it
use scripting nodes in KNIME to develop a text language version of your workflow, verifying as you go that the output adequately matches the KNIME nodes at each step, and deploy the text language version to your target. If you need data mining methods you might want to look at the Weka integration nodes which you could then substitute with calls to Weka methods.

ESP8266 - How does it understand what "AT+RST" means without the firmware

I am new to ESP8266 and to the electronics world. I request you to please be patient if I am mis-understanding a point.
I am using ESP8266-01.
I successfully re-flashed the at official ai-thinker firmware version 1.1.1 to my module and I noticed that it was being written at memory location 0x00000.
Later I successfully uploaded a basic blink program using Arduino ide. Again the program was written to the memory location 0x00000(over-writing the firmware I guess).
I want to use it as a web server. The code for that uses the AT commands, something like "Serial.println("AT+RST");"
Now from what I understand the firmware would be over-written. Then how would the module understand what "AT+RST" means?
Thanks
"Arduino" is more than just the IDE you see and the boards (e.g. Uno). Arduino is also a kind of firmware/OS that runs on those boards. A sketch you write in the IDE is compiled together with the firmware into a single package that's written to memory.
There are a few more hints at https://github.com/esp8266/Arduino:
This project brings support for ESP8266 chip to the Arduino
environment. It lets you write sketches using familiar Arduino
functions and libraries, and run them directly on ESP8266, no external
microcontroller required.
ESP8266 Arduino core comes with libraries to communicate over WiFi
using TCP and UDP, set up HTTP, mDNS, SSDP, and DNS servers, do OTA
updates, use a file system in flash memory, work with SD cards,
servos, SPI and I2C peripherals.
When you hit that 'Upload' button in the IDE you're effectively replacing anything that's been written to the ESP8266 before.
This question is wrong on many levels.
It was based on the assumption that "Serial.println("AT+RST")" was a command to the esp while in reality the programmer meant it as a logging message to the serial monitor.
I had asked "How does it understand what “AT+RST” means without the firmware". The answer is that it doesn't and it can't unless I write some code in my sketch to handle it. The firmware is a kind of interpretter. It accepts a command in a particular format (e.g. "AT \n"), parses it, executes some corresponding low level function and returns the result.
After I have uploaded a sketch and over-written the AT firmware, the interpretter code is gone and hence the esp cannot interpret/understand the AT command.
#AdrianoRepetti:
"How "web server" and AT commands are related is unknown to me" - You are right, they are not related. My bad.
"Anyway AT commands is understood directly by ESP8266 chip" I doubt it.
Thanks for your answers.
This question is not adding any knowledge to anything. I think it should be deleted.

How should I implement an auto-updater?

Many programs include an auto-updater, where the program occasionally looks online for updates, and then downloads and applies any updates that are found. Program bugs are fixed, supporting files are modified, and things are (usually) made better.
Unfortunately no matter how hard I look, I can't find information on this process anywhere. It seems like the auto-updaters that have been implemented have either been proprietary or not considered important.
It seems fairly easy to implement the system that looks for updates on a network and downloads them if they are available. That part of the auto-updater will change significantly from implementation to implementation. The question is what are the different approaches of applying patches. Just downloading files and replacing old ones with new ones, running a migration script that was downloaded, monkey patching parts of the system, etc.? Concepts are preferred, but examples in Java, C, Python, Ruby, Lisp, etc. would be appreciated.
I think that "language agnostic" is going to be a limiting factor here. Applications come in so many shapes and sizes that there is no one-size-fits-all answer. I have implemented several auto-updaters in several languages, and no two were similar.
The most general philosophy is that the application checks with some home location (web address, web query, corporate network location, etc.) to either ask if it's version is current, or ask what the most current version is. If the answer calls for an update, that process will be different for each situation.
A popular alternative is to invite the home location to run a script when the application is initiated. The script can check the version, download updates if necessary, and ask for usage feedback, for example.
We can probably help better if you narrow the parameters.
UPDATE: The approach to "patching" also depends on the nature of the application, and there's a very wide diversity here. If you have a single executable file, for instance, then it's probably most practical to replace the executable. If your application has many files, you should look for ways to minimize the number of files replaced. If your application is highly customized or parameterized, you should strive to minimize the re-tailoring effort. If your application employs interpreted code (such as an Excel VBA application or MS Access MDB application), then you may be able to replace parts of the code. In a Java application you may only need to replace a JAR file, or even a subset of the JAR contents. You'll also need to have a way to recognize the current client version, and update it appropriately. I could go on and on, but I hope you see my point about diversity. This is one of those many times when the best answer usually starts with "Well, it depends ...!" That's why so many answers include "Please narrow the parameters."
Be sure to also consider the security implications of sucking down information about the update, as well as the update binaries themselves.
Do you trust the source of the download? You maybe phoning home to got your update, but what if there is a man in the middle who redirects to a malicious server. An HTTPS or similar secure connection will help, but double checking the bits that you eventually download by using a digital signature check is recommended.
First you need a file on your application home web site with the latest version.
The best way I think to have special SQL table for this task and populate it automatically after publishing new version / nightly build completion.
Your application creates new thread which requests built-in http link with version and compares in with current. In .NET use can use code like this:
Version GetLatestVersion() {
HttpWebRequestrequest = (HttpWebRequest)WebRequest.Create(new Uri(new Uri(http://example.net), "version.txt));
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (request.HaveResponse)
{
StreamReader stream = new StreamReader(response.GetResponseStream(), Encoding.Default);
return new Version(stream.ReadLine());
}
else
{
return null;
}
}
Version latest = GetLatestVersion();
Version current = new Version(Application.ProductVersion);
if (current < latest)
{
// you need an update
}
else
{
// you are up-to-date
}
In this example, version.php in only one plain string like 1.0.1.0.
Another tip I can give - how to download an update.
I like very much next idea: in the resources of your application there is a string of CLR-code which you compile on-the-fly (using CodeDom) to a temporary folder, main application calls it and goes to close. Updater reads arguments, settings or registry and downloads new modules. And calls main application which deletes all temporary files. Done!
(But everything here is about .NET)
The simplest solutions (used by many programs) is running the uninstaller for the previous version and the running the installer for the new one (optionally skipping questions which the user has already answered, like the EULA). The only catch is that the new version must be able to read the configuration options from the old version.
Also, on Windows you can't delete an executable file which is in use, so you probably will want to drop a small executable in Temp folder, which runs the whole process and then delete it at the end from the instance of the new version which got launched (or just register it to be deleted at the next reboot).
Because auto updating is a common scenario, most languages have at least one package available to support this. (Below I list some of the available packages)
One of the really nice idea's is the ClickOnce distribution for .NET, it's an installer which sandboxes your application and installs in the user context, so no administrator rights required. You can configure the ClickOnce in your publish to check for updates each application start.
Java has Java Web Start which offers the same kind of functionality for java applets.
Delphi has numerous articles about auto-updating, Torry has a list of WebUpdate components, for instance GoUpdater seems to have a very wide range of functionality.
They all use a website/network share to check for a new version and than retrieve either a patch or a complete install file and run it. So you should try to find a nice package for your application, to save you the hassle of developing and maintaining your own solution.
The simplest approach would be to have your program query a server (website) to see if there is an update. If there is an update you could display a message to the user that prompts them to download a newer version and provides a link.
An alternative and more complex solution would be to create a small windows service (or unix daemon) that checks periodically to see if there are updates, this service can download the update and launch the installer.
The general architecture is that you have a central server that you control that knows the latest version and where to get it. Then the programs query the server. I am not going to include sample code because it is highly defendant on the server and the format you choose. It is not terrible difficult though.
This is not so much a complete answer, but rather one example of auto-updating mechanism I implemented recently. The situation is a little different from the tradition Firefox-type of user application, since it was an internal tool used at work.
Basically, it's a little script that manages a queue of Subversion branches to be built and packaged in an installer. It reads a little file, where the names of the branches are written, takes the first one, re-writes it at the end of the file, and launches the build process, which involves calling a bunch of scripts. The configuration for each branch to build is written in a .INI file, stored in a Subversion repository along with the tool itself.
Because this tool runs on several computers, I wanted a way to update it automatically on all machines as soon as I made a change either to the tool itself, or to the configuration scripts.
The way I implemented it was simple: when I launch the tool, it becomes an "outer shell". This outer shell does 2 very simple things:
svn update on itself and on the configuration files
launch itself again, this time as the "inner shell", the one that actually handles one configuration (and then exits again).
This very simple update-myself-in-a-loop system has served us very well for a few months now. It's very elegant, because it is self-contained: the auto-updater is the program itself. Because "outer shell" (the auto-updater part) is so simple, it doesn't matter that it does not benefit from the updates as the "inner shell" (which gets executed from the updated source file every time).
One thing that hasn't really been mentioned is that you should seriously consider that the user running your program might not actually have sufficient privileges to upgrade it. This should be pretty common at least for business users, probably less so for home users.
I'm always working with a (self-imposed) limited account for security reasons and it always pisses me off that most auto-updaters simply assume that I'm running as admin and then after downloading just fail and offer no other way of performing the update other than actually closing the program and running it again in an administrative context. Most do not even cache the downloaded update and have to do it all over again.
It'd be much better if the auto-updater would simply prompt for admin credentials when needed and get on with it.
I'm going to assume answer for Windows.
This way seems to work well.
In the installer do:
1. Create a manual-start service that runs as LocalSystem that when started does the update then stops.
2. Change the service permissions so all users can start the service (if all users should be able to update w/o admin rights).
3. Change the main program to check for updates when started using a simple mechanism. If it detects an update, prompt if the user wants to apply it.
4. If user accepts the update, start the service.
If the architecture allows for it, create a way to monitor the update as it is running.
In a Java-Webstart setting you start a JNLP file which then triggers the download of the Jar files needed to run the application. Everytime webstart checks if there are newer versions of the Jars and would download them replacing the locally cached ones. With a tool named jardiff you will create only diffs towards the newer jars and distribute these via the server (e.g. only get an update).
Pros:
always up to date
Cons:
you need an application server (tomcat, JBoss) in order to distribute the files
you need an internet connection in order to get the application
Reading Carl Seleborgs answer gave me some ideas how a generic code-repository could be useful.
svn comes with a tool called svnsync, which sort of behaves like an svn export but keeps track of the actual revision your export is at.
Someone could utilize this system in order to only fetch the changed files from the users actual revision.
In actuality, you will have a repository with the binaries compiled, and running svnsync will only fetch the binaries that has been modified. It might also be able to merge local changes to text-based configuration files with new configuration-options.
The function of installing a patch to a program is basically one of the basic functions of an installer. Installer software is documented in numerous places but usually on a per-installer basis: There the Microsoft Installer (with Install Shield Extensions), Ruby gems, Java .jar files, the various Linux package manager systems (RPM, Apt-get)and others.
These are all complex systems which solve the problem of patching program in general but for slightly different systems. To decide what is best for you, consider which of these system your application most resembles. Rolling your own is fine but looking at these systems is a place to start.
You can write an internal module of your application to do updates. You can write an external mini application to do updates.
Also look at .NET on-the-fly compilation technology, it makes possible to create such mini application on-the-fly on demand. For example, http://fly.sf.net/
You can use my solution (part of the Target Eye project).
http://www.codeproject.com/Articles/310530/Target-Eye-Revealed-part-Target-Eyes-Unique-Auto
If your software is open sourced, and target Linux or developers. It is interesting to install your software as a git repo. And having it pull the stable branch occasionally or everytime when it is launched.
This is particular easy when your application is managed via npm, sbt, mavan, stack, elm-package or alike.
After hours of searching some working solution for this problem I've finally implemented auto update mechanism for python script that works on Linux and Windows.
In short - the script before running actual work checks for update on S3 and if it's available downloads it, unzips, creates or updates the symlink (or junction on Windows) and re-runs the script with already the new version with original arguments.
The full source code and the explanation can be found here.
If you are searching for an cross-platform software update solution, take a look at www.updatenode.com
Some highlights:
free for Open Source projects
cross-platform & Open Source update client tool
localized already for the most important languages
easy to integrate and easy to handle
cloud based management platform to define and manage updates
provides additionally support for displaying messages (inform about new events, products, etc.)
web interface is open (you can create your own client using the service)
many usage statistics, as used operating systems, geo location, version usage, etc.
Android API for mobile App updates
Just try it.
BTW, I am part of the dev team for the open source client. :)