I am trying to find ways to use oprofile on a stripped binary yet
still get detailed information when symbol tables are available
later, but so far I haven't find any solution.
Here is my situation: Our software is shipped to our customer stripped, but we have unstripped version on our build machine. When
we have a software crash, the backtrace can be sent back and we
can interpret it with local unstripped build.
Since oprofile is about taking samples and interpreting samples, is there a way to de-couple this process? Is there a way for oprofile/opreport to generate hex-address based profile information that can be interpretted on a different machine with all symbols available. Maybe I can do so by copying back the sample files?
I am sure it must be possible, so I am reaching out to the experts for advice. Detailed steps would be nice.
Thanks in advance.
I found a way to do it. It might not be the best, but I am surprised this didn't generate enough interest.
Say you have a binary called "mybin" and running its stripped version at customer site. Here would be my procedure:
Ask the customer (or your field engineer) run oprofile, the
whole nine yard (setup, start, dump and shutdown), then do:
tar czf OP-`date +"%Y%m%d%H%M%S"`.tgz /var/lib/oprofile
and ship that back.
On your build machine, do the following
mkdir /tmp/migrate && cd /tmp/migrate
tar xzf OP-*.tgz
Now you can check the overall CPU usage on target, by
opreport -% --session-dir=/tmp/migrate/var/lib/oprofile
If your "mybin" is built under /home/nobody/build/1.2.0/, you can use
opreport -l mybin --image-path /home/nobody/build/1.2.0/ --session-dir=/tmp/migrate/var/lib/oprofile
to see the details just for "mybin".
Related
I am creating a kbskit for my tcl executable application as follows on Suse :
./kbs.tcl -builddir=85 -r -mk-bi -bi="itcl3.4 itk3.4 iwidgets4.0.2 img1.4.1" install kbskit8.5
cp 85/bin/kbsmk8.5-bi kbsmk8.5-bi-run
./kbsmk8.5-bi sdx.kit wrap sim -runtime kbsmk8.5-bi-run
The application will be used on several flavours of linux like Redhat,Ubuntu etc. I am trying my best to test it myself under many combinations. Neverthless, i would be like to know someone thinks this would/wouldn’t work seamlessly across different platforms since I wont be able to cover all combinations exhaustively.
A Linux/x86 kbskit is at least reasonable to run on that collection of platforms. Unfortunately, the only way to be sure is to try. It should work, but if your script refers to files in a particular location and another platform (or deployment!) puts them elsewhere, then things will fail. The other thing that might go wrong is if there are significant incompatibilities in the small number of system libraries that Tcl uses, especially the C library; I do not know whether such problems exist, but I suspect they're not a major problem in practice.
You can try using the platform package (a standard part of Tcl since at least 8.5) to report what platform you're dealing with. That's the usual level of granularity you need to pay attention to.
package require platform
puts [platform::identify]
Essentially I want to rewrite a binary file to perform additional tasks regarding its actual tasks.
Regarding binary rewriting the process seems to be following:
Create a Control Flow Graph from an existing binary
Create a Code Snippet with the desired changes in an appropriate format
Create a binary file from the modified CFG
I came across a couple of tools, which either won't compile on my ubuntu 12.04, are not available for download or I can not find a decent tutorial / howto on how to hot patch / rewrite a binary. Those tools are:
ParseAPI, Code-Surfer/x86, EEL, LEEL, Jakstab, DynInst, Diablo + Lancet
To be more precise I want to analyze a given binary for its most frequently used functions and change it in such a way that before executing these functions, a given set of instructions are performed.
These instructions comprise of loading an array of stored bytes, reading a byte at a certain position and comparing it with a pre-defined value.
I want to make sure that the binary definitely executes these instructions during every trial.
There are 2 alternative approaches I came across which basically alter standard c functions (like memcpy(), strcpy(), printf(), etc.) since I assume these functions to be part of the binary with high probability:
LD_PRELOAD: Define my own libraries and let them get loaded before the ordinary ones
Compile the binary (of sourcecode is given) with own versions of the standard functions using something like gcc -fno-builtin -o strcpy strcpy.c
Drawback of this approach is that eventhough I subsitute standard c functions they do not necessarily have to get called, hence my instruction will not get executed neither.
Do you guys have experience regarding binary rewriting or do your have clues for accomplishing this rather exotic task?
Best regards!
BAP and Dyninst would help you. You may use BAP (http://bap.ece.cmu.edu/) to get the control flow graph of a binary. It have a very easy to use utility to create control flow graph from binaries. And you may use dyninst to instrument binaries and perform your desired operations. BAP absolutely runs on ubuntu12.04. Dyninst might not compile on 12.04 (there might be some linking problems). A simple walk around is that you do instrumentation on 10.04 and run the rewritten binaries on 12.04. Both tools are free.
I am a new one to Common Lisp (using Clozure Common Lisp under Microsoft Windows), who is familiar with c and python before. So maybe the questions are stupid here, but be patient to give me some help.
1) What's is the usual way to run a common lisp script?
Now, I wrote a bat file under windows to call ccl exe(wx86cl.exe) and evaluate (progn (load "my_script_full_path") (ccl:quit)) every time when I want to "run" my script. Is this a standard way to "run" a script for common lisp?
Any other suggestion about this?
2) What's the difference between (require 'cxml) and (asdf:operate 'asdf:load-op :cxml)?
They are seems to be the same for my script, which one should I use?
3) ignore it, not a clear question
4) When I want to load some library (such as require 'cxml), it always takes time(3s or even 5s) to load cxml every time when I "run" my script, there is also much log to standard output I show below, it seems like checking something internal. Does it means I have to spent 3-5s to load cxml every time when I want to run a simple test? It seems like a little inefficient and the output is noisy. Any suggestion?
My Script
(require 'cxml) (some-code-using-cxml)
And the output
; Loading system definition from D:/_play_/lispbox-0.7/quicklisp/dists/quicklisp/software/cxml-20101107-git/cxml.asd into #<Package "ASDF0">
;;; Checking for wide character support... yes, using code points.
; Registering #<SYSTEM "cxml-xml">
......
some my script output
---EDIT TO ADD MORE----
5) I must say that I almost forget the way of dumping image to accelerate the loading speed of lisp library. So, what is the normal process for us to develop a (maybe very simple) lisp script?
Base on the answer of what I got now, I guess maybe
a) edit your script
b) test it via a REPL environment, SLIME is a really good choice, and there should be many loop between a <==> b
c) dump the image to distribute it?( I am no sure about this)
6) Furthermore, what is the common way/form for us to release/distribute the final program?
For a lisp library, we just release our source code, and let someone else can "load/require" them.
For a lisp program, we dump a image to distribute it when we confirm that all functions go well.
Am I right?
What form do we use in a real product? Do we always dump all the thing into a image at final to speed up the loading speed?
1) Yes, the normal way to run a whole programme is to use a launcher script. However, windows has much, much better scripting support these days than just the bat interpreter. Windows Scripting Host and PowerShell ship as standard.
1a) During development, it is usual to simply type things in a the REPL (Read-Eval-Print-Loop, i.e. the lisp command line), or to use something like SLIME (for emacs or xemacs) as a development environment. If you don't know what they are, look them up. You may wish to use Cygwin to install xemacs, which will give you access to a range of linux-ish tools.
2) Require is, IIRC, a part of the standard. ASDF is technically not, it is a library that operates to make libraries work more conveniently. ASDF has a bunch of features that you will eventually want if you really get into writing large Lisp programmes.
3) Question unclear, pass.
4) See 1a) - do your tests and modifications in a running instance, thus avoiding the need to load the library more than once (just as you would in Python - you found the python repl, right?). In addition, when your programme is complete, you can probably dump an image which has all of your libraries pre-loaded.
Edit: additional answers:
5) Yes
6) Once you have dumped the image, you will still need to distribute the lisp binary to load the memory image. To make this transparent to the user, you will also have to have a loader script (or binary) to run the lisp binary with the image.
You don't have to start the lisp from scratch and load everything over again each time you want to run a simple test. For more efficient development, interactively evaluate code in the listener (REPL) of a running lisp environment.
For distribution, I use Zachary Beane's Buildapp tool. Very easy to install and use.
Regarding distribution -
I wrote a routine (it's at home and unavailable at the moment) that will write out the current image as a standard executable and quit. It works for both CLISP and SBCL.
I can rummage it up if you like.
I'm working on a Mercurial GUI client that interacts with hg.exe through the command line (the preferred high-level API, as I understand it).
However, I am having trouble determining the possible outputs of each command. I can see several outputs by simulating situations, but I was wondering if there is a complete reference of the possible outputs for each command.
For instance, for the command hg fetch, some possible outputs are:
pulling from https://User#server.com/Repo
searching for changes
no changes found
if there are no changes, or:
abort: outstanding uncommitted changes
or one of several other messages, depending on the situation.
I would like to structure my program to handle as many of these cases as possible, but it's hard for me to know in advance what they all are.
Is there a documented reference for the command-line? I have not been able to find one with The Google.
Look through the translation strings file. Then you'll know you have every message handled and be able to see what parts of it vary.
Also, fetch is just a convenience wrapper around pull/update/merge. If you're invoking mercurial programmatically you probably want to keep those three very different concepts separate in your running it so you know which part failed. In your example above it's the 'update' failing, so the 'pull' would have succeeded and the 'update's failing would allow you to provide the user with a better message.
(fetch is an abomination, which is part of why it's disabled by default)
Is this what you were looking for: https://www.mercurial-scm.org/wiki/MercurialBook ?
Mercurial 1.9 brings a command server, a stable (in a sense that API doesn't change that much) and low overhead (there is no need to run hg process for every command). The communication is done via a pipe.
Everyone managing open-source-software runs into the problem, that with the time the process of releasing a new version gets more and more work. You have to tag the release in your version-control, create the distributions (that should be easy with automated builds), upload them to your website and/or open-source-hoster. You have to announce the new release with nearly the same message on chosen web-forums, the news-system on sourceforge, mailinglists and your blog or website. And you have to update the entry of your software on freshmeat. Possible more tasks have to be done for the release.
Do you developed techniques to automate some of these tasks? Does software exist that supports you with this?
Pragmatic Project Automation shows how to do all of that. They use Ant for practically everything in the book, so if you know Ant you can make different targets to do any step in the build-release cycle.
For my Perl stuff, I wrote Module::Release. In the top-level directory I type a single command:
% release
If checks several things and dies if anything is wrong. If everything checks out, it uploads the distribution.
It automates my entire process:
Test against multiple versions of Perl
Test distribution files
Check the status of source control
Check for code and distribution quality metrics
Update changes file
Determine new version number
Release code to multiple places
Tag source control with new version number
Everyone seems to write their own release automator though. Most people like their process how they like their process, so general solutions don't work out that well socially.
Brad Fitzpatrick has ShipIt which is a Perl program to automate releases. There's slightly more info in his original announcement.