I am using a commercial tool interfaced with an homebrew tclsh(Synopsys EDA).
In their version, they removed the load command. Thus I cannot use third party libraries (Graphviz library in my case).
I wonder if there is a another way to import binary files (.so files)
The only command in standard Tcl that brings in a dynamic library is load. (OK, package require can do too, but that's because it can call load inside.) Without that command, you only have options like statically linking your own code in and creating the commands in the Tcl_AppInit function, but that's really unlikely to work if you're already using someone else's code that's already done that sort of thing.
The easiest approach might be to run a normal tclsh as a subprocess via exec tclsh script.tcl (run and wait for termination) or open |tclsh r+ (open pipeline). If they've not turned off those capabilities as well; you might be running in a safe interpreter where all those things are systematically disabled. I don't know of any way to break out of a standard safe interpreter (the mechanism for locking them down errs on the side of caution) so if that's the case, you'll just have to save the data you want to a file somewhere (by any mechanism that works; safe interpreters also can't touch the filesystem at all by default though that is often profiled back in in protected ways) and use a completely separate program to work with it.
Related
I am currently trying to solve a reversing challenge, where c code is compiled for a 32bit linux system.
To solve this challenge I am trying to make use of ghidra but am faced with a few issues. A bit of a summary what I have done up to this point:
I have two OS available to me, one 64bit Linux System on my Laptop and this 64bit Windows 10. Apparantly the programm was compiled with gcc without a -g option making ghidra fail to debug the programm. Manually debugging it with gdb in Terminal is possible but terrible to use (at least for me).
So all I can do is look at the assembler code in the CodeBrowser of Ghidra and its respective decomipled c code. With that I got to understand that some of the instructions are decrypted during the runtime of the programm and in order to further analyse the code, I want to be able to execute parts of the instructions to slowly but surely decrypt and understand the hidden parts of the programm.
That being said, the only issue here is that I do not know how I can do that. I have noticed that ghidra has the ability to run java code, but all the examples I looked at that were provided by ghidra allow me to only patch hardcoded instructions into the programm but not to actually execute/evaluate them.
My specific issue at hand is following part of the programm (green marked part):
Ghidra has all the knowledge it needs to execute this part and I just do not know how to do that. I could of cause do it by hand, but that is just boring and not really why I am doing these challenges and that is the same reason as why I am not looking for finished scripts that unpack this programm for me but for a way to execute my analysis.
Finally to summarize my question: I am asking for a way to execute the green marked decrypting part of the targeted programm in ghidra without starting the debugger (since the ghidra debugger keeps failing on me).
I think you are mixing up a few things here. You say:
the programm was compiled with gcc without a -g option making ghidra fail to debug the programm
The debug information added with -g makes it easier to analyze and debug a program because you have information that would have otherwise have to be recovered by reverse engineering. This should not have an influence on whether you can run the program under a debugger in the first place, and as you noted running it with gdb in the terminal works. The Ghidra debugger basically just runs gdb in the background and attaches to it to exchange information, so it should work.
You have a few options now:
1. Get the Ghidra Debugger to run with this binary
Whatever issue you are encountering with the Ghidra debugger is probably a valid question for https://reverseengineering.stackexchange.com/
From then on you can pursue your initial plan to solve this via debugging.
2. Write a GhidraScript to reimplement the decryption
Understand the basic idea of what you recognized correctly as some kind of decryption loop. Then you can use one of Ghidra's scripting options[0] to write a simple script that reimplements this decryption, but writes the decrypted values to the Ghidra memory directly.
Any scripting language will obviously include basic arithmetic operations like + -, and xor and loops, and the Ghidra API provides the functions byte getByte(Address address) and setByte(Address address, byte value). If you encounter any issues or API questions while writing this script that will also be a valid follow up question for the RE Stack Exchange.
This approach has the advantage that you can then statically analyse the resulting data inside Ghidra again, e.g. disassemble the resulting code.
[0] Ghidra natively supports Python 2.7 and Java based Scripts and a rudimentary Python REPL, but there are other options like Jupyter and Script based Kotlin or Ruby, Kotlin and Clojure Scripts
I think this is more of a Tcl configuration question rather than a Tcl coding question...
I inherited a whole series of Tcl scripts that are used within a simulation tool that my company built in-house. In my scripts, I'm finding numerous instances where there are function calls to functions that don't seem to be declared anywhere. How can I trace the path to these phantom functions?
For example, rather than use source, someone build a custom include function that they named INCLUDE. Tclsh obviously balks when I try to run it there, but with my simulation software, it runs fine.
I've tried grep-ing through the entire simulation software for INCLUDE, but I'm not having any luck. Are there any other obvious locations outside the simulation software where a Tcl function might be defined?
The possibilities:
Within your software. (you have checked for this).
Within some other package included by the software.
Check and see if the environment variable TCLLIBPATH is set.
Also check and see if the simulation software sets TCLLIBPATH.
This will be a list of directories to search for Tcl packages, and you
will need to search the packages that are located outside of the
main source tree.
Another possibility is that the locations are specified in the pkgIndex.tcl file.
Check any pkgIndex.tcl files and look for locations outside the main source tree.
Within an unknown command handler. This could be in
your software or within some other package. You should be able to find
some code that processes the INCLUDE statement.
Within a binary package. These are shared libraries that are loaded
by Tcl. If this is the case, there should be some C code used to
build the shared library that can be searched.
Since you say there are numerous instances of unknown functions, my first
guess is that you have
not found all the directories where packages are loaded from. But an
''unknown'' command handler is also a possibility.
Edit:
One more possibility I forgot. Check and see if your software sets the auto_path variable. Check any directories added to the auto_path for
other packages.
This isn't a great answer for you, but I suspect it is the best you're going to get...
The procedure could be defined in a great many places. Your best bet for finding it is to use a tool like findstr (on Windows) or grep -R (on POSIX platforms) to search across all the relevant source files. But that still might not help! It might not be a procedure but instead a general command, which could be implemented in C and not as a procedure, or it could be defined in a packaged application archive (which are usually awkward to look inside). There are also other types of script-implemented command too, which could make things awkward. Generally searching and investigating is your best bet, but it might not work.
Tcl doesn't really differentiate strongly between different types of command except in some introspection operations. If you're lucky, you could find that info body tells you the definition of the procedure (and info args and info default tell you about the arguments) but that won't help with other command types at all. Tcl 8.7 will include a command (info cmdtype) that would help a lot with narrowing down what to do next, but that's no use to you now and it definitely doesn't exist in older versions.
I have a strange problem I am using fedora 20 and installed tcllib on my system.
But if I use package require uri in example I got an package not found in response.
Does anyone know what the issue here is or how to determine if the tcllib is added in the package index?
Tcl looks up packages in two ways: with auto_path and with tcl::tm::path.
1. The auto_path — the traditional mechanism.
When you do package require, the package manager looks to see if the package is already present, or if instructions for obtaining the package from the filesystem are present. If neither of these is true, it asks the package unknown handler to load it (strictly, it's the handler installed using the package unknown command). The default implementation of that handler loads packages by looking for pkgIndex.tcl files in the directories on your auto_path, and their immediate subdirectories.
auto_path is a global variable holding a Tcl list of directories to search. You can probably just lappend the right place to it. pkgIndex.tcl is a Tcl script that describes how to make the package available, which it does by calling an appropriate package ifneeded command. The actual loading of the
Once a package is required that isn't present but its instructions for obtaining are, Tcl will simply eval those instructions: they're just a plain old script (that usually just calls source and/or load to do the grunt work).
2. Tcl modules — the new (in 8.5) mechanism.
The Tcl module system uses a different search system managed with the tcl::tm::path command. The tcl::tm::path list subcommand will tell you where it looks (a huge list, to be honest) and you can use the tcl::tm::path add subcommand to extend the path with extra locations to search. Tcl modules have the entire package placed into a single file (with the extension .tm) and have a structured name so that they can avoid having a separate pkgIndex.tcl file; the TM loader can synthesise the package ifneeded calls from the filename itself (in all cases, this is done with source; there are some clever ways to package binary code inside files so they can be loaded, but they're far outside the scope of this answer).
At that point, you're back to the source of the file when the package is actually required; that part is the same whether you're using a module or a traditional package.
The module system is much faster than the traditional search mechanism since it doesn't need to open any files to figure out what to do: it just uses glob with the right options. It is, however, less flexible in how things can be packaged: multi-file packages (e.g., almost anything you make yourself) can't be made into modules (well, not without extra work).
Is there a way to make Tcl interpreter source a file and open a pipe from shell command parallel?
In more details, I have a GUI built from tcl/tk. I want my tcl script to source a setting file for GUI variables, and at the same time, open a pipe from [tclsh setting_file] to redirect the output to my GUI stdout.
Thank you very much!
I'm not convinced that running the processing of the settings command in a subprocess is a good idea. Maybe a safe interpreter would be better?
Re trapping the output, you could pick a technique for doing stdout capture and then show the contents of the captured buffer in the GUI (after using encoding convertfrom to get the characters back if you're using my solution to that problem) but you've got a general issue that it is possible for user code to block things up if it takes a long time to run. You could work around that by using threads, but I suspect it is easier to avoid the complexity and to just let badly-written setup code cause problems that the user will have to fix. (The catch command can help you recover from any outright errors during the sourcing of the settings file.)
I am a new one to Common Lisp (using Clozure Common Lisp under Microsoft Windows), who is familiar with c and python before. So maybe the questions are stupid here, but be patient to give me some help.
1) What's is the usual way to run a common lisp script?
Now, I wrote a bat file under windows to call ccl exe(wx86cl.exe) and evaluate (progn (load "my_script_full_path") (ccl:quit)) every time when I want to "run" my script. Is this a standard way to "run" a script for common lisp?
Any other suggestion about this?
2) What's the difference between (require 'cxml) and (asdf:operate 'asdf:load-op :cxml)?
They are seems to be the same for my script, which one should I use?
3) ignore it, not a clear question
4) When I want to load some library (such as require 'cxml), it always takes time(3s or even 5s) to load cxml every time when I "run" my script, there is also much log to standard output I show below, it seems like checking something internal. Does it means I have to spent 3-5s to load cxml every time when I want to run a simple test? It seems like a little inefficient and the output is noisy. Any suggestion?
My Script
(require 'cxml) (some-code-using-cxml)
And the output
; Loading system definition from D:/_play_/lispbox-0.7/quicklisp/dists/quicklisp/software/cxml-20101107-git/cxml.asd into #<Package "ASDF0">
;;; Checking for wide character support... yes, using code points.
; Registering #<SYSTEM "cxml-xml">
......
some my script output
---EDIT TO ADD MORE----
5) I must say that I almost forget the way of dumping image to accelerate the loading speed of lisp library. So, what is the normal process for us to develop a (maybe very simple) lisp script?
Base on the answer of what I got now, I guess maybe
a) edit your script
b) test it via a REPL environment, SLIME is a really good choice, and there should be many loop between a <==> b
c) dump the image to distribute it?( I am no sure about this)
6) Furthermore, what is the common way/form for us to release/distribute the final program?
For a lisp library, we just release our source code, and let someone else can "load/require" them.
For a lisp program, we dump a image to distribute it when we confirm that all functions go well.
Am I right?
What form do we use in a real product? Do we always dump all the thing into a image at final to speed up the loading speed?
1) Yes, the normal way to run a whole programme is to use a launcher script. However, windows has much, much better scripting support these days than just the bat interpreter. Windows Scripting Host and PowerShell ship as standard.
1a) During development, it is usual to simply type things in a the REPL (Read-Eval-Print-Loop, i.e. the lisp command line), or to use something like SLIME (for emacs or xemacs) as a development environment. If you don't know what they are, look them up. You may wish to use Cygwin to install xemacs, which will give you access to a range of linux-ish tools.
2) Require is, IIRC, a part of the standard. ASDF is technically not, it is a library that operates to make libraries work more conveniently. ASDF has a bunch of features that you will eventually want if you really get into writing large Lisp programmes.
3) Question unclear, pass.
4) See 1a) - do your tests and modifications in a running instance, thus avoiding the need to load the library more than once (just as you would in Python - you found the python repl, right?). In addition, when your programme is complete, you can probably dump an image which has all of your libraries pre-loaded.
Edit: additional answers:
5) Yes
6) Once you have dumped the image, you will still need to distribute the lisp binary to load the memory image. To make this transparent to the user, you will also have to have a loader script (or binary) to run the lisp binary with the image.
You don't have to start the lisp from scratch and load everything over again each time you want to run a simple test. For more efficient development, interactively evaluate code in the listener (REPL) of a running lisp environment.
For distribution, I use Zachary Beane's Buildapp tool. Very easy to install and use.
Regarding distribution -
I wrote a routine (it's at home and unavailable at the moment) that will write out the current image as a standard executable and quit. It works for both CLISP and SBCL.
I can rummage it up if you like.