I'm trying to translating the MySql C connector 6.02 headers to D, but I get some weird crashes.
My guess is I've made some mistakes on translating the structs or function (I'm not very good at C).
I used implib /system on the libmysql.dll to create a lib file.
I couldn't get htod.exe to work. Using -hs (include system files) complained it couldn't find system files.
Coffimplib.exe didn't have an option to prepend _ to exported internal names.
I couldn't find a free version of coff2omf.
mysql.d is the wrapper. I've included the C definition before each wrapped definition to easier spot bugs. The file includes the mysql dll and converted libfile too.
When compiling mytest_fails.d it crashes. mytest_works.d only has an assert, and this makes it work.. Compiling mytest_works with -release makes it crash too.
I've been using dmd 2.051
Download mytest.zip from share1t.com
Update: I've also asked some question regarding this on the D.learn newsgroup, but I don't think anyone has gone through the code.
C Const
Compiler extensions
Connot get htod.exe to work
The weird crashes
stdcall is a Windows function calling convention (very different from the C calling convention). HTOD failed to mark several functions with extern(Windows). This is normal, since HTOD isn't equipped to handle macros (STDCALL is defined as a macro, I can see that from the leftover comments in msyql.d).
Here's an updated mysql.d file:
http://dl.dropbox.com/u/9218759/mysql.d
Now, you need the proper import library in OMF format. I'd generally advise that you do not use implib for this. I've had several problems with it and others have reported having problems using it. Using coffimplib is the way to go. But first, you will need a COFF import library.
If you need it, the Mysql release with the COFF import library can be downloaded from here: http://dev.mysql.com/downloads/mirror.php?id=377977#mirrors (Libraries in DLL form marked for Visual Studio usually come with a COFF import library).
But I'm providing you the translated COFF import library in OMF format here: http://dl.dropbox.com/u/9218759/libmysql.lib
I've tried both of your test cases and they both seem to work fine now. In case of problems, try to check the translated header file again (mysql.d), it's possible that I might have missed to specify all the calling conventions properly.
Related
I've been experimenting with loading functions from the Windows system DLLs using only the loader functions exported by NTDLL. This works as expected. For the sake of curiosity and getting an even better understanding of the process structure in NT-based systems, I've started trying to load functions from NTDLL by doing the following steps:
Load the PEB of the process from gs:[60h]
Iterate over the modules loaded into the process according to the loader to find NTDLL's base address
Parse the PE headers of NTDLL
Try to parse the export table to find LdrLoadDll, LdrGetDllHandle, and LdrGetProcedureAddress
This fails at step 4. After stepping through it in a debugger (both VS2019 and WinDbg Preview), it seems as though the offsets I've tried yield an invalid structure that leads to an access violation when my code compares the current function name to one of the ones I'm searching for. My code is being compiled and run on a 64-bit copy of Windows 10 Pro build 21364. Note that I'm using my own header that contains definitions for the structures used for this (these definitions are from winnt.h and here) because the Windows headers don't really play nice with the rest of my code. The function trying to do this is here. For the record, this is part of an attempt to implement my own libc (again, for the sake of curiosity). The code that calls the functions is here. Any help with this is tremendously appreciated.
Nevermind, turns out I had outdated verbose definitions of the structures I was using. I found better (more up-to-date) definitions at https://vergiliusproject.com.
I would like to identify all functions needed to run a specific function in octave. I need this to deploy an application written in Octave.
While Matlab offers some tools to analyse a function on its dependencies, I could not find something similar for Octave.
Trying inmem as recommended in matlab does not produce the expected result:
> inmem
warning: the 'inmem' function is not yet implemented in Octave
Is there any other solution to this problem available?
First, let me point out that from your description, the matlab tool you're after is not inmem, but deprpt.
Secondly, while octave does not have a built-in tool for this, there is a number of ways to do so yourself. I have not tried these personally, so, ymmv.
1) Run your function while using the profiler, then inspect the functions used during the running process. As suggested in the octave archives: https://lists.gnu.org/archive/html/help-octave/2015-10/msg00135.html
2) There are some external tools on github that attempt just this, e.g. :
https://git.osuv.de/m/about
https://github.com/KaeroDot/mDepGen
3) If I had to attack this myself, I would approach the problem as follows:
Parse and tokenise the m-file in question. (possibly also use binary checks like isvarname to further filter useless tokens before moving to the next step.)
For each token x, wrap a "help(x)" call to a try / catch block
Inspect the error, this will be one of:
"Invalid input" (i.e. token was not a function)
"Not found" (i.e. not a valid identifier etc)
"Not documented" (function exists but has no help string)
No error, in which case you stumbled upon a valid function call within the file
To further check if these are builtin functions or part of a loaded package, you could further parse the first line of the "help" output, which typically tells you where this function came from.
If the context for this is that you're trying to check if a matlab script will work on octave, one complication will be that typically packages that will be required on octave are not present in matlab code. Then again, if this is your goal, you should probably be using deprpt from matlab directly instead.
Good luck.
PS. I might add that the above is for creating a general tool etc. In terms of identifying dependencies in your own code, good software engineering practices go a long way towards providing maintenable code and easily resolving dependency problems for your users. E.g: -- clearly identifying required packages (which, unlike matlab, octave does anyway by requiring such packages to be visibly loaded in code) -- similarly, for custom dependencies, consider wrapping and providing these as packages / namespaces, rather than scattered files -- if packaging dependencies isn't possible, you can create tests / checks in your file that throw errors if necessary files are missing, or at least mention such dependencies in comments in the file itself, etc.
According to Octave Compatibility FAQ here,
Q. inmem
A. who -functions
You can use who -function. (Note: I have not tried yet.)
First off I have very little Python experience and don't understand much about the terminology. Secondly I have never had to use a file path in Python before and therefore related terminology and methods may need to be explained (or have an easy to understand link, that isn't too strong in pythonic terminology like the Python Docs are).
I'm using modular programming and my functions are separate into three different files all in one file on my desktop C:\Users\User\Desktop\Folder_name\file1.py.
I plan on using all of the functions within and as such would like it to work similar to import random.
The problem is your python file doesn't appear in python search path.
A few ways to solve.
As #Jeff says:
import os
path = os.path.dirname(os.path.realpath(__file__))
sys.path.append(path)
this one will add the current file's directory to the search path in the current run context. With these lines in file /path/b.py, you can use /path/a.py or any python program in /path/.
Using IDEs function for it. Just run it from your IDE, they will fill the path.
Put your python file into any search path directory. This fulfiil your demand that using all of the functions within and as such would like it to work similar to import random. However it will make you running environment dirty. Better to use virtualenv.
If the dependent modules are in the same folder as the script you are executing:
import os
path = os.path.dirname(os.path.realpath(__file__))
sys.path.append(path)
If they're in a different spot you can just specify path to something manually, but you should try to make it robust to moving around by defining the path in relation to where you execute from.
I am using Delphi XE6 and using Datasnap and JSON in my project. There is a bug I want to correct in the VCL unit System.JSON.pas (in the TJSONString.ToString function) where it should be escaping backslash characters as well as quotes. In order to fix this I carried out the following :
Copied System.JSON.pas from the standard VCL source folder to my project source folder
Added System.JSON.pas to my project (using the newly copied file)
Fixed the bug and attempted to compile
I get the error 'Unit Data.DBXCommon was compiled with a different version of System.JSON.TJSONObject'
I can see that the Data.DBXCommon unit references System.JSON, so I guess the compiler is now seeing 2 versions - my fixed version and the standard VCL version.
What is the correct way to implement VCL changes to avoid this problem?
There are two common reasons for this issue:
You made changes to the interface section of the unit. You cannot do this without also re-compiling all units that use the unit you are modifying.
You re-compile the unit with different compiler options from those used to build it originally. Deal with that by ensuring the compiler options used to compile the unit you modify are the same as used by Embarcadero. Typically Embarcadero compiles with default options. Impose these directly in the source file being modified, right at the very top of the file.
Having said this, a recent question here on a similar topic could not be resolved using option 2 above. In that question, under XE6 only, the unmodified Classes unit could not be re-compiled and linked at all. Which makes me wonder if this particular technique has had its day. Perhaps it's not even possible. Before you give up, see if you can compile and link the unmodified unit.
More broadly, using a detour is generally an easier way to solve such problems as you face. Using a detour rather than re-compiling makes the management of the fix cleaner and simpler.
Update 1
I cannot get the unmodified System.JSON unit to re-compile and link. Which I think means that the issue raised in that other question is broader than just the Classes unit. I think you will find this a tricky hurdle to overcome and recommend the use of a detour.
Update 2
The problem that appears to have been introduced in XE6, seems to have been resolved by the release of XE7. The unmodified System.JSON unit will compile and link in XE7.
What if Delphi XE6 original System.JSON.dcu wasn't compiled with Delphi XE6 but it was compiled with one of the previous versions of Delphi.
You claim that you managed to implement your fix in Delphi XE2 using same approach by changing source and then recompiling System.JSON. SO I suggest you first make a comparison between original System.JSON files that ship with both Delphi XE2 and Delphi XE6.
If they are the same then the changed System.JSON.dcu that you managed to recompile with Delphi XE2 might also work with Delphi XE6.
I resolved a similar issue by :
Deleting the .dcu files which are on different versions ( i.e. conflicting files).
Re-build the project to create new .dcu files.
Here is my problem:
I have two C++ modules, A and B, which are built as dynamically-linked libraries. A offers basic math functions, and custom exception types. B is a higher level module that uses A.
B::someFunction() calls a function from A, and tries to catch custom exception A:MyExceptionFromA in order to convert it into a custom type B:MyExceptionFromB (since users of module B do not need to know about the implementation details of A).
Everything works fine as long as I remain in the C++ domain. However, if I expose B::someFunction() in python via boost python, the exception is not caught anymore in the C++ module.
I can catch std::runtime_error, from which A:MyExceptionFromA derives, and call typeid(e).name() to get the retrieve the correct mangled name, so I know the correct exception is thrown. Therefore I suspect that the problem comes from resolving this mangled symbol into the correct exception type.
I have found this link, which explains that "python uses [the insular] model to open extension modules, so that extension module writers don't need to know what symbols other extension modules might be using.". I'm suspecting this is part of the problem/solution, but I do not know enough about symbol resolution to figure out how to solve my problem.
Any ideas?
I found a work-around to my problem. Based on this and link text, I figured out that adding
import sys, dl
sys.setdlopenflags(dl.RTLD_NOW|dl.RTLD_GLOBAL)
before my includes solves the problem, by forcing python to open libraries in immediate, global mode. But I'm still hoping for an alternative solution, if there's one. As mentioned in the first link, I'm suspicious that this could have unforeseen effects (I already know that name clashing could be a problem, and I suspect performance can be affected as well, but are there other side effects?)