I am trying to write a simple function to test why numba.cuda isn't working. The function should set a variable to a fixed value. When I call the function, it seems to compile, but nothing happens. I added, that it should raise an exception, just to see, that it gets called, but again nothing happens. I don't get any kind of exception to give me a hint, why it is not working.
Function:
from numba import cuda
#cuda.jit # also tried it with brackets: #cuda.jit()
def cuda_func(out):
out = 1
raise NameError('MyException')
I call the funtction like this:
>>> import Cuda_Class
>>> a = 0
>>> Cuda_Class.cuda_func[1, 1](a)
>>> a
0
numba.cuda.is_available returns True.
I am working inside a miniconda environment and had some trouble with installing cuda. I accidentally installed multiple versions, so I had to purge everything, and installed cuda 10.2 in my base environment. In the conda environment I installed the cudatoolkit (10.2.89).
I set CUDA_HOME to /usr/local/cuda-10.2. So nvcc --version gives me the right version. So the compiler is accessible.
NUMBA_CUDA_DRIVER should lead to cudalib.so, which I had trouble finding. I did not manually install the nvidia driver, it was installed in combination with cuda. I found cudalib.so under /usr/local/cuda-10.2/targets/x86_64-linux/lib/stubs/libcuda.so. There was no other file named libcuda.so, only libcuda.so.7. However, even before I set NUMBA_CUDA_DRIVER and it was empty, the behavior was exactly the same. No reaction, no exceptions. It looks as if the function would be called correctly, but nothing happens.
The only idea I have left is, that maybe it is a problem, that libcuda.so is in a "stubs" folder?
The first comment solved my problem:
I tried to change the value of an immutable Integer.
Although raise is supported by numba.cuda, apparently exceptions are not supported.
Neither of those mistakes lead to an error message.
Changing "out" to an array and manipulating the first value works.
Related
If I made a python file named hello.py that has a script made like this.
msg = input("insert your message here: ")
script = '''
def say_something():
print("{msg}")
'''
exec(script)
say_something()
And then I tried to use Cython
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules=cythonize("Hello.py")
)
It will show an error like this: undeclared name not builtin: say_something
I do understand why this happens but I'm not really an expert with python and C just yet. This is just an example, but it's similar to what I'm trying to do with one of my projects. Is there any way I could resolve this? I want to find a way to convert the script string into C as well.
I was trying to build an editable python script.
Cython compiles the Python functions to a native binary that does what the CPython interpreter should do. exec is a function that execute arbitrary code at runtime (which is generally a very bad idea for speed, maintainability/readability and security). Cython does not support exec because it would mean that the could would be compiled at runtime. Thus, the code executed by exec cannot be a Cython code. However, the exec function can still be used to execute a pure-Python code. The error can be removed by turning off the Cython.Compiler.Options.error_on_unknown_names in the setup script (just before calling setup) as pointed out by #DavidW. With this Cython will not complain when it does not find a function defined by exec (or similar methods). Please keep in mind that CPython can only be used in this case instead of Cython (which partially defeat the purpose of using Cython in the first place).
when I reverse the binary with IDA gui, all the functions get decompiled without a problem.
but when I am running an automatic script on ida without gui, there is always the same function, that refuses to be decompiled. (when I am openning the same IDB that the automation script worked on, the function get decompiled without a problem)
I am using bip. and using BipFunc.can_decompile to check if a function can get decompiled.
EDIT:
according to an answer bellow, I have tried to add the following:
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, trying again")
decomp_all()
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, trying again")
decomp_all_twice_cacheclear()
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, skipping...")
return
sadly it did not work, I get all 3 prints every time, even on different binaries
it seems to be fixed on IDA Pro 7.6
There is several reason you can get an error on the decompilation from IDA. If it works on some case and other it does not it is probably because of the call analysis. When decompiling a function IDA will try to gather information on the function called by this one and in some case fail to get those information which will make the decompilation fail. But once that function has been decompiled, the information fetched by IDA will be updated, and so the decompilation of the caller function might now work. So basically it means you have to decompile the function in an order, which is a pain, for fixing that the simplest way is to just decompile everything twice, but it can take quite some time if you do it on "big" binaries.
I though I put that in the Bip repository somewhere but I can't find it, so here is a small plugin/code which should allows to do that:
from bip import *
class DecompileAll(BipPlugin):
"""
Plugin for decompiling all the function in the binary.
"""
#menu("Bip/DecompileAll/", "Invalidate hexrays caches")
def clear_hxcCache(self):
HxCFunc.invalidate_all_caches()
#menu("Bip/DecompileAll/", "Decompile all func")
def decomp_all(self):
count = 0
for f in HxCFunc.iter_all():
count += 1
print("0x{:X} functions decompiled".format(count))
#menu("Bip/DecompileAll/", "Decompile twice with cache clear")
def decomp_all_twice_cacheclear(self):
HxCFunc.invalidate_all_caches()
self.decomp_all()
self.decomp_all()
Just for information the basic reason for decompilation error, is that it is not able to make a correct translation of some piece of code because it does not understand the assembly, this is typically true if there is a problem during the analysis and the code is not correctly detected (also happens a lot if you are dealing with obfuscation). You can typically view this case by an error telling you "failed analysis at ADDR" in the IDAPython console, and then look at the problem. Probably not your case but might still help.
Glad to hear you are using bip. So about the BipFunc.can_decompile function: like indicated in the documentation (https://synacktiv.github.io/bip/build/html/base/func.html#bip.base.BipFunction.can_decompile) it will just try to decompile the function and see if an error occurs. The code is pretty straight forward (https://github.com/synacktiv/bip/blob/master/bip/base/func.py#L372), this is mostly written for being done while using one-liner, its the same thing as catching the exception when trying to decompile.
I'm trying to deploy an app to production and getting a little confused by environment and application variables and what is happening at compile time vs runtime.
In my app, I have a genserver process that requires a token to operate. So I use config/releases.exs to set the token variable at runtime:
# config/releases.exs
import Config
config :my_app, :my_token, System.fetch_env!("MY_TOKEN")
Then I have a bit of code that looks a bit like this:
defmodule MyApp.SomeService do
use SomeBehaviour, token: Application.get_env(:my_app, :my_token),
other_config: :stuff
...
end
In production the genserver process (which does some http stuff) gives me 403 errors suggesting the token isn't there. So can I clarify, is the use keyword getting evaluated at compile time (in which case the application environment doest exist yet)?
If so, what is the correct way of getting runtime environment variables in to a service like this. Is it more correct to define the config in application.ex when starting the process? eg
children = [
{MyApp.SomeService, [
token: Application.get_env(:my_app, :my_token),
other_config: :stuff
]}
...
]
Supervisor.start_link(children, opts)
I may have answered my own questions here, but would be helpful to get someone who knows what they're doing confirm and point me in the right way. Thanks
elixir has two stages: compilation and runtime, both written in Elixir itself. To clearly understand what happens when one should figure out, that everything is macro and Elixir, during compilation stage, expands these macros until everything is expanded. That AST comes to runtime.
In your example, use SomeBehaviour, foo: :bar is implicitly calling SomeBehaviour.__using__/1 macro. To expand the AST, it requires the argument (keyword list) to be expanded as well. Hence, Application.get_env(:my_app, :my_token) call happens in compile time.
There are many possibilities to move it to runtime. If you are the owner of SomeBehaviour, make it accept the pair {:my_app, :my_token} and call Application.get_env/2 somewhere from inside it.
Or, as you suggested, pass it as a parameter to children; this code belongs to function body, meaning it won’t be attempted to expand during compilation stage, but would rather be passed as AST to the resulting BEAM to be executed in runtime.
Environment Details
Mac OS X 10.9
Oracle JDK 1.7.0_55 64-bit
jython-standalone-2.5.3.jar
junit-4.11
What I have done so far
I have added the junit jar to /Library/Java/Extensions.
I invoked Jython as follows java -jar jython-standalone-2.5.3.jar
In the Jython interpreter, I imported the following import org.junit.Assert, and this import was successful.
Problem
When I tried to use assertTrue, I got a NameError in the interpreter. Why is this so?
I understand that assertTrue is a static method. Not sure what implication this has when I try to use it in Jython.
Additional Context
I am using XMLUnit in Jython. Was able to successfully import the Diff class from org.custommonkey.xmlunit in Jython. Also able to use the methods in this class, and call them on a Diff object. The result of this method call is what I am trying to pass to assertTrue, when it throws the error.
from org.custommonkey.xmlunit import Diff
import org.junit.Assert
xml1 = ...some XML string...
xml2 = ...some XML string...
myDiff = Diff(xml1, xml2)
assertTrue(myDiff.similar())
Hope this additional information is useful in identifying a solution to this problem.
Latest Status
I narrowed it down to setting this property python.security.respectJavaAccessibility = false, since the Assert() constructor is protected.
Still trying to get it to work. Any help is greatly appreciated.
Figured it out.
In addition to junit.jar file, the hamcrest-core.jar file also needed to be copied to /Library/Java/Extensions.
Then I got rid of the jython.jar file, and instead installed it using the jython installer.
After the installation was completed, I updated the registry file in the installation folder, specifically setting this property python.security.respectJavaAccessibility = false.
Now I am able to see the assertTrue method, and no longer getting a NameError.
I am trying to use multicore-0.2.15 toolbox with Octave v3.6.4 on Windows7 64bit
( http://octave.sourceforge.net/multicore/ )
but even the demo script doesn't seem to work, it's not possible to create a pipe and I received an error message. So if I try to evaluate the following command in Octave
[read_fd, write_fd, err, msg] = pipe ()
I receive the following output:
read_fd = -1
write_fd = -1
err = -1
msg = pipe: not supported on this system
The fork function doesn't work either.
Does anyone have an idea what the problem might be?
Zoltan
The error message pipe: not supported on this system says it all. There is no support for pipe() in your system (Windows 7). You can:
Not use the multicore which you will notice is unmaintained (see the unmaintained section at the bottom of the package list). You could instead use the parallel package.
Try another build of Octave. Maybe the MinGW builds will work with pipes.
Try another version of Octave. Version 3.8.1 has already been released and if it is a problem on the side of Octave instead of Windows, maybe it has been fixed.
Change operating system (pipe() works fine in Debian)