Is there a ready or idiomatic way of declaring an entry point in a Julia program (i.e. the equivalent of main in C or the if __name__ == "__main__" construct in Python)?
This seems to be an important functionality in order to write larger pieces of structured code that won't be used in interactive mode but I couldn't find any hints as to how this is accomplished in Julia, if at all (a possible escape route could be writing an arbitrary function to serve as main and then calling it once on the top level at the end of the main module but that's not elegant and maybe not even efficient). TIA.
You could write a main function and not call it from the top level of the file. To run the program from the command line you would use julia -L file.jl -e 'main(some,args)'. The -L switch tells Julia to load your file, and then -e tells it to evaluate the following expression. There is also an -E switch that evaluates and prints (I think of it as "evaluating out loud", since capital letters seem "loud").
This has a couple of advantages over C's main or Python's if __name__ == "__main__":
You don't have to have a single entry point! You can evaluate any expression at all after loading your file, so you don't have to cram all your command line functionality into one function.
The calls you write use full Julia syntax, so often you can avoid parsing the arguments. Soemthing like -e main(53) calls main with the integer 53, no need for atoi inside main.
When modules are loaded, if they have a function called __init__ it will be called. Does that help?
If you want to do what the if __name__ == "__main__": idiom in python does, I found that
if !isdefined(Base, :active_repl)
main()
end
does the trick.
I often find myself wanting to be able to load my main file into a REPL and selectively poke at some of the functions without invoking main or staple a CLI onto a module that is mostly intended as a library module, so I really like this trick from python.
Related
If I made a python file named hello.py that has a script made like this.
msg = input("insert your message here: ")
script = '''
def say_something():
print("{msg}")
'''
exec(script)
say_something()
And then I tried to use Cython
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules=cythonize("Hello.py")
)
It will show an error like this: undeclared name not builtin: say_something
I do understand why this happens but I'm not really an expert with python and C just yet. This is just an example, but it's similar to what I'm trying to do with one of my projects. Is there any way I could resolve this? I want to find a way to convert the script string into C as well.
I was trying to build an editable python script.
Cython compiles the Python functions to a native binary that does what the CPython interpreter should do. exec is a function that execute arbitrary code at runtime (which is generally a very bad idea for speed, maintainability/readability and security). Cython does not support exec because it would mean that the could would be compiled at runtime. Thus, the code executed by exec cannot be a Cython code. However, the exec function can still be used to execute a pure-Python code. The error can be removed by turning off the Cython.Compiler.Options.error_on_unknown_names in the setup script (just before calling setup) as pointed out by #DavidW. With this Cython will not complain when it does not find a function defined by exec (or similar methods). Please keep in mind that CPython can only be used in this case instead of Cython (which partially defeat the purpose of using Cython in the first place).
when I reverse the binary with IDA gui, all the functions get decompiled without a problem.
but when I am running an automatic script on ida without gui, there is always the same function, that refuses to be decompiled. (when I am openning the same IDB that the automation script worked on, the function get decompiled without a problem)
I am using bip. and using BipFunc.can_decompile to check if a function can get decompiled.
EDIT:
according to an answer bellow, I have tried to add the following:
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, trying again")
decomp_all()
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, trying again")
decomp_all_twice_cacheclear()
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, skipping...")
return
sadly it did not work, I get all 3 prints every time, even on different binaries
it seems to be fixed on IDA Pro 7.6
There is several reason you can get an error on the decompilation from IDA. If it works on some case and other it does not it is probably because of the call analysis. When decompiling a function IDA will try to gather information on the function called by this one and in some case fail to get those information which will make the decompilation fail. But once that function has been decompiled, the information fetched by IDA will be updated, and so the decompilation of the caller function might now work. So basically it means you have to decompile the function in an order, which is a pain, for fixing that the simplest way is to just decompile everything twice, but it can take quite some time if you do it on "big" binaries.
I though I put that in the Bip repository somewhere but I can't find it, so here is a small plugin/code which should allows to do that:
from bip import *
class DecompileAll(BipPlugin):
"""
Plugin for decompiling all the function in the binary.
"""
#menu("Bip/DecompileAll/", "Invalidate hexrays caches")
def clear_hxcCache(self):
HxCFunc.invalidate_all_caches()
#menu("Bip/DecompileAll/", "Decompile all func")
def decomp_all(self):
count = 0
for f in HxCFunc.iter_all():
count += 1
print("0x{:X} functions decompiled".format(count))
#menu("Bip/DecompileAll/", "Decompile twice with cache clear")
def decomp_all_twice_cacheclear(self):
HxCFunc.invalidate_all_caches()
self.decomp_all()
self.decomp_all()
Just for information the basic reason for decompilation error, is that it is not able to make a correct translation of some piece of code because it does not understand the assembly, this is typically true if there is a problem during the analysis and the code is not correctly detected (also happens a lot if you are dealing with obfuscation). You can typically view this case by an error telling you "failed analysis at ADDR" in the IDAPython console, and then look at the problem. Probably not your case but might still help.
Glad to hear you are using bip. So about the BipFunc.can_decompile function: like indicated in the documentation (https://synacktiv.github.io/bip/build/html/base/func.html#bip.base.BipFunction.can_decompile) it will just try to decompile the function and see if an error occurs. The code is pretty straight forward (https://github.com/synacktiv/bip/blob/master/bip/base/func.py#L372), this is mostly written for being done while using one-liner, its the same thing as catching the exception when trying to decompile.
I am trying to find a clean way to access the regmap that is used with *RegisterNode for creating documentation and testing files. The TLRegisterNode has methods for generating the json through some Annotations. These are done in the regmap method by adding them to the ElaborationArtefacts object. Other protocols don't seem to have these annotations.
Is there anyway to iterate over the "regmap" Register Fields post elaboration or during?
I cannot just access the regmap as it's not really a val/var since it's a method. I can't quite figure out where this information is being stored. I don't really believe it's actually "storing" any information as much as it is simply creating the hardware to attach the specified logic to the RegisterNode based logic.
The JSON output is actually fine for me as I could just write a post processing script to convert JSON to my required formats, but I'm wondering if I can access this information OR if I could add a custom function call at the end. I cannot extend the case class *RegisterNode, but I'm not sure if it's possible to add custom functions to run at the end of the regmap method.
Here is something I threw together quickly:
//in *RegisterRouter.scala
def customregmap(customFunc: (RegField.Map*) => Unit, mapping: RegField.Map*) = {
regmap(mapping:_*)
customFunc(mapping:_*)
}
def regmap(mapping: RegField.Map*) = {
//normal stuff
}
A user could then create a custom function to run and pass it to the regmap or to the RegisterRouter
def myFunc(mapping: RegField.Map*): Unit = {
println("I'm doing my custom function for regmap!")
}
// ...
node.customregmap(myFunc,
0x0 -> coreControlRegFields,
0x4 -> fdControlRegFields,
0x8 -> fdControl2RegFields,
)
This is just a quick example I have. I believe what would be better, if something like this was possible, would be to have a Seq of functions that could be added to the RegisterNode that are ran at the end of the regmap method, similar to how TLRegisterNode currently works. So a user could add an arbitrary number and you still use the regmap call.
Background (not directly part of question):
I have a unified register script that I have built over the years in which I describe the registers for a particular IP. It works very similar to the RegField/node.regmap, except it obviously doesn't know about diplomacy and the like. It will generate the Verilog, but also a variety of files for DV (basic `defines for simple verilog simulations and more complex uvm_reg_block defines also with the ability to describe multiple of the IPs for a subsystem all the way up to an SoC level). It will also print out C Header files for SW and Sphinx reStructuredText for documentation.
Diplomacy actually solves one of the main issues I've been dealing with so I'm obviously trying to push most of my newer designs to Chisel/Diplo.
I ended up solving this by creating my own RegisterNode which is the same as the rocketchip RegisterNodes except that I use a different Elaboration Artifact to grab the info and store it for later.
I'm trying to deploy an app to production and getting a little confused by environment and application variables and what is happening at compile time vs runtime.
In my app, I have a genserver process that requires a token to operate. So I use config/releases.exs to set the token variable at runtime:
# config/releases.exs
import Config
config :my_app, :my_token, System.fetch_env!("MY_TOKEN")
Then I have a bit of code that looks a bit like this:
defmodule MyApp.SomeService do
use SomeBehaviour, token: Application.get_env(:my_app, :my_token),
other_config: :stuff
...
end
In production the genserver process (which does some http stuff) gives me 403 errors suggesting the token isn't there. So can I clarify, is the use keyword getting evaluated at compile time (in which case the application environment doest exist yet)?
If so, what is the correct way of getting runtime environment variables in to a service like this. Is it more correct to define the config in application.ex when starting the process? eg
children = [
{MyApp.SomeService, [
token: Application.get_env(:my_app, :my_token),
other_config: :stuff
]}
...
]
Supervisor.start_link(children, opts)
I may have answered my own questions here, but would be helpful to get someone who knows what they're doing confirm and point me in the right way. Thanks
elixir has two stages: compilation and runtime, both written in Elixir itself. To clearly understand what happens when one should figure out, that everything is macro and Elixir, during compilation stage, expands these macros until everything is expanded. That AST comes to runtime.
In your example, use SomeBehaviour, foo: :bar is implicitly calling SomeBehaviour.__using__/1 macro. To expand the AST, it requires the argument (keyword list) to be expanded as well. Hence, Application.get_env(:my_app, :my_token) call happens in compile time.
There are many possibilities to move it to runtime. If you are the owner of SomeBehaviour, make it accept the pair {:my_app, :my_token} and call Application.get_env/2 somewhere from inside it.
Or, as you suggested, pass it as a parameter to children; this code belongs to function body, meaning it won’t be attempted to expand during compilation stage, but would rather be passed as AST to the resulting BEAM to be executed in runtime.
The following code defines a simple Cython function (using Ipython magic, for convenience).
%load_ext cython
%%cython
def f(float x, float y=2):
return x+y
Then, calling help(f) gives this message:
Help on built-in function f in module _cython_magic_e37eeabbc63d5167217465ba978239fc:
f(...)
Note that the arguments of f are not shown. Also, the tab-completion does not work either for the argument names in ipython (e.g. typing f(x then tab).
If I define this function without using Cython:
def g(x, y=2):
return x+y
Calling help(g) gives this and the tab-completion works as expected:
Help on function g in module __main__:
g(x, y=2)
Is there a way to get this behavior with the Cython function? I tried with def, cdef, cpdef, with and without ipython magic, with no success.
Disagree with the agreed answer.
While enabling binding does have the side-effect of documentation strings showing up in code, it also binds all other Python class attributes to Cython extension classes which makes for lower performance, more memory used for each extension object and so on.
The correct flag to enable docstrings only without side effects is embedsignature=True.
It can either be used as decorator - #cython.embedsignature(True) on top of all functions, or part of cython directives in setup.py Extension to apply to all Cython functions - {'embedsignature': True}
From docs:
embedsignature (True / False)
If set to True, Cython will embed a textual copy of the call signature in the docstring of all Python visible functions and
classes. Tools like IPython and epydoc can thus display the signature,
which cannot otherwise be retrieved after compilation. Default is
False.
import cython
#cython.binding(True)
def f(float x, float y=2):
# ...
now help(f) gives
Help on cython_function_or_method in module cy_exc:
f(x, y=2.0)
The documentation says
When enabled, functions will bind to an instance when looked up as a class attribute (hence the name) and will emulate the attributes of Python functions, including introspections like argument names and annotations. Default is False.
You can enable the compilation option in other ways (for example, if you want it enabled everywhere).
You might also want to look at this related question