I have a function in Julia that requires to do things in a loop. A parameter is passed to the loop, and the bigger this parameter, the slower the function gets. I would like to have a message to know in which iteration it is, but it seems that Julia waits for the whole function to be finished before printing anything. This is Julia 1.4. That behaviour was not on Julia 1.3.
A example would be like this
function f(x)
rr=0.000:0.0001:x
aux=0
for r in rr
print(r, " ")
aux+=veryslowfunction(r)
end
return aux
end
As it is, f, when called, does not print anything until it has finished.
You need to add after the print command:
flush(stdout)
Explanation
The standard output of a process is usually buffered. The particular buffer size and behavior will depend on your system setting and perhaps the terminal type.
By flushing the buffer you make sure that the contents is actually sent to the terminal.
Alternatively, you can also use a library like ProgressLogging.jl (needs TerminalLoggers.jl to see actual output), or ProgressMeter.jl, which will automatically update a nicely formatted status bar during each step of the loop.
For example, with ProgressMeter, a call to
function f(x)
rr=0.000:0.0001:x
aux=0
#showprogress for r in rr
aux += veryslowfunction(r)
end
return aux
end
will show something like (in the end):
Progress: 100%|██████████████████████████████████████████████████████████████| Time: 0:00:10
Again I can't reproduce the behaviour in my terminal (it always prints), but I wanted to add that for these types of situations the #show macro is quite neat:
julia> function f(x)
rr=0.000:0.0001:x
aux=0
for r in rr
#show r
aux+=veryslowfunction(r)
end
return aux
end
f (generic function with 1 method)
julia> f(1)
r = 0.0
r = 0.0001
r = 0.0002
...
It uses println under the hood:
julia> using MacroTools
julia> a = 5
5
julia> prettify(#expand(#show a))
quote
Base.println("a = ", Base.repr($(Expr(:(=), :ibex, :a))))
ibex
end
Related
How can I declare a Julia function that returns a function with a specific signature. For example, say I want to return a function that takes an Int and returns an Int:
function buildfunc()::?????
mult(x::Int) = x * 2
return mult
end
What should the question marks be replaced with?
One thing needs to be made clear.
Adding a type declaration on the returned parameter is just an assertion, not part of function definition. To understand what is going on look at the lowered (this is a pre-compilation stage) code of a function:
julia> f(a::Int)::Int = 2a
f (generic function with 1 method)
julia> #code_lowered f(5)
CodeInfo(
1 ─ %1 = Main.Int
│ %2 = 2 * a
│ %3 = Base.convert(%1, %2)
│ %4 = Core.typeassert(%3, %1)
└── return %4
)
In this case since the returned type is obvious this assertion will be actually removed during the compilation process (try #code_native f(5) to see yourself).
If you want for some reason to generate functions I recommend to use the #generated macro. Be warned: meta-programming is usually an overkill for solving any Julia related problem.
#generated function f2(x)
if x <: Int
quote
2x
end
else
quote
10x
end
end
end
Now we have a function f2 where the source code of f2 is going to depend on the parameter type:
julia> f2(3)
6
julia> f2(3.)
30.0
Note that this function generation is actually happening during the compile time:
julia> #code_lowered f2(2)
CodeInfo(
# REPL[34]:1 within `f2'
┌ # REPL[34]:4 within `macro expansion'
1 ─│ %1 = 2 * x
└──│ return %1
└
)
Hope that clears things out.
You can use Function type for this purpose. From Julia documentation:
Function is the abstract type of all functions
function b(c::Int64)::Int64
return c+2;
end
function a()::Function
return b;
end
Which prints:
julia> println(a()(2));
4
Julia will throw exception for Float64 input.
julia> println(a()(2.0));
ERROR: MethodError: no method matching b(::Float64) Closest candidates are: b(::Int64)
I have code similar to the one below, where a function with a parameter depending on the loop iteration is plotted after every iteration. I would like to save the plot with the name trigplot_i.ps where i is the iteration number, but don't know how.
I have tried trigplot_"i".ps but didn't work, and have not been able to find how to cast i to a string either.
I'm a beginner so any help is very welcome.
f(x) := sin(x);
g(x) := cos(x);
for i:1 thru 10 do
(plot2d([i*f(x), i*g(x)], [x,-5,5],[legend,"sin(x)","cos(x)"],
[xlabel,"x"],[ylabel,"y"],
[ps_file,"./trigplot_i.ps"],
[gnuplot_preamble,"set key box spacing 1.3 top right"])
);
code after edits gives an error:
f(x) := sin(x);
g(x) := cos(x);
for i:1 thru 10
do block([myfile],
myfile: sconcat("./trigplot_", i, ".ps"),
printf (true, "iteration ~d, myfile = ~a~%", myfile),
plot2d([i*f(x), i*g(x)], [x,-5,5],[legend,"sin(x)","cos(x)"],
[xlabel,"x"],[ylabel,"y"],
[ps_file, myfile],
[gnuplot_preamble,"set key box spacing 1.3 top right"])
);
error:
"declare: argument must be a symbol; found "./trigplot_1.ps
-- an error.
To debug this try: debugmode(true);"
Looks good. To construct a file name, try this: sconcat("./trigplot_", i, ".ps") or also you can try: printf(false, "./trigplot_~d.ps", i). My advice is to make that a separate step in the loop, and then you can use it in the call to plot2d, e.g.:
for i:1 thru 10
do block ([myfile],
myfile: sconcat("./trigplot_", i, ".ps"),
printf (true, "iteration ~d, myfile = ~a~%", i, myfile),
plot2d (<stuff goes here>, [ps_file, myfile], <more stuff>));
EDIT: Fixed a bug in printf (omitted argument i).
I write a function that print all contents of file like that
let rec print_file channel =
try
begin
print_endline (input_line channel);
print_file channel
end
with End_of_file -> ()
And because of print_file is last operation I was thinking that it will be optimized to regular loop. but when I had runned my program on really big file, I had got stack overflow.
So I has tried to wrap input_line function to input_line_opt that don't raise exception and little change code of print_file.
let input_line_opt channel =
try Some (input_line channel)
with End_of_file -> None
let rec print_file channel =
let line = input_line_opt channel in
match line with
Some line -> (print_endline line; print_file channel)
| None -> ()
And now it works like regular tail recursive function and don't overflow my stack.
What difference between this two function?
In the first example, try ... with is an operation that occurs after the recursive call to print_file. So the function is not tail recursive.
You can imagine that try sets up some data on the stack, and with removes the data from the stack. Since you remove the data after the recursive call, the stack gets deeper and deeper.
This was a consistent problem in earlier revisions of OCaml. It was tricky to write tail-recursive code for processing a file. In recent revisions you can use an exception clause of match to get a tail position for the recursive call:
let rec print_file channel =
match input_line channel with
| line -> print_endline line; print_file channel
| exception End_of_file -> ()
Here is a simple function in Julia 0.5.
function foo{T<:AbstractFloat}(x::T)
a = zero(T)
b = zero(T)
return x
end
I started with julia --track-allocation=user. then include("test.jl"). test.jl only has this function. Run foo(5.). Then Profile.clear_malloc_data(). foo(5.) again in the REPL. Quit julia. Look at the file test.jl.mem.
- function foo{T<:AbstractFloat}(x::T)
- a = zero(T)
194973 b = zero(T)
0 return x
- end
-
Why is there 194973 bytes of memory allocated here? This is also not the first line of the function. Although after Profile.clear_malloc_data(), this shouldn't matter.
Let's clarify some parts of the relevant documentation, which can be a little misleading:
In interpreting the results, there are a few important details. Under the user setting, the first line of any function directly called from the REPL will exhibit allocation due to events that happen in the REPL code itself.
Indeed, the line with allocation is not the first line. However, it is still the first tracked line, since Julia 0.5 has some issues with tracking allocation on the actual first statement (this has been fixed on v0.6). Note that it may also (contrary to what the documentation says) propagate into functions, even if they are annotated with #noinline. The only real solution is to ensure the first statement of what's being called is something you don't want to measure.
More significantly, JIT-compilation also adds to allocation counts, because much of Julia’s compiler is written in Julia (and compilation usually requires memory allocation). The recommended procedure is to force compilation by executing all the commands you want to analyze, then call Profile.clear_malloc_data() to reset all allocation counters. Finally, execute the desired commands and quit Julia to trigger the generation of the .mem files.
You're right that Profile.clear_malloc_data() prevents the allocation for JIT compilation being counted. However, this paragraph is separate from the first paragraph; clear_malloc_data does not do anything about allocation due to "events that happen in the REPL code itself".
Indeed, as I'm sure you suspected, there is no allocation in this function:
julia> function foo{T<:AbstractFloat}(x::T)
a = zero(T)
b = zero(T)
return x
end
foo (generic function with 1 method)
julia> #allocated foo(5.)
0
The numbers you see are due to events in the REPL itself. To avoid this issue, wrap the code to measure in a function. That is to say, we can use this as our test harness, perhaps after disabling inlining on foo with #noinline. For instance, here's a revised test.jl:
#noinline function foo{T<:AbstractFloat}(x::T)
a = zero(T)
b = zero(T)
return x
end
function do_measurements()
x = 0. # dummy statement
x += foo(5.)
x # prevent foo call being optimized out
# (it won't, but this is good practice)
end
Then a REPL session:
julia> include("test.jl")
do_measurements (generic function with 1 method)
julia> do_measurements()
5.0
julia> Profile.clear_malloc_data()
julia> do_measurements()
5.0
Which produces the expected result:
- #noinline function foo{T<:AbstractFloat}(x::T)
0 a = zero(T)
0 b = zero(T)
0 return x
- end
-
- function do_measurements()
155351 x = 0. # dummy statement
0 x += foo(5.)
0 x # prevent foo call being optimized out
- # (it won't, but this is good practice)
- end
-
As a beginner in Z3, I am wondering if there is a way to make Z3Py work with a predefined Python function. Following is a small example which explains my question.
from z3 import *
def f(x):
if x > 0:
print " > 0"
return 1
else:
print " <= 0"
return 0
a=Int('a')
s=Solver()
s.insert(f(a) == 0)
t = s.check()
print t
print a
m = s.model()
print m
f(x) is a function defined in python. When x <= 0, f(x) returns 0. I add a constraint s.insert(f(a) == 0) and hope that Z3 can find a appropriate value for variable "a" (e.g. -3). But these codes are not working correctly. How should I change it?
(Please note that I need f(x) defined outside Z3, and then is called by Z3. )
What I am trying to do is calling a predefined function provided by a graph library without translating it to Z3. I am using the NetworkX library, and some codes are given as following:
import networkx as nx
G1=nx.Graph()
G1.add_edge(0,1)
G2=nx.Graph()
G2.add_edge(0,1)
G2.add_edge(1,2)
print(nx.is_isomorphic(G1, G2))
#False
I need Z3 to help me find a vertex in G2 such that after removing this vertex, G2 is isomorphic to G1. e.g.
G2.remove_node(0)
print(nx.is_isomorphic(G1, G2))
#True
I think this will be tough if f is a general function (e.g., what if it's recursive?), although if you assume it has some structure (e.g., if then else), you might be able to write a simple translator. The issue is that Z3's functions are mathematical in nature and not directly equivalent to Python functions (see http://en.wikipedia.org/wiki/Uninterpreted_function ). If possible for your purpose, I would propose to go the opposite direction: define your function f in terms of Z3 constructs, then evaluate it within your program (e.g., using this method: Z3/Python getting python values from model ). If that won't work for you, please include some additional details on how you need to use the function. Here's a minimal example (rise4fun link: http://rise4fun.com/Z3Py/pslw ):
def f(x):
return If(x > 0, 1, 0)
a=Int('a')
s=Solver()
P = (f(a) == 0)
s.add(P)
t = s.check()
print t
print a
m = s.model()
print m
res = simplify(f(m[a])) # evaluate f at the assignment to a found by Z3
print "is_int_value(res):", is_int_value(res)
print res.as_long() == 0 # Python 0
print f(1)
print simplify(f(1)) # Z3 value of 1, need to convert as above