I have a simple plan for my program: do all input parsing and data preprocessing with Python and then pass a lot of structures and call ONCE number-crunching function that I'm going to write in C.
Here goes my question: what would be faster (in terms of execution speed)? To call C function with ctypes or write it using cython?
Given an infinite amount of time to optimize your code Ctypes will probably be faster as you can push as much of the heavy lifting into compiled code as possible.
cython takes python-like code and converts it to C code with lots of caveats and exceptions. If cython were perfect, it would be as fast as coding in the C-API for python. cython is a fantastic tool for quick code dev, but is not quite that perfect yet.
Ctypes will require some manipulation of your input variables to get them into C cleanly. This may not be a problem if you are doing a fair bit of data preprocessing anyway.
As usual you need to decide how important the execution speed is compared with coding time/code readability/code reusability
it would seem strange to me to be only interested in execution speed, and still want to code any of it in python.
Conversely, the general paradigms of python coding would tend to contrast with having large sections of C code held together by python wrappers
Related
I have no problem with the IO Monad. But I want to understand the followings:
In All/almost Haskell tutorials/ text books they keep saying that getChar is not a pure function, because it can give you a different result. My question is: Who said that this is a function in the first place. Unless you give me the implementation of this function, and I study that implementation, I can't guarantee it is pure. So, where is that implementation?
In All/almost Haskell tutorials/ text books, it's said that, say (IO String) is an action that (When executed) it can give you back a value of type String. This is fine, but who/where this execution is taking place. Of course! The computer is doing this execution. This is OK too. but since I am only a beginner, I hope you forgive me to ask, where is the recipe for this "execution". I would guess it is not written in Haskell. Does this later idea mean that, after all, that a Haskell program is converted into a C-like program, which will eventually be converted into Assembly -> Machine code? If so, where one can find the implementation of the IO stuff in Haskell?
Many thanks
Haskell functions are not the same as computations.
A computation is a piece of imperative code (perhaps written in C or Assembler, and then compiled to machine code, directly executable on a processor), that is by nature effectful and even unrestricted in its effects. That is, once it is ran, a computation may access and alter any memory and perform any operations, such as interacting with keyboard and screen, or even launching missiles.
By contrast, a function in a pure language, such as Haskell, is unable to alter arbitrary memory and launch missiles. It can only alter its own personal section of memory and return a result that is specified in its type.
So, in a sense, Haskell is a language that cannot do anything. Haskell is useless. This was a major problem during the 1990's, until IO was integrated into Haskell.
Now, an IO a value is a link to a separately prepared computation that will, eventually, hopefully, produce a. You will not be able to create an IO a out of pure Haskell functions. All the IO primitives are designed separately, and packaged into GHC. You can then compose these simple computations into less trivial ones, and eventually your program may have any effects you may wish.
One point, though: pure functions are separate from each other, they can only influence each other if you use them together. Computations, on the other hand, may interact with each other freely (as I said, they can generally do anything), and therefore can (and do) accidentally break each other. That's why there are so many bugs in software written in imperative languages! So, in Haskell, computations are kept in IO.
I hope this dispels at least some of your confusion.
Will reading a file using TCL C API's and populating a TCL array be much faster compared
to doing the same with standard TCL. I have a large file about 100+MB which I need to read and set some hash entries. Using TCL C API's doesn't seems to provide atmost 2 to 4 times speed advantage. Is this usual or am I missing something?
You're unlikely to get much of a performance gain in this case, as when you're setting array entries from the C API, you're bearing much of the cost that you'd experience if you just wrote the code as Tcl inside a procedure. In particular, you could very easily get worse performance through using an inefficient sub-API; some of Tcl's API functions are not very fast (e.g., Tcl_SetVar) but they're kept because of the large amount of existing code that uses them (and the fact that the faster functions require more C code to use). Bear in mind that setting an array element requires a mandatory hash table lookup, and those have a real cost (despite the fact that Tcl uses a very fast — if rather stupid — hash).
What's more, you can get better performance by using a Tcl list or dictionary (depending on what exactly you want to store) and the C API to those is quite quick (especially for lists, which are really C arrays of Tcl_Obj references). What I don't know is whether doing that would be a suitable substitute for your purposes.
The C API is there primarily to allow you to write Tcl extensions and just exposes the routines that 'pure Tcl' itself is written in. In a case like you describe I wouldn't expect to see much performance difference and remember:
Premature optimization is the root of all evil (or at least most of
it) in programming.
Computer Programming as an Art (1974) by Donald Knuth
If you really need to load lots of data, maybe some extension like NAP (http://wiki.tcl.tk/4015) or similar would be appropriate?
We know that a compiler can be written in its own language using a trick known as bootstrapping. My question is whether this trick can be applied to interpreters as well?
In theory the answer is certainly yes, but there is one worry that the interpretation of source code will become more and more inefficient as we go through the iterations. Would that be a serious problem?
I'm bootstrapping a very dynamical system where the programs will be constantly changing, so it rules out a compiler.
Let me spell it out this way:
Let the i's be interpreters.
Let the L's be programming languages.
We can write i1 in machine code (lowest level), to interpret L1.
We then write i2 in L1, interpreting L2 -- a new language.
We then write i3 in L2, interpreting L3 -- another new language.
and so on...
We don't need any compiler above, just interpreters. Right?
It could be inefficient. That is my question, and how to overcome it if it is indeed inefficient.
That doesn't make sense. An interpreter doesn't produce a binary, so can't create something that can run itself standalone. Somewhere, ultimately, you need to have a binary that is the interpreter.
Example of a compiler bootstrapping. Let's say we have two languages A(ssembler) and C. We want to bootstrap a C compiler written in C. But we only have an assembler to start with.
Write basic C compiler in A
Write C compiler in C and compile with earlier compiler written in A
You now have a C compiler which can compile itself, you don't need A or the original compiler any more.
Later runs become just
Compile C program using compiler written in C
Now let's say you have an interpreted language instead, I'll call it Y. The first version can be called Y1, the next Y2 and so on. Let's try to "bootstrap" it.
First off we don't have anything that can interpret Y programs, we need to write a basic interpreter. Let's say we have a C compiler and write a Y1 interpreter in C.
Write Y1 interpreter in C, compile it
Write Y2 interpreter in Y1, run it on Y1 interpreter written in C
Write Y3 interpreter in Y2, run it on Y2 interpreter running on Y1 interpreter... Written in C.
The problem is that you can never escape the stack of interpreters as you never compile a higher level interpreter. So you're always going to need to compile and run that first version interpreter written in C. You can never escape it, which I think is the fundamental point of the compiler bootstrapping process. This is why I say your question does not make sense.
The answer depends on what is being interpreted. If you're targeting a virtual machine which interprets bytecode, and your language is being developed iteratively while the bytecode doesn't change, then it is not a given that you will lose performance along the way. There are plenty of examples of languages which are bootstrapped on a target VM which wasn't designed particularly for that language, and they don't suffer a significant performance hit as a direct result (Scala on the JVM, for instance).
Using the JVM for example, you'd write the first compiler in Java, which compiles your source language to JVM bytecode. Then you'd rewrite your compiler to do exactly the same thing but in your new source language. The resulting bytecode could be indistinguishable between the two. Note that this is not the same thing as writing an interpreter in an interpreted language, which will become slower with each iteration.
This sentence does not seem to make sense:
I'm bootstrapping a very dynamical system where the programs will be constantly changing, so it rules out a compiler.
No matter if you have an interpreter or a compiler: both will have to deal with something that is not changing, i.e. with your language. And even if the language is somehow "dynamic", then there will be a meta-language that is fixed. Most probably you also have some low-level code, or at least a data structure the interpreter is working with.
You could first design and formalize this low-level code (whatever it is) and write some program that can "run" this. Once you have this, you can add a stack of interpreters and as long as they all produce this low level code, efficiency should not be an issue.
You can indeed, and this is the approach used by squeak (and I believe many other smalltalks). Here is one approach to doing just that: https://github.com/yoshikiohshima/SqueakBootstrapper/blob/master/README
Anyone know any good math functions that causes a lot of load on the CPU. I am wanting to create a simple program the just creates load for X amount of seconds while another program monitors it. I'm just looking for functions, not actual stress testing programs.
The Computer Language Benchmark Game has quite a few benchmarks, many of which are math-based. It's a good source because the source code for each algorithm is included and there are implementations of each benchmark in dozens of languages. That way, you can just use the implementation in whatever language you're comfortable compiling and running.
Try the Lucas-Lehmer primality test. It's what's used in the Prime95 executable, and Prime95's fairly standard for CPU stress testing.
A naive implementation of Fibonacci? Something like:
let fib = Seq.unfold(fun (p, c) -> Some((p, c), (c, p+c))) (1,1)
I have 5 lines of code as function which is being called 100000 times is there a way to make the function inline so that I don't experienced a call overhead in PL/SQL. Oracle 9i is the version I am using.
Update:
The In lining of code does not have much improvement but I gained 2 seconds though. Now I am looking for efficient version of float data type in pl/sql. BINARY FLOAT did not work, NUMBER(10,5) is the one already being employed.
You've determined that the function call overhead is not contributing much to the performance issue, so inlining would not be helpful here anyway.
I assume you're not running any SQL or calling any SQL functions in your PL/SQL? If so I'd look there next.
Your next option is native compilation - see Compiling PL/SQL Code for Native Execution for details. You might get some improvement since you're only doing maths.
Otherwise, you may need to look beyond PL/SQL. You can call code written in many other languages from PL/SQL, including C, C++ and Java. Refer to Calling External Procedures for more info.