I notice there is a macro uint4korr in the MySQL/MariaDB source code.
include/byte_order_generic.h
I merely understand this macro is correlated with byte order. But I looked for the comments about this macro, found nothing. I don't know the meaning of the suffix korr. What does the abbreviation express?
I want to know why the code implements like this? What are the effects on different platforms?
"korr" is an abbreviation for "Korrekt" of the phonic and meaning equivalent of the English word "Correct".
The purpose of the code is to provide a uniform byte order of storage and communication components so the storage files are portable between different endian architectures without conversion, and the client/server communication doesn't need to know which endian the other architecture is.
I believe that the related Swedish verb is korrigera, to correct. uint4korr() is kind of the opposite of ntohl(), because it will swap the bytes on a big-endian architecture and not little-endian.
Somewhat related to this, the InnoDB storage engine stores its data in big-endian byte order, so that a simple memcmp() can be used for comparing keys. (It also inverts the sign bit of signed integers due to this.) The InnoDB function mach_read_from_4() is basically ntohl() combined with a 32-bit load via an unaligned pointer. Recent versions of GCC and clang impress me by translating that into the IA-32 or AMD64 instructions mov and bswap or simply movbe.
Related
I am modeling algorithm to hardware mapping with Gecode and standard Gecode::Int::Limits is too small at least because I want to target systems with more than 2^32 memory.
Is there a way to get use of arbitrary-precision arithmetic with Gecode or at least 64-bits integers?
I know that Gecode can be built with MPIR or GMP support, but seems those are just for trigonometric operations?
If I understand Gecode documentation properly:
The totally available number of bits for all variable implementation types used by Gecode is 32
So seems there is no way to model with values bigger than 2147483646, but I still think I'm fundamentally wrong about something, since it's almost obligatory for modeling toolkit/library to have an ability to model with values bigger than that. Especially, Wikipedia says that:
ECLiPSe interfaces to external solvers, in particular ... and the Gecode solver library
but ECLiPSe tutorial stands that
Numbers in ECLiPSe come in several flavors:
Integers can be as large as fits into memory, e.g.:
123 0 -27 393423874981724
I cannot understand how just an interface being able to have numbers bigger than underlying library.
We write OpenCL C code and clCreateProgramWithSource and use clGetProgramInfo to get the binary. This binary is then integrated to the product binary which uses clCreateProgramWithBinary when initializing it.
We create a .h file and include the same in the source file. The content of the .h file is the binary generated after compiling OpenCL C Kernel.
The issue with the above step is, the compatibility of the binary is expected to break with any minor/major change in OpenCL and it will most likely break across vendors. We need to generate the OpenCL Kernel binary for each vendor or OpenCL release.
It is possible to integrate the OpenCL Kernel binary in header form to the project. In this case, if the binary is incompatible, we will not be in position to replace the binary. In such cases, the project initialization fails.
Expected Solution
The OpenCL C source is proprietary to the company and cannot be shared with the customers.
Since the OpenCL Kernel binary is integrated with the project
library, we need to understand if it is possible to generate binary
which can re-organize itself while clCreateProgramWithBinary to fit
to the target platform.
If it is absolutely necessary to generate the binary once for each
vendor/OpenCL minor/major revision and store it to disk (which will
be done at end user’s machine), how can we protect the source which
proprietary to the company (is SPIR the only option)?
I already visited Universal binaries for OpenCL but it suggests that SPIR also takes long time in compilation and hence it might not be the solution I am looking for since the init time is also important.
In practice the Intel Gen binary format can change on driver changes for the same platform/hardware (e.g. for bug fix workarounds and performance improvements). Hence, the bits returned by clGetProgramInfo are only sure to work in clCreateProgramWithBinary on the same device x driver x etc... Sadly, this means that the binary path is a poor match for the intellectual property security problem.
SPIR sort of splits the difference as it would be hardware independent while still being harder to reverse engineer. If startup performance is somehow important, you can always try the clCreateProgramWithBinary path; just be able to fall back to SPIR should the binary load fail (meaning the driver changed or something).
Recently, I have been making program (FDTD Operation) using the CUDA
development environment, OS is Windows server 2008 , Graphic card is TeslaC2070, compiler is VS2010. This program calculates using single and double precision floating-point.
I was reading the CUDA programming guide 3.2 and 4.0 . In appendix, guide tell me sin(), cos() has maximum accuracy of 2 ULP. My original CPU program produces results which are different to the CUDA Version.
I want to make results correctly same. Is it possible?
To quote Goldberg (a paper that every Computer Scientist, Computational Scientist, and possibly even every scientist who programs, should read):
Due to roundoff errors, the associative laws of algebra do not
necessarily hold for floating-point numbers.
This means that when you change the order of operations—even when using ostensibly associative arithmetic—you are likely to get slightly different answers.
Parallelism, by definition, results in different ordering of operations relative to serial arithmetic. "Embarrasingly parallel" computations, that is, computations where each output element is computed independently from all others, sometimes do not have to worry about this. But collective operations, like reductions or scans, and spatial neighborhood computations, such stencils (as in FDTD), do experience this effect.
In practice, even using a different compiler (and even different compiler options) can change the result of floating point computation, even when compiling the same code, with or without parallelism.
This is what wikipedia says:
In computer software, an application
binary interface (ABI) describes the
low-level interface between an
application (or any type of) program
and the operating system or another
application.
ABIs cover details such as data type,
size, and alignment; the calling
convention, which controls how
functions' arguments are passed and
return values retrieved; the system
call numbers and how an application
should make system calls to the
operating system; and in the case of a
complete operating system ABI, the
binary format of object files, program
libraries and so on. A complete ABI,
such as the Intel Binary Compatibility
Standard (iBCS), allows a program
from one operating system supporting
that ABI to run without modifications
on any other such system, provided
that necessary shared libraries are
present, and similar prerequisites are
fulfilled.
I guess that an ABI is a convention or standard, and compilers/linkers use this convention to produce object codes. Is that right? If so who made these conventions(companies or some organization)? What was it like when there was no ABIs? Is there documents about these ABIs that we can refer to?
You're correct about the definition of an ABI, up to a point. The classic example is the syscall interface in Linux (and other UNIXes).
They are a standard way for code to request the operating system to carry out certain duties.
As such, they're decided by the people that wrote the OS or, in the case where the syscalls have been added later, by whoever added them (in cases where the OS allows this). For example, the Linux syscall interface on x86 states that you load the syscall number into eax, with other parameters placed in ebx, ecx and so on, depending on the syscall you're making (eax).
Typically, it's not the compiler or linker which do the work of interfacing, rather it's the libraries provided for the language you're using.
Returning to Linux, the GNU C libraries contain code for fopen (for example) which eventually call the relevant syscall to perform the lower level tasks (syscall number 5, open). A list of the syscalls can be found in this PDF file.
Specification is more suitable term than convention, as convention is loose term for widely accepted practice whereas specification is well-defined.
You are right. The specification is made by standardization body. Take a look at POSIX specification which is supported by Windows and compiler/build tool-chains such as gcc assume OS's to adhere by it, and even Linux kernel partially (almost exactly) adheres to it.
Before ABIs? Even today, firmware is hand-crafted as new chips come along for set-top boxes and such other devices having embedded systems.
The documentation is digital logic content in the data-sheet for the chips to be programmed by assembly language and for higher-level language, the cross-compiler tool-chain documentation gives away the assumptions that should be part of ABI.
Well, the concept of ABI was presumably conceived to support the binary compatibility of your program on other operating systems and machine architectures. So, lets suppose that you wrote a program on some operating system distribution running on x86 architecture. Now, for a programmer the most important thing is that this program that you wrote on your machine should be able to run exactly the same on any other machine running on same or different architecture lets say for the sake of discussion that the other machine is running on i386 architecture and this is where the concept of ABI or Application Binary Interfaces comes in. As every machine architecture defines its own way in which the operating system kernal talks to the outside world i.e user-space programs, hence every architecture defines a different set of system calls, machine registers, how those registers are used, how are software interrupts handled by the kernal and so on. ABI is the thing that handles these things for you like compiling, linking, byte ordering and so on. System programmers have had hard luck defining a uniform ABI for same operating systems running on different architectures and that is why every machine architecture has its own and you need to compile your programs in order to confirm to the format those machines have.
What considerations do I need to make if I want my code to run correctly on both 32bit and 64bit platforms ?
EDIT: What kind of areas do I need to take care in, e.g. printing strings/characters or using structures ?
Options:
Code it in some language with a Virtual Machine (such as Java)
Code it in .NET and don't target any specific architecture. The .NET JIT compiler will compile it for you to the right architecture before running it.
One solution would be to target a virtual environment that runs on both platforms (I'm thinking Java, or .Net here).
Or pick an interpreted language.
Do you have other requirements, such as calling existing code or libraries?
The same things you should have been doing all along to ensure you write portable code :)
mozilla guidelines and the C faq are good starting points
I assume you are still talking about compiling them separately for each individual platform? As running them on both is completely doable by just creating a 32bit binary.
The biggest one is making sure you don't put pointers into 32-bit storage locations.
But there's no proper 'language-agnostic' answer to this question, really. You couldn't even get a particularly firm answer if you restricted yourself to something like standard 'C' or 'C++' - the size of data storage, pointers, etc, is all terribly implementation dependant.
It honestly depends on the language, because managed languages like C# and Java or Scripting languages like JavaScript, Python, or PHP are locked in to their current methodology and to get started and to do anything beyond the advanced stuff there is not much to worry about.
But my guess is that you are asking about languages like C++, C, and other lower level languages.
The biggest thing you have to worry about is the size of things, because in the 32-bit world you are limited to the power of 2^32 however in the 64-bit world things get bigger 2^64.
With 64-bit you have a larger space for memory and storage in RAM, and you can compute larger numbers. However if you know you are compiling for both 32 and 64, you need to make sure to limit your expectations of the system to the 32-bit world and limitations of buffers and numbers.
In C (and maybe C++) always remember to use the sizeof operator when calculating buffer sizes for malloc. This way you will write more portable code anyway, and this will automatically take 64bit datatypes into account.
In most cases the only thing you have to do is just compile your code for both platforms. (And that's assuming that you're using a compiled language; if it's not, then you probably don't need to worry about anything.)
The only thing I can think of that might cause problems is assuming the size of data types, which is something you probably shouldn't be doing anyway. And of course anything written in assembly is going to cause problems.
Keep in mind that many compilers choose the size of integer based on the underlying architecture, given that the "int" should be the fastest number manipulator in the system (according to some theories).
This is why so many programmers use typedefs for their most portable programs - if you want your code to work on everything from 8 bit processors up to 64 bit processors you need to recognize that, in C anyway, int is not rigidly defined.
Pointers are another area to be careful - don't use a long, or long long, or any specific type if you are fiddling with the numeric value of the pointer - use the proper construct, which, unfortunately, varies from compiler to compiler (which is why you have a separate typedef.h file for each compiler you use).
-Adam Davis