I have been reading a Elements of Programming Interview and am struggling to understand the passage below:
"The algorithm taught in grade-school for decimal multiplication does
not use repeated addition- it uses shift and add to achieve a much
better time complexity. We can do the same with binary numbers- to
multiply x and y we initialize the result to 0 and iterate through the
bits of x, adding (2^k)y to the result if the kth bit of x is 1.
The value (2^k)y can be computed by left-shifting y by k. Since we
cannot use add directly, we must implement it. We can apply the
grade-school algorithm for addition to the binary case, i.e, compute
the sum bit-by-bit and "rippling" the carry along.
As an example, we show how to multiply 13 = (1101) and 9 = (1001)
using the algorithm described above. In the first iteration, since
the LSB of 13 is 1, we set the result to (1001). The second bit of
(1101) is 0, so we move on the third bit. The bit is 1, so we shift
(1001) to the left by 2 to obtain (1001001), which we add to (1001) to
get (101101). The forth and final bit of (1101) is 1, so we shift
(1001) to the left by 3 to obtain (1001000), which we add to (101101)
to get (1110101) = 117.
My Questions are:
What is the overall idea behind this, how is it a "bit-by-bit" addition
where does (2^k)y come from
what does it mean by "left-shifting y by k"
In the example, why do we set result to (1001) just because the LSB of 13 is 1?
The algorithm relies on the way numbers are coded in binary.
Let A be an unsigned number. A is coded by a set of bits an-1an-2...a0 in such a way that A=∑i=0n-1ai×2i
Now, assume you have two numbers A and B coded in binary and you wand to compute A×B
B×A=B×∑i=0n-1ai×2i
=∑i=0n-1B×ai×2i
ai is equal to 0 or 1. If ai=0, the sum will not be modified. If ai=1, we need to add B×ai
So, we can simply deduce the multiplication algorithm
result=0
for i in 0 to n-1
if a[i]=1 // assumes a[i] is the ith bit
result = result + B * 2^i
end
end
What is the overall idea behind this, how is it a "bit-by-bit" addition
It is just an application of the previous method where you process successively every bit of the multiplicator
where does (2^k)y come from
As mentioned above from the way binary numbers are coded. If ith bit is set, then there is a 2i in the decomposition of the number.
what does it mean by "left-shifting y by k"
Left shift means "pushing" the bits leftwards and filling the "holes" with zeroes. Hence if number is 1101 and it is left shifted by three, it becomes 1101000.
This is the way to multiply the number by 2i (just as when "left shifting" by 2 a decimal number and putting zeroes at the right places is the way to multiply by 100=102)
In the example, why do we set result to (1001) just because the LSB of 13 is 1?
Because there is a 1 at right most position, that corresponds to 20. So we left shift by 0 and add it to the result that is initialized to 0.
As the title of the question says, I want to take a number (declared preferably as int or char or std::uint8_t), convert it into its binary representation, then truncate or pad it by a certain variable number of bits given, and then insert it into a bit container (preferably std::vector<bool> because I need variable bit container size as per the variable number of bits). For example, I have int a= 2, b = 3. And let's say I have to write this as three bits and six bits respectively into the container. So I have to put 010 and 000011 into the bit container. So, how would I go from 2 to 010 or 3 to 000011 using normal STL methods? I tried every possible thing that came to my mind, but I got nothing. Please help. Thank you.
You can use a combination of 'shifting' (>>) and 'bit-wise and' (&).
First lets look at the bitwise &: For instance if you have an int a=7 and you do the &-operation on it with 13, you will get 5. Why?
Because & gives 1 at position i iff both operands have a 1 at position i. So we get:
00...000111 // binary 7
& 00...001101 // binary 13
-------------
00...000101 // binary 5
Next, by using the shift operation >> you can shift the binary representation of your ints. For instance 5 >> 1 is 2. Why?
Because each position gets displaced by 1 to the right. The rightmost bit "falls out". Hence we have:
00...00101 //binary for 5
shift by 1 to the right gives:
00...00010 // binary for 2
Another example: 13 (01101) shifted by 2 is 3 (00011). I hope you get the idea.
Hence, by repeatedly shifting and doing & with 1 (00..0001), you can read out the binary representation of a number.
Finally, you can use this 1 to set the corresponding position in your vector<bool>. Assuming you want to have the representation you show in your post, you will have to fill in your vector from the back. So, you could for instance do something along the lines:
unsigned int f=13; //the number we want to convert
std::vector<bool> binRepr(size, false); //size is the container-size you want to use.
for(int currBit=0; currBit<size; currBit++){
binRepr[size-1-currBit] = (f >> currBit) & 1;
}
If the container is smaller than the binary representation of your int, the container will contain the truncated number. If it is larger, it will fill in with 0s.
I'm using an unsigned int since for an int you would still have to take care of negative numbers (for positive numbers it should work the same) and we would have to dive into the two's complement representation, which is not difficult, but requires a bit more bit-fiddling.
Say I have a 4 bit ALU, I have a carry flag, overflow flag, and a sign flag(MSB). How would I go about subtracting for example, two signed 8 bit numbers? I take the lower nibble of both numbers and subtract them right, but I don't understand how to know if there needs to be a 5th bit, and carry that over to the LSB of the high nibble of the number, and if so, how to add it considering I am doing this in 2's complement so I already have Carryin being used.. Any help would be appreciated.
This has been asked and answered here many times.
You know the rule about twos complement yes? Invert and add one. Also from grade school
a + b = a + (-b).
We dont have subtract hardware we have add hardware. What you do is a + (-b). Also from grade school we learned about carrying, 9+3 = 2 carry the 1. And from the second column on we have either two or three operands that are added together (a + (-b) + c, c being the carry in). If you think about it we can have a carry in on every column, sometimes it is zero. That is how the hardware works, each column is three in two out, carry in[n], a[n], b[n] the output is result[n] and carry out[n]. and as we know from grade school the carry out of this column is the carry in of the next column. So for a normal add the carry in of the least significant bit is always a zero, but for subtract we want to invert and add one so what we do is invert b and change the carry in of that first bit to a 1 which is the same as
a + (~b) + 1 which equals a + (-b) which equals a - b.
As far as addition and subtract hardware is concerned there is no such thing as signed or unsigned add or subtract. There does exist an unsigned overflow (carry out of the msbit) and a signed overflow (true if carry in and carry out of the msbit are not the same, false if they match).
This works for any number of bits, for example if you have 8 bit hardware but want to do math on 256 bit numbers, just do them 8 bits at a time and apply the carry out to the next 8 bits (add with carry or subtract with borrow instruction). Visualize the single columns one at a time, 4 bits is just four of those columns, 8, 9 bits 37 bits, etc. You can easily take any of those larger numbers draw a vertical line anywhere separating it into two operations all you have to do is what you do for single columns the carry out of the msbit of the thing on the right becomes the carry in of the lsbit of the thing on the left of the dividing line. Apply this to 8 bit math with 4 bit hardware...
So a subtract is an add with a carry in of 1 and the second operand inverted. Now some hardware inverts the carry out (unsigned overflow) on a subtract so that it becomes 1 for borrow and 0 for not borrow (unsigned borrow/overflow). Some dont. So you have to know how this works if you dont have a subtract with borrow instruction. If you have a subtract with borrow it doesnt matter if they invert carry out they will generically invert carry in (on a subtract). If they dont then again they wont on a subtract with borrow. but if you have to use an add with carry to simulate a subtract with borrow you need to possibly not only invert the second operand but invert the carry bit. If you dont have an add with carry then you have to simulate that as well by simply adding 1 or not.
I've been attempting to learn C in my spare time, and other languages (C#, Java, etc.) have the same concept (and often the same operators)...
At a core level, what does bit-shifting (<<, >>, >>>) do, what problems can it help solve, and what gotchas lurk around the bend? In other words, an absolute beginner's guide to bit shifting in all its goodness.
The bit shifting operators do exactly what their name implies. They shift bits. Here's a brief (or not-so-brief) introduction to the different shift operators.
The Operators
>> is the arithmetic (or signed) right shift operator.
>>> is the logical (or unsigned) right shift operator.
<< is the left shift operator, and meets the needs of both logical and arithmetic shifts.
All of these operators can be applied to integer values (int, long, possibly short and byte or char). In some languages, applying the shift operators to any datatype smaller than int automatically resizes the operand to be an int.
Note that <<< is not an operator, because it would be redundant.
Also note that C and C++ do not distinguish between the right shift operators. They provide only the >> operator, and the right-shifting behavior is implementation defined for signed types. The rest of the answer uses the C# / Java operators.
(In all mainstream C and C++ implementations including GCC and Clang/LLVM, >> on signed types is arithmetic. Some code assumes this, but it isn't something the standard guarantees. It's not undefined, though; the standard requires implementations to define it one way or another. However, left shifts of negative signed numbers is undefined behaviour (signed integer overflow). So unless you need arithmetic right shift, it's usually a good idea to do your bit-shifting with unsigned types.)
Left shift (<<)
Integers are stored, in memory, as a series of bits. For example, the number 6 stored as a 32-bit int would be:
00000000 00000000 00000000 00000110
Shifting this bit pattern to the left one position (6 << 1) would result in the number 12:
00000000 00000000 00000000 00001100
As you can see, the digits have shifted to the left by one position, and the last digit on the right is filled with a zero. You might also note that shifting left is equivalent to multiplication by powers of 2. So 6 << 1 is equivalent to 6 * 2, and 6 << 3 is equivalent to 6 * 8. A good optimizing compiler will replace multiplications with shifts when possible.
Non-circular shifting
Please note that these are not circular shifts. Shifting this value to the left by one position (3,758,096,384 << 1):
11100000 00000000 00000000 00000000
results in 3,221,225,472:
11000000 00000000 00000000 00000000
The digit that gets shifted "off the end" is lost. It does not wrap around.
Logical right shift (>>>)
A logical right shift is the converse to the left shift. Rather than moving bits to the left, they simply move to the right. For example, shifting the number 12:
00000000 00000000 00000000 00001100
to the right by one position (12 >>> 1) will get back our original 6:
00000000 00000000 00000000 00000110
So we see that shifting to the right is equivalent to division by powers of 2.
Lost bits are gone
However, a shift cannot reclaim "lost" bits. For example, if we shift this pattern:
00111000 00000000 00000000 00000110
to the left 4 positions (939,524,102 << 4), we get 2,147,483,744:
10000000 00000000 00000000 01100000
and then shifting back ((939,524,102 << 4) >>> 4) we get 134,217,734:
00001000 00000000 00000000 00000110
We cannot get back our original value once we have lost bits.
Arithmetic right shift (>>)
The arithmetic right shift is exactly like the logical right shift, except instead of padding with zero, it pads with the most significant bit. This is because the most significant bit is the sign bit, or the bit that distinguishes positive and negative numbers. By padding with the most significant bit, the arithmetic right shift is sign-preserving.
For example, if we interpret this bit pattern as a negative number:
10000000 00000000 00000000 01100000
we have the number -2,147,483,552. Shifting this to the right 4 positions with the arithmetic shift (-2,147,483,552 >> 4) would give us:
11111000 00000000 00000000 00000110
or the number -134,217,722.
So we see that we have preserved the sign of our negative numbers by using the arithmetic right shift, rather than the logical right shift. And once again, we see that we are performing division by powers of 2.
Let's say we have a single byte:
0110110
Applying a single left bitshift gets us:
1101100
The leftmost zero was shifted out of the byte, and a new zero was appended to the right end of the byte.
The bits don't rollover; they are discarded. That means if you left shift 1101100 and then right shift it, you won't get the same result back.
Shifting left by N is equivalent to multiplying by 2N.
Shifting right by N is (if you are using ones' complement) is the equivalent of dividing by 2N and rounding to zero.
Bitshifting can be used for insanely fast multiplication and division, provided you are working with a power of 2. Almost all low-level graphics routines use bitshifting.
For example, way back in the olden days, we used mode 13h (320x200 256 colors) for games. In Mode 13h, the video memory was laid out sequentially per pixel. That meant to calculate the location for a pixel, you would use the following math:
memoryOffset = (row * 320) + column
Now, back in that day and age, speed was critical, so we would use bitshifts to do this operation.
However, 320 is not a power of two, so to get around this we have to find out what is a power of two that added together makes 320:
(row * 320) = (row * 256) + (row * 64)
Now we can convert that into left shifts:
(row * 320) = (row << 8) + (row << 6)
For a final result of:
memoryOffset = ((row << 8) + (row << 6)) + column
Now we get the same offset as before, except instead of an expensive multiplication operation, we use the two bitshifts...in x86 it would be something like this (note, it's been forever since I've done assembly (editor's note: corrected a couple mistakes and added a 32-bit example)):
mov ax, 320; 2 cycles
mul word [row]; 22 CPU Cycles
mov di,ax; 2 cycles
add di, [column]; 2 cycles
; di = [row]*320 + [column]
; 16-bit addressing mode limitations:
; [di] is a valid addressing mode, but [ax] isn't, otherwise we could skip the last mov
Total: 28 cycles on whatever ancient CPU had these timings.
Vrs
mov ax, [row]; 2 cycles
mov di, ax; 2
shl ax, 6; 2
shl di, 8; 2
add di, ax; 2 (320 = 256+64)
add di, [column]; 2
; di = [row]*(256+64) + [column]
12 cycles on the same ancient CPU.
Yes, we would work this hard to shave off 16 CPU cycles.
In 32 or 64-bit mode, both versions get a lot shorter and faster. Modern out-of-order execution CPUs like Intel Skylake (see http://agner.org/optimize/) have very fast hardware multiply (low latency and high throughput), so the gain is much smaller. AMD Bulldozer-family is a bit slower, especially for 64-bit multiply. On Intel CPUs, and AMD Ryzen, two shifts are slightly lower latency but more instructions than a multiply (which may lead to lower throughput):
imul edi, [row], 320 ; 3 cycle latency from [row] being ready
add edi, [column] ; 1 cycle latency (from [column] and edi being ready).
; edi = [row]*(256+64) + [column], in 4 cycles from [row] being ready.
vs.
mov edi, [row]
shl edi, 6 ; row*64. 1 cycle latency
lea edi, [edi + edi*4] ; row*(64 + 64*4). 1 cycle latency
add edi, [column] ; 1 cycle latency from edi and [column] both being ready
; edi = [row]*(256+64) + [column], in 3 cycles from [row] being ready.
Compilers will do this for you: See how GCC, Clang, and Microsoft Visual C++ all use shift+lea when optimizing return 320*row + col;.
The most interesting thing to note here is that x86 has a shift-and-add instruction (LEA) that can do small left shifts and add at the same time, with the performance as an add instruction. ARM is even more powerful: one operand of any instruction can be left or right shifted for free. So scaling by a compile-time-constant that's known to be a power-of-2 can be even more efficient than a multiply.
OK, back in the modern days... something more useful now would be to use bitshifting to store two 8-bit values in a 16-bit integer. For example, in C#:
// Byte1: 11110000
// Byte2: 00001111
Int16 value = ((byte)(Byte1 >> 8) | Byte2));
// value = 000011111110000;
In C++, compilers should do this for you if you used a struct with two 8-bit members, but in practice they don't always.
Bitwise operations, including bit shift, are fundamental to low-level hardware or embedded programming. If you read a specification for a device or even some binary file formats, you will see bytes, words, and dwords, broken up into non-byte aligned bitfields, which contain various values of interest. Accessing these bit-fields for reading/writing is the most common usage.
A simple real example in graphics programming is that a 16-bit pixel is represented as follows:
bit | 15| 14| 13| 12| 11| 10| 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
| Blue | Green | Red |
To get at the green value you would do this:
#define GREEN_MASK 0x7E0
#define GREEN_OFFSET 5
// Read green
uint16_t green = (pixel & GREEN_MASK) >> GREEN_OFFSET;
Explanation
In order to obtain the value of green ONLY, which starts at offset 5 and ends at 10 (i.e. 6-bits long), you need to use a (bit) mask, which when applied against the entire 16-bit pixel, will yield only the bits we are interested in.
#define GREEN_MASK 0x7E0
The appropriate mask is 0x7E0 which in binary is 0000011111100000 (which is 2016 in decimal).
uint16_t green = (pixel & GREEN_MASK) ...;
To apply a mask, you use the AND operator (&).
uint16_t green = (pixel & GREEN_MASK) >> GREEN_OFFSET;
After applying the mask, you'll end up with a 16-bit number which is really just a 11-bit number since its MSB is in the 11th bit. Green is actually only 6-bits long, so we need to scale it down using a right shift (11 - 6 = 5), hence the use of 5 as offset (#define GREEN_OFFSET 5).
Also common is using bit shifts for fast multiplication and division by powers of 2:
i <<= x; // i *= 2^x;
i >>= y; // i /= 2^y;
Bit Masking & Shifting
Bit shifting is often used in low-level graphics programming. For example, a given pixel color value encoded in a 32-bit word.
Pixel-Color Value in Hex: B9B9B900
Pixel-Color Value in Binary: 10111001 10111001 10111001 00000000
For better understanding, the same binary value labeled with what sections represent what color part.
Red Green Blue Alpha
Pixel-Color Value in Binary: 10111001 10111001 10111001 00000000
Let's say for example we want to get the green value of this pixel's color. We can easily get that value by masking and shifting.
Our mask:
Red Green Blue Alpha
color : 10111001 10111001 10111001 00000000
green_mask : 00000000 11111111 00000000 00000000
masked_color = color & green_mask
masked_color: 00000000 10111001 00000000 00000000
The logical & operator ensures that only the values where the mask is 1 are kept. The last thing we now have to do, is to get the correct integer value by shifting all those bits to the right by 16 places (logical right shift).
green_value = masked_color >>> 16
Et voilà, we have the integer representing the amount of green in the pixel's color:
Pixels-Green Value in Hex: 000000B9
Pixels-Green Value in Binary: 00000000 00000000 00000000 10111001
Pixels-Green Value in Decimal: 185
This is often used for encoding or decoding image formats like jpg, png, etc.
One gotcha is that the following is implementation dependent (according to the ANSI standard):
char x = -1;
x >> 1;
x can now be 127 (01111111) or still -1 (11111111).
In practice, it's usually the latter.
I am writing tips and tricks only. It may be useful in tests and exams.
n = n*2: n = n<<1
n = n/2: n = n>>1
Checking if n is power of 2 (1,2,4,8,...): check !(n & (n-1))
Getting xth bit of n: n |= (1 << x)
Checking if x is even or odd: x&1 == 0 (even)
Toggle the nth bit of x: x ^ (1<<n)
Note that in the Java implementation, the number of bits to shift is mod'd by the size of the source.
For example:
(long) 4 >> 65
equals 2. You might expect shifting the bits to the right 65 times would zero everything out, but it's actually the equivalent of:
(long) 4 >> (65 % 64)
This is true for <<, >>, and >>>. I have not tried it out in other languages.
Some useful bit operations/manipulations in Python.
I implemented Ravi Prakash's answer in Python.
# Basic bit operations
# Integer to binary
print(bin(10))
# Binary to integer
print(int('1010', 2))
# Multiplying x with 2 .... x**2 == x << 1
print(200 << 1)
# Dividing x with 2 .... x/2 == x >> 1
print(200 >> 1)
# Modulo x with 2 .... x % 2 == x & 1
if 20 & 1 == 0:
print("20 is a even number")
# Check if n is power of 2: check !(n & (n-1))
print(not(33 & (33-1)))
# Getting xth bit of n: (n >> x) & 1
print((10 >> 2) & 1) # Bin of 10 == 1010 and second bit is 0
# Toggle nth bit of x : x^(1 << n)
# take bin(10) == 1010 and toggling second bit in bin(10) we get 1110 === bin(14)
print(10^(1 << 2))
The Bitwise operators are used to perform operations a bit-level or to manipulate bits in different ways. The bitwise operations are found to be much faster and are some times used to improve the efficiency of a program.
Basically, Bitwise operators can be applied to the integer types: long, int, short, char and byte.
Bitwise Shift Operators
They are classified into two categories left shift and the right shift.
Left Shift(<<): The left shift operator, shifts all of the bits in value to the left a specified number of times. Syntax: value << num. Here num specifies the number of position to left-shift the value in value. That is, the << moves all of the bits in the specified value to the left by the number of bit positions specified by num. For each shift left, the high-order bit is shifted out (and ignored/lost), and a zero is brought in on the right. This means that when a left shift is applied to 32-bit compiler, bits are lost once they are shifted past bit position 31. If the compiler is of 64-bit then bits are lost after bit position 63.
Output: 6, Here the binary representation of 3 is 0...0011(considering 32-bit system) so when it shifted one time the leading zero is ignored/lost and all the rest 31 bits shifted to left. And zero is added at the end. So it became 0...0110, the decimal representation of this number is 6.
In the case of a negative number:
Output: -2, In java negative number, is represented by 2's complement. SO, -1 represent by 2^32-1 which is equivalent to 1....11(Considering 32-bit system). When shifted one time the leading bit is ignored/lost and the rest 31 bits shifted to left and zero is added at the last. So it becomes, 11...10 and its decimal equivalent is -2.
So, I think you get enough knowledge about the left shift and how its work.
Right Shift(>>): The right shift operator, shifts all of the bits in value to the right a specified of times. Syntax: value >> num, num specifies the number of positions to right-shift the value in value. That is, the >> moves/shift all of the bits in the specified value of the right the number of bit positions specified by num.
The following code fragment shifts the value 35 to the right by two positions:
Output: 8, As a binary representation of 35 in a 32-bit system is 00...00100011, so when we right shift it two times the first 30 leading bits are moved/shifts to the right side and the two low-order bits are lost/ignored and two zeros are added at the leading bits. So, it becomes 00....00001000, the decimal equivalent of this binary representation is 8.
Or there is a simple mathematical trick to find out the output of this following code: To generalize this we can say that, x >> y = floor(x/pow(2,y)). Consider the above example, x=35 and y=2 so, 35/2^2 = 8.75 and if we take the floor value then the answer is 8.
Output:
But remember one thing this trick is fine for small values of y if you take the large values of y it gives you incorrect output.
In the case of a negative number:
Because of the negative numbers the Right shift operator works in two modes signed and unsigned. In signed right shift operator (>>), In case of a positive number, it fills the leading bits with 0. And In case of a negative number, it fills leading bits with 1. To keep the sign. This is called 'sign extension'.
Output: -5, As I explained above the compiler stores the negative value as 2's complement. So, -10 is represented as 2^32-10 and in binary representation considering 32-bit system 11....0110. When we shift/ move one time the first 31 leading bits got shifted in the right side and the low-order bit got lost/ignored. So, it becomes 11...0011 and the decimal representation of this number is -5 (How I know the sign of number? because the leading bit is 1).
It is interesting to note that if you shift -1 right, the result always remains -1 since sign extension keeps bringing in more ones in the high-order bits.
Unsigned Right Shift(>>>): This operator also shifts bits to the right. The difference between signed and unsigned is the latter fills the leading bits with 1 if the number is negative and the former fills zero in either case. Now the question arises why we need unsigned right operation if we get the desired output by signed right shift operator. Understand this with an example, If you are shifting something that does not represent a numeric value, you may not want sign extension to take place. This situation is common when you are working with pixel-based values and graphics. In these cases, you will generally want to shift a zero into the high-order bit no matter what it's the initial value was.
Output: 2147483647, Because -2 is represented as 11...10 in a 32-bit system. When we shift the bit by one, the first 31 leading bit is moved/shifts in right and the low-order bit is lost/ignored and the zero is added to the leading bit. So, it becomes 011...1111 (2^31-1) and its decimal equivalent is 2147483647.
Be aware of that only 32 bit version of PHP is available on the Windows platform.
Then if you for instance shift << or >> more than by 31 bits, results are unexpectable. Usually the original number instead of zeros will be returned, and it can be a really tricky bug.
Of course if you use 64 bit version of PHP (Unix), you should avoid shifting by more than 63 bits. However, for instance, MySQL uses the 64-bit BIGINT, so there should not be any compatibility problems.
UPDATE: From PHP 7 Windows, PHP builds are finally able to use full 64 bit integers:
The size of an integer is platform-dependent, although a maximum value of about two billion is the usual value (that's 32 bits signed). 64-bit platforms usually have a maximum value of about 9E18, except on Windows prior to PHP 7, where it was always 32 bit.