Good afternoon!
I am trying to develop an NTT algorithm based on the naive recursive FFT implementation I already have.
Consider the following code (coefficients' length, let it be m, is an exact power of two):
/// <summary>
/// Calculates the result of the recursive Number Theoretic Transform.
/// </summary>
/// <param name="coefficients"></param>
/// <returns></returns>
private static BigInteger[] Recursive_NTT_Skeleton(
IList<BigInteger> coefficients,
IList<BigInteger> rootsOfUnity,
int step,
int offset)
{
// Calculate the length of vectors at the current step of recursion.
// -
int n = coefficients.Count / step - offset / step;
if (n == 1)
{
return new BigInteger[] { coefficients[offset] };
}
BigInteger[] results = new BigInteger[n];
IList<BigInteger> resultEvens =
Recursive_NTT_Skeleton(coefficients, rootsOfUnity, step * 2, offset);
IList<BigInteger> resultOdds =
Recursive_NTT_Skeleton(coefficients, rootsOfUnity, step * 2, offset + step);
for (int k = 0; k < n / 2; k++)
{
BigInteger bfly = (rootsOfUnity[k * step] * resultOdds[k]) % NTT_MODULUS;
results[k] = (resultEvens[k] + bfly) % NTT_MODULUS;
results[k + n / 2] = (resultEvens[k] - bfly) % NTT_MODULUS;
}
return results;
}
It worked for complex FFT (replace BigInteger with a complex numeric type (I had my own)). It doesn't work here even though I changed the procedure of finding the primitive roots of unity appropriately.
Supposedly, the problem is this: rootsOfUnity parameter passed originally contained only the first half of m-th complex roots of unity in this order:
omega^0 = 1, omega^1, omega^2, ..., omega^(n/2)
It was enough, because on these three lines of code:
BigInteger bfly = (rootsOfUnity[k * step] * resultOdds[k]) % NTT_MODULUS;
results[k] = (resultEvens[k] + bfly) % NTT_MODULUS;
results[k + n / 2] = (resultEvens[k] - bfly) % NTT_MODULUS;
I originally made use of the fact, that at any level of recursion (for any n and i), the complex root of unity -omega^(i) = omega^(i + n/2).
However, that property obviously doesn't hold in finite fields. But is there any analogue of it which would allow me to still compute only the first half of the roots?
Or should I extend the cycle from n/2 to n and pre-compute all the m-th roots of unity?
Maybe there are other problems with this code?..
Thank you very much in advance!
I recently wanted to implement NTT for fast multiplication instead of DFFT too. Read a lot of confusing things, different letters everywhere and no simple solution, and also my finite fields knowledge is rusty , but today i finally got it right (after 2 days of trying and analog-ing with DFT coefficients) so here are my insights for NTT:
Computation
X(i) = sum(j=0..n-1) of ( Wn^(i*j)*x(i) );
where X[] is NTT transformed x[] of size n where Wn is the NTT basis. All computations are on integer modulo arithmetics mod p no complex numbers anywhere.
Important values
Wn = r ^ L mod p is basis for NTT
Wn = r ^ (p-1-L) mod p is basis for INTT
Rn = n ^ (p-2) mod p is scaling multiplicative constant for INTT ~(1/n)
p is prime that p mod n == 1 and p>max'
max is max value of x[i] for NTT or X[i] for INTT
r = <1,p)
L = <1,p) and also divides p-1
r,L must be combined so r^(L*i) mod p == 1 if i=0 or i=n
r,L must be combined so r^(L*i) mod p != 1 if 0 < i < n
max' is the sub-result max value and depends on n and type of computation. For single (I)NTT it is max' = n*max but for convolution of two n sized vectors it is max' = n*max*max etc. See Implementing FFT over finite fields for more info about it.
functional combination of r,L,p is different for different n
this is important, you have to recompute or select parameters from table before each NTT layer (n is always half of the previous recursion).
Here is my C++ code that finds the r,L,p parameters (needs modular arithmetics which is not included, you can replace it with (a+b)%c,(a-b)%c,(a*b)%c,... but in that case beware of overflows especial for modpow and modmul) The code is not optimized yet there are ways to speed it up considerably. Also prime table is fairly limited so either use SoE or any other algo to obtain primes up to max' in order to work safely.
DWORD _arithmetics_primes[]=
{
2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61,67,71,73,79,83,89,97,101,103,107,109,113,127,131,137,139,149,151,157,163,167,173,
179,181,191,193,197,199,211,223,227,229,233,239,241,251,257,263,269,271,277,281,283,293,307,311,313,317,331,337,347,349,353,359,367,373,379,383,389,397,401,409,
419,421,431,433,439,443,449,457,461,463,467,479,487,491,499,503,509,521,523,541,547,557,563,569,571,577,587,593,599,601,607,613,617,619,631,641,643,647,653,659,
661,673,677,683,691,701,709,719,727,733,739,743,751,757,761,769,773,787,797,809,811,821,823,827,829,839,853,857,859,863,877,881,883,887,907,911,919,929,937,941,
947,953,967,971,977,983,991,997,1009,1013,1019,1021,1031,1033,1039,1049,1051,1061,1063,1069,1087,1091,1093,1097,1103,1109,1117,1123,1129,1151,
0}; // end of table is 0, the more primes are there the bigger numbers and n can be used
// compute NTT consts W=r^L%p for n
int i,j,k,n=16;
long w,W,iW,p,r,L,l,e;
long max=81*n; // edit1: max num for NTT for my multiplication purposses
for (e=1,j=0;e;j++) // find prime p that p%n=1 AND p>max ... 9*9=81
{
p=_arithmetics_primes[j];
if (!p) break;
if ((p>max)&&(p%n==1))
for (r=2;r<p;r++) // check all r
{
for (l=1;l<p;l++)// all l that divide p-1
{
L=(p-1);
if (L%l!=0) continue;
L/=l;
W=modpow(r,L,p);
e=0;
for (w=1,i=0;i<=n;i++,w=modmul(w,W,p))
{
if ((i==0) &&(w!=1)) { e=1; break; }
if ((i==n) &&(w!=1)) { e=1; break; }
if ((i>0)&&(i<n)&&(w==1)) { e=1; break; }
}
if (!e) break;
}
if (!e) break;
}
}
if (e) { error; } // error no combination r,l,p for n found
W=modpow(r, L,p); // Wn for NTT
iW=modpow(r,p-1-L,p); // Wn for INTT
and here is my slow NTT and INTT implementations (i havent got to fast NTT,INTT yet) they are both tested with Schönhage–Strassen multiplication successfully.
//---------------------------------------------------------------------------
void NTT(long *dst,long *src,long n,long m,long w)
{
long i,j,wj,wi,a,n2=n>>1;
for (wj=1,j=0;j<n;j++)
{
a=0;
for (wi=1,i=0;i<n;i++)
{
a=modadd(a,modmul(wi,src[i],m),m);
wi=modmul(wi,wj,m);
}
dst[j]=a;
wj=modmul(wj,w,m);
}
}
//---------------------------------------------------------------------------
void INTT(long *dst,long *src,long n,long m,long w)
{
long i,j,wi=1,wj=1,rN,a,n2=n>>1;
rN=modpow(n,m-2,m);
for (wj=1,j=0;j<n;j++)
{
a=0;
for (wi=1,i=0;i<n;i++)
{
a=modadd(a,modmul(wi,src[i],m),m);
wi=modmul(wi,wj,m);
}
dst[j]=modmul(a,rN,m);
wj=modmul(wj,w,m);
}
}
//---------------------------------------------------------------------------
dst is destination array
src is source array
n is array size
m is modulus (p)
w is basis (Wn)
hope this helps to someone. If i forgot something please write ...
[edit1: fast NTT/INTT]
Finally I manage to get fast NTT/INTT to work. Was little bit more tricky than normal FFT:
//---------------------------------------------------------------------------
void _NFTT(long *dst,long *src,long n,long m,long w)
{
if (n<=1) { if (n==1) dst[0]=src[0]; return; }
long i,j,a0,a1,n2=n>>1,w2=modmul(w,w,m);
// reorder even,odd
for (i=0,j=0;i<n2;i++,j+=2) dst[i]=src[j];
for ( j=1;i<n ;i++,j+=2) dst[i]=src[j];
// recursion
_NFTT(src ,dst ,n2,m,w2); // even
_NFTT(src+n2,dst+n2,n2,m,w2); // odd
// restore results
for (w2=1,i=0,j=n2;i<n2;i++,j++,w2=modmul(w2,w,m))
{
a0=src[i];
a1=modmul(src[j],w2,m);
dst[i]=modadd(a0,a1,m);
dst[j]=modsub(a0,a1,m);
}
}
//---------------------------------------------------------------------------
void _INFTT(long *dst,long *src,long n,long m,long w)
{
long i,rN;
rN=modpow(n,m-2,m);
_NFTT(dst,src,n,m,w);
for (i=0;i<n;i++) dst[i]=modmul(dst[i],rN,m);
}
//---------------------------------------------------------------------------
[edit3]
I have optimized my code (3x times faster than code above),but still i am not satisfied with it so i started new question with it. There I have optimized my code even further (about 40x times faster than code above) so its almost the same speed as FFT on floating point of the same bit size. Link to it is here:
Modular arithmetics and NTT (finite field DFT) optimizations
To turn Cooley-Tukey (complex) FFT into modular arithmetic approach, i.e. NTT, you must replace complex definition for omega. For the approach to be purely recursive, you also need to recalculate omega for each level based on current signal size. This is possible because min. suitable modulus decreases as we move down in the call tree, so modulus used for root is suitable for lower layers. Additionally, as we are using same modulus, the same generator may be used as we move down the call tree. Also, for inverse transform, you should take additional step to take recalculated omega a and instead use as omega: b = a ^ -1 (via using inverse modulo operation). Specifically, b = invMod(a, N) s.t. b * a == 1 (mod N), where N is the chosen prime modulus.
Rewriting an expression involving omega by exploiting periodicity still works in modular arithmetic realm. You also need to find a way to determine the modulus (a prime) for the problem and a valid generator.
We note that your code works, though it is not a MWE. We extended it using common sense, and got correct result for a polynomial multiplication application. You just have to provide correct values of omega raised to certain powers.
While your code works, though, like from many other sources, you double spacing for each level. This does not lead to recursion that is as clean, though; this turns out to be identical to recalculating omega based on current signal size because the power for omega definition is inversely proportional to signal size. To reiterate: halving signal size is like squaring omega, which is like giving doubled powers for omega (which is what one would do for doubling of spacing). The nice thing about the approach that deals with recalculating of omega is that each subproblem is more cleanly complete in its own right.
There is a paper that shows some of the math for modular approach; it is a paper by Baktir and Sunar from 2006. See the paper at the end of this post.
You do not need to extend the cycle from n / 2 to n.
So, yes, some sources which say to just drop in a different omega definition for modular arithmetic approach are sweeping under the rug many details.
Another issue is that it is important to acknowledge that the signal size must be large enough if we are to not have overflow for result time-domain signal if we are performing convolution. Additionally, it may be useful to find certain implementations for exponentiation subject to modulus exist that are fast, even if the power is quite large.
References
Baktir and Sunar - Achieving efficient polynomial multiplication in Fermat fields using the fast Fourier transform (2006)
You must make sure that roots of unity actually exist. In R there are only 2 roots of unity: 1 and -1, since only for them x^n=1 can be true.
In C you have infinitely many roots of unity: w=exp(2*pi*i/N) is a primitive N-th roots of unity and all w^k for 0<=k
Now to your problem: you have to make sure the ring you're working in offers the same property: enough roots of unity.
Schönhage and Strassen (http://en.wikipedia.org/wiki/Sch%C3%B6nhage%E2%80%93Strassen_algorithm) use integers modulo 2^N+1. This ring has enough roots of unity. 2^N == -1 is a 2nd root of unity, 2^(N/2) is a 4th root of unity and so on. Furthermore, these roots of unity have the advantage that they are powers of two and can be implemented as binary shifts (with a modulo operation afterwards, which comes down to a add/subtract).
I think QuickMul (http://www.cs.nyu.edu/exact/doc/qmul.ps) works modulo 2^N-1.
How many representable floats are there between 0.0 and 0.5? And how many representable floats are there between 0.5 and 1.0? I'm more interested in the math behind it, and I need the answer for floats and doubles.
For IEEE754 floats, this is fairly straight forward. Fire up the Online Float Calculator and read on.
All pure powers of 2 are represented by a mantissa 0, which is actually 1.0 due to the implied leading 1. The exponent is corrected by a bias, so 1 and 0.5 are respectively 1.0 × 20 and 1.0 × 2−1, or in binary:
S Ex + 127 Mantissa - 1 Hex
1: 0 01111111 00000000000000000000000 0x3F800000
+ 0 + 127 1.0
0.5: 0 01111110 00000000000000000000000 0x3F000000
+ -1 + 127 1.0
Since the floating point numbers represented in this form are ordered in the same order as their binary representation, we only need to take the difference of the integral value of the binary representation and conclude that there are 0x800000 = 223, i.e. 8,388,608 single-precision floating point values in the interval [0.5, 1.0).
Similarly, the answer is 252 for double and 263 for long double.
A floating point number in IEEE754 format is between 0.0 (inclusive) and 0.5 (exclusive) if and only if the sign bit is 0 and the exponent is < -1. The mantissa bits can be arbitrary. For float, that makes 2^23 numbers per admissible exponent, for double 2^52. How many admissible exponents are there? For float, the minimal exponent for normalised numbers is -126, for double it's -1022, so there are
126*2^23 = 1056964608
float values in [0, 0.5) and
1022*2^52 = 4602678819172646912
double values.
Kerrek gave the best explanation :)
Just in case here is the code to play with other intervals too
http://coliru.stacked-crooked.com/a/7a75ba5eceb49f84
#include <iostream>
#include <cmath>
template<typename T>
unsigned long long int floatCount(T a, T b)
{
if (a > b)
return 0;
if (a == b)
return 1;
unsigned long long int count = 1;
while(a < b) {
a = std::nextafter(a, b);
++count;
}
return count;
}
int main()
{
std::cout << "number of floats in [0.5..1.0] interval are " << floatCount(0.5f, 1.0f);
}
prints
number of floats in [0.5..1.0] interval are 8388609
For 0.0..0.5: you need to worry about exponents from -1 down to as low as possible, and then multiply how many you get time the number of distinct values you can represent in the mantissa.
For every value in that range, if you double it, you get a value in the range of 0.5..1.0. And doubling it means just bumping up the exponent.
You also need to worry about unnormalized numbers, where the mantissa isn't used to represent 1.x, but 0.x, and thus will all be in your lower range, but can't be doubled by bumping up the exponent (since a particular value of the exponent is used to indicate that the value is unnormalized).
This isn't an answer per-se, but you might get some milage out of the nextafter function. Something like this ought to help you answer your question, though you'll have to work out the math yourself:
float f = 0;
while(f < 0.5)
{
print("%f (repr: 0x%x)\n", f, *(unsigned *)&f);
f = nextafterf(f, 0.5);
}
I have to check a number if it satisfies the following criteria:
in binary, all one-bits must be successive.
the number must have at least one bit set.
the successive one-bits may start at the MSB or end at the LSB, so it's perfectly valid if the number is made up of a single one-bit stream followed by a zero-bit stream or vice versa.
I wrote a code that checks for these conditions for a real-world problem (checking data-file integrity).
It works without problems and it's anything but time-critical, but I'm an old bit-twiddeling freak and love such puzzles, so I've tried to came up with a more clever way to check for single-one-bit streams.
Cases where the string is surrounded by zeros is easy, but that one can't deal with the special cases.
Any ideas, binary hacks and partial solutions are welcome!
To make my requirements more clear some examples: The following numbers satisfy my criteria:
0x80000000
0x00000001
0xff000000
0x000000ff
0xffffffff
0x000ff000
The following numbers don't (as they have more than one successive string of ones):
0xf00000f <- one-bit streams should not wrap-around at 2^n
0x0001700 <- a trivial example.
0x0000000 <- no one bit at all.
This should do what you want.
if(i == 0)
return false;
while(i % 2 == 0) {
i = i / 2;
}
return (i & (i + 1)) == 0;
bool isOK(uint val) {
while (val != 0 && (val & 1u) == 0) val >>= 1;
if (val == 0) return false;
while (val != 0 && (val & 1u) == 1) val >>= 1;
return val == 0;
}
; x86 assembly
mov eax, THE_NUMBER ; store the number in eax
bsf ecx, eax
jz .notok
mov edi, 1
shl edi, cl
mov esi, eax
add esi, edi
test esi, eax
jnz .notok
mov eax, 1
jmp .end
.notok:
mov eax, 0
.end: ; eax = 1 if satisfies the criteria, otherwise it's 0
I solved almost same problem there http://smallissimo.blogspot.fr/2012/04/meteor-contest-part-3-reducing.html in method hasInsetZero: (and used several other bit tricks)
hasInsetZero: aMask
| allOnes |
allOnes := aMask bitOr: aMask - 1.
^(allOnes bitAnd: allOnes + 1) > 0
It's Smalltalk code, but translated in C (we need the negation and care for 0), that would be:
int all_set_bits_are_consecutive( x )
/* return non zero if at least one bit is set, and all set bits are consecutives */
{
int y = x | (x-1);
return x && (! (y & (y+1)));
}
I would use bsr and bsf to determine the min and max bit set in the number. From that, create a valid number that satisfies the criteria. Compare the known valid number to the actual number for equality. No loops and only one compare.
pseudo-code:
min_bit = bsf(number);
max_bit = bsr(number);
valid_number = ~(-1 << (max_bit - min_bit) ) << min_bit;
return ( number == valid_number );
My work-in-progress version. Not ment as an answer, just to give some more ideas and to document my current approach:
int IsSingleBitStream (unsigned int a)
{
// isolate lowest bit:
unsigned int lowest_bit = a & (a-1);
// add lowest bit to number. If our number is a single bit string, this will
// result in an integer with only a single bit set because the addition will
// propagate up the the highest bit. We ought to end up with a power of two.
a += lowest_bit;
// check if result is a power of two:
return (!a || !(a & (a - 1)));
}
This code will not work if the line:
a+= lowest_bit
overflows. Also I'm unsure if my "isolate single bit" code works if there is more than one bit-field.
Assuming you want fast, the basic algorithm will be:
Find the lowest bit set.
Shift number by the index of lowest bit set.
Add 1.
If result is a power of two and non-zero, success!
All of these operations are either O(1) or O(log(integer bit width)), as follows:
unsigned int lowest_power_of_2(unsigned int value)
{ return value & -value; }
unsigned int count_bits_set(unsigned int value)
{ /* see counting bits set in parallel */ }
unsigned int lowest_bit_set_or_overflow_if_zero(unsigned int value)
{ return count_bits_set(lowest_power_of_2(value) - 1); }
unsigned int is_zero_or_power_of_2(unsigned int value)
{ return value && (value & (value - 1))==0; }
bool magic_function(unsigned in value)
{ return is_zero_or_power_of_2((value >> (lowest_bit_set_or_overflow_if_zero(lowest_power_of_2(value)))) + 1); }
Edit: updated final operation to account for zero, but the OP's algorithm is much faster since it's all constant operation (although accounting for overflow would be a PITA).
MSN
There are (N^2 - N) / 2 possibilities for a N-bit integer (496 for 32-bit).
You may use a lookup table.