I’ve started writing some experimental code for a low level implementation (based on mpn functions). The goal is that a single ball addition or multiplication should be as cheap as or cheaper than a single MPFR operation, ideally close to a mpz fixed-point arithmetic with no error bounding at all.

There are many different ways to do the low-level implementation, giving various tradeoffs between simplicity, speed, and accuracy. So far I’ve tried two approaches.

The first is to do straight up floating-point arithmetic with a full-limb radix (i.e. the digits are base 2³² or 2⁶⁴, and the exponent measures the number of digits), like the GMP mpf type, but with a single limb (aligned with the bottom of the mantissa) for the radius. When the radius overflows one limb, the precision of the midpoint is reduced one limb. This has the advantage of requiring few adjustments, and adding numbers can be done with a single mpn_add without temporary space or extra shifts. One disadvantage is that since the top limb can be nearly empty, about half an extra limb is needed on average to obtain the same effective precision as with base 2 arithmetic.

The second is to do base-2 floating-point arithmetic, i.e. with a base-2 exponent, but with the mantissa still having a precision measured in a whole number of limbs, and with a single-limb radius aligned with the last limb as in the first representation. This requires some more adjustment code to make sure that the top bit of the top limb always is set. Advantages of this representation include the fact that the full representable precision is used, and that the error propagated in multiplication can be bounded very easily if one accepts always losing 1-2 bits of precision (this is too much in general, but can be useful for dot products).

Both representations have some minor disadvantages: whenever the radius overflows a limb, the error is amplified by a large factor (however, this is a rare event on average), the results are machine-dependent (this is not a problem if one uses the ball arithmetic as a lower-level routine for a computation intended to produce an exact or precisely defined result in the end, using adaptive precision or Ziv’s strategy for correct rounding), and the lack of precise semantics makes it somewhat difficult to write strong test code.

Other possible approaches include using base-2 arithmetic with a precision measured in bits, perhaps with the radius taking up less than a whole limb (making radius calculations even cheaper). This is likely to require more code, with more (potentially expensive) adjustment steps, but might make the arithmetic more well-behaved with fewer strange worst-case situations.

In any case, I’ve compared how long time it takes to multiply two n-limb numbers using mpn, mpz, FLINT fmpz, and MPFR functions, as well as the working fmpz-based ball arithmetic code (arb), and the two experimental implementations mentioned above (arb1 and arb2). To be precise, the mpz and fmpz timings indicate the combined time to do a multiplication and a shift, necessary to do a fixed-point multiplication.

We see that arb1 and arb2 are nearly equal, and both are faster than MPFR and nearly as fast as mpz, accomplishing the goal. There are a few special cases left (e.g. zero radius) that would need handling in a proper implementation, which should add a few nanoseconds to each, but it’s also possible that the code for either could be streamlined a bit.

I also implemented addition for the arb1 type:

Again, speed is much better for arb1 than for MPFR and arb. The timings for arb1 should actually be a bit better, closer to the mpz numbers. Right now I’m actually not sure why the difference is so large. The numbers get better if you only time it with a fixed set of inputs.

So far, the code is to incomplete to draw any conclusions about which approach is better. Code complexity is probably the biggest issue. In both versions, multiplication takes 150 lines or so of very dense code, and this code does not even handle all cases yet. It would be great to get rid of more branches, but that alone is difficult. Addition is not any easier (the arb1 implementation, which is supposed to be the one where addition is simple, has 150 lines of code and about 20 branches).

Newton iteration is extremely efficient. Starting from a sufficiently accurate initial value, the number of correct digits roughly doubles with each iteration; moreover, the algorithm is “self-correcting”, so we only need to use full precision for the last iteration, 1/2 the precision for the second last iteration, 1/4 the precision for the iteration before that, etc. The total cost turns out to be only a fraction more than the time to evaluate a single exponential to the target precision.

In fact, it seems even better to use Halley’s method, which has the update step $y_{n+1} = y_n + 2 – 4t/(x + t)$ where $t = \exp(y_n)$. The amount of work per step is basically the same (one exponential and one division), but now the number of digits triples with each iteration! (We could avoid the division in the ordinary Newton iteration using $1/\exp(y_n) = \exp(-y_n)$, but it is more convenient to limit the basecase exponential to positive arguments at least for now.)

Newton’s method and Halley’s method are just the first two instances of Householder’s method, of order $d = 1$ and $d = 2$ respectively. Using higher-order Householder iterations, we can obtain arbitrarily rapid convergence, but for $d > 2$ the formulas start to become unwieldy. For $d = 3$, we have

$$y_{n+1} = \frac{6 x \left(x+2 t\right)}{x^2+4 x t+t^2}+t-3.$$

I have now implemented a first version of a basecase logarithm function, which evaluates $\log(1+x)$ on the standard interval $[0,1)$. It starts from a double-precision approximation and refines it to the target precision (modulo some rounding error) using Halley iteration. Here is how it performs (timings are in nanoseconds):

Comparing the numbers to the table from the last post, we find that log runs at around half the speed of exp. This ratio should improve at higher precision, but even at low precision, it is not a bad result, and the code is still about an order of magnitude faster than MPFR. Some savings are possible: spending around 70 nanoseconds on the initial value by calling the system logarithm is a bit excessive (a faster, sloppier double-precision logarithm could be used instead), and the division could probably be done slightly faster. But apart from that, the code is quite lean, and the only way to improve it is to improve the exponential function.

I should mention that the current code typically gives 1-2 bits less than full accuracy. To guarantee full accuracy, it is necessary to add a few guard bits. It shouldn’t actually be all that difficult to compute a rigorous error bound for the Halley iteration to make this automatic, at least in principle (translating it to correct and efficient code is probably a bit hairy), but this will have to wait since it requires error bounding for exp first.

This is actually probably the best way to compute the logarithm, up to a precision of several thousand digits. We could of course use a table lookup and a Taylor series correction, just as for exp, avoiding the overhead of Newton/Halley iteration. But a table lookup requires at least one division to transform the argument, and moreover nested table lookups aren’t possible, so we would need a much larger table for the same speed. If we use the convergence acceleration formula $\log(1+x) = 2^n \log((1+x)^{1/2^n})$), the complexity is roughly the same as for exp (with $\exp(x) = (\exp(x/2^n))^{2^n}$), but square roots are so expensive that this only becomes competitive at very high precision. Either way, it would be difficult to get rid of the 2x overhead compared to exp.

There are three main ideas. The first is simply to do arithmetic very efficiently, only using the GMP functions that are written in assembly (mpn_add_n, mpn_addmul_1, mpn_mul_basecase, etc.), avoiding divisions like the plague, eliminating temporary memory allocations and copies, and so on.

The second is to use complexity reduction techniques aggressively. For elementary (or hypergeometric) functions, one can get a complexity of $d^{2.333}$ instead of the naive $d^3$ at $d$-digit precision by combining argument reduction with baby-step/giant-step polynomial evaluation. With the proper low-level implementation, this should be effective essentially from $d = 1$.

The third is to take advantage of precomputation to speed up repeated evaluations. RAM is cheap, and in the range up to 100 digits or so, the space needed to speed up the most commonly used functions by large factors is measured in kilobytes. Of course, use of lookup tables increases cache pressure, but a cache miss is much cheaper than a whole sequence of operations on multi-limb integers, so this tradeoff is generally worth it (some benchmarking I did a while back also indicated that prefetch instructions worked wonderfully for this purpose, at least on my CPU).

I have now written a new basecase implementation for exp(), substantially improved compared to my previous experiments; the code can be found here. Using a modest 224 KB lookup table (with a small modification, this could be trimmed to about 160 KB) for 16-bit argument reduction allows going up to a precision of about 384 bits, or 115 decimal digits, on a 64-bit system. In other words plenty for most practical purposes. The lookup table itself takes just a few microseconds to generate dynamically.

Here is how it compares to some other libraries: the system double-precision exp, the double-double (dd) and quad-double (qd) types from D. H. Bailey’s QD library (measured using the supplied tests/qd_timer program), MPFR, and mpmath (using the Python implementation with GMPY types, not the code in Sage which wraps MPFR). All timings are in nanoseconds.

A couple of things need to be pointed out.

Right now, this code is experimental. I have not tested it thoroughly, and it does not come with an error bound (however, the error is never larger than a few ulp, and a bound can be bounded quite easily, for which I intend to include code later on).

All other libraries take an arbitrary $x$, not necessarily restricted to $[0,\ln 2)$. The initial argument reduction is a division with remainder, where a low-precision value of $1/\ln 2$ can be precomputed, meaning that only a single mpn_submul_1 plus some adjustments should be necessary.

The system exponential function, MPFR and mpmath also effectively use higher internal precision to compute the exponential with 0.5 or 1 ulp error, so to be fair a few guard bits should be added.

Even after adding all remaining corrections, the code can actually probably be made faster. At 1 or 2 limb precision, replacing all calls to GMP functions with inline assembly should improve performance; the current code also contains a redundant division, which should save 60 nanoseconds or so when removed.

Among the elementary functions, exp is the easiest to implement; cos, sin, atan and log can be computed using similar principles, although there is some more overhead, so they will perhaps be a factor two slower. The next logical step is to add a second version of exp, for precisions between 384 bits and a few thousand digits. It will necessarily be a bit slower than the basecase version, but should still be a decent factor faster than the MPFR function in this range.

Here are timings compared to MPFR for even $n$:

Here the Arb function either uses the formula $\zeta(2n) = (-1)^{n+1} B_{2n} (2\pi)^{2n} / (2(2n)!)$ (computing the Bernoulli number $B_{2n}$ exactly using FLINT), or the Euler product $(\zeta(n))^{-1} = \prod_{p} (1-p^{-n})$ for large $n$. You can see the cutoff quite clearly in the plot. At any size, this strategy is much faster than the generic algorithm used by MPFR.

In fact, since FLINT uses the Euler product to compute Bernoulli numbers, the Euler product effectively always ends up being used. That is, one basically uses the Euler product to compute $\zeta(n)$ to full precision, or to compute $B_n$ exactly, whichever requires less precision.

Timings for odd $n$ look a bit different:

There is a visible speedup at very high precision for small $n$ due to the use of binary splitting (timings can be improved further by writing better code for the special cases $n = 5, 7$), and a speedup for asymptotically large $n$ where Arb again uses the Euler product. In between, the code simply falls back to the MPFR function, so the timings are identical.

The pressing question is whether one can do better than MPFR here. Euler-Maclaurin summation might be a bit faster if optimized carefully, but I can’t really think of anything that would make a huge difference.

Another way to compute $\zeta(n)$ to high precision when $n$ is not too large is to use the algorithm of E. Karatsuba. However, my tests so far indicate that it is much slower than other methods in practice.

These timings are all for computing $\zeta(n)$ as an isolated value. Much better performance is possible when computing $\zeta(2), \zeta(3), \ldots, \zeta(n)$ simultaneously. More on that at a later time…

Ball arithmetic is intended to allow efficient, rigorous high-precision numerical evaluation. A real ball is implemented as an interval $[x-r, x+r] \times 2^e$ where $x, e, r$ are integers. This representation supports error propagation while avoiding the ~2x speed and memory overhead of the more conventional endpoint interval representation $[a \times 2^e, b \times 2^f]$.

There are two drawbacks: balls are somewhat less flexible, and they have slightly higher overhead at low precision. At the moment multiplying two arb_t numbers is about 3x slower than multiplying two mpfr_t numbers at 100-400 bits (which would make it somewhat less efficient than MPFI arithmetic). Above 1000 bits, the difference is only a few percent.

The current code is completely naive, and with a bit of low-level hacking it should be possible to get similar performance to a single mpfr_t multiplication at 200-1000 bits. Up to perhaps 256 bits, it would be attractive to have faster, fixed-precision interval types as a complement. (There is actually some prototype code for this hidden somewhere in a branch of my FLINT repository on GitHub…)

I will soon add complex numbers, polynomial balls, and matrix balls (based on exact FLINT polynomials and matrices). In fact, the main reason that I created an arb_t scalar type at all is to simplify interfacing with and writing test code for polynomial balls. But the type can clearly be useful on its own. When the code stabilizes enough, I plan to merge all this back into FLINT.

Part of the design of the arb_t type is that it can hold small integers efficiently, only truncating them to approximate numbers when they grow very large. This is useful for performing binary splitting on infinite series, reducing memory usage (and possibly improving performance) compared to using exact integers throughout.

The arb_t type is still very incomplete (and some functions don’t quite work as they should in all cases), but the module is functional enough that I’ve been able to implement binary splitting of a couple of constants. It’s interesting to see how much of an improvement (if any) ball arithmetic gives over using plain integers, so I’m including some benchmarks below. These functions are not the goal of the Arb library; I plan to do more useful things with it later, but they’re fun to look at as a start.

First out are some timings for π using the Chudnovsky algorithm, compared to the MPFR pi function, Mathematica 7, and the slightly modified version of the gmp-chudnovsky program included in FLINT. All times are measured in seconds.

Using ball arithmetic turns out to be pretty much exactly as fast as using plain integers (timings not included), so at least the arb_t is not adding much overhead compared to integers. The gmp-chudnovsky code is around 40% faster because it works with partially factored integers, requiring a lot of extra code (it would be interesting to turn this into a general integer type and find other uses for it) and because it uses heuristic numerical code for the final square root and division. MPFR is slower because it uses the AGM algorithm. Mathematica is probably slower because it relies on an old GMP version.

Timings for Euler’s constant (γ) using the Brent-McMillan binary splitting algorithm, compared to Mathematica, MPFR and the integer (fmpz) binary splitting code in the FLINT arith in FLINT:

Ball arithmetic gives a small speedup. Memory usage is reduced significantly compared to the fmpz code. I was also able to compute 10⁷ digits, but I lost the output of how long it took (roughly half an hour, I think). MPFR is slower because it uses a different algorithm.

The speedup for the binary splitting stage is actually larger than indicated, because 1/3 of the total time is spent computing log(n) with MPFR, where n is a parameter. One can save time by choosing n to be a power of two (log(2) is fast), but this makes the binary splitting slower when n is much smaller than the next power of two. I don’t know a general way to compute log(n) faster. Perhaps one could round n up to $m 2^e$ where $m \le 7$, and try to compute log(2), log(3), log(5), log(7) quickly. It’s worth pointing out that the binary splitting for power-of-two parameter can be optimized a bit by using bit shifting; this is not implemented yet.

Riemann zeta constants $\zeta(n)$ for integer n, compared to MPFR and the fmpz binary splitting code in the FLINT arith module:

Here ball arithmetic gives a nice speedup which increases with larger n. Mathematica and MPFR both use algorithms with complexity somewhere around $\tilde O(b^2)$ where $b$ is the number of digits, whereas the algorithm I implemented has complexity $\tilde O(nb)$. Due to its overhead, it only becomes fast for small n and b in the tens of thousands.

Finally, using a separate hypergeometric series for $\zeta(3)$, this particular constant can be computed even faster. Here ball arithmetic gives a very small speed improvement over using fmpzs:

(I’m omitting MPFR because it doesn’t have a separate algorithm implemented for this constant.)

Later on, I will add faster code for $\zeta(n)$ where n is large or the precision is small, and code for combined evaluation of $\zeta(2), \zeta(3), \ldots, \zeta(n)$ (used for series expansions of various functions).

The number $p(10^{19})$ turns out to have 3,522,804,578 digits, starting and ending with

56469284039962075996762611156427010823552403269435
56912115036797282138529825391237281687215156415242
...
          <3522804378 digits omitted>
                                               ...
18143637106903146536175827132211646720469971909621
95008234537397439410884324996338596264493674631046

More simply, $p(10^{19}) \approx 5.65 \times 10^{3,522,804,577}$ (this kind of approximation is easy to compute; it is the least significant digits that are challenging — or the more significant digits, for the $p$-adically inclined). This number answers the following practically important combinatorial problem: in how many ways can one choose sticks whose lengths are multiples of one meter, such that they add up to $10^{19}$ meters — approximately the thickness of the Milky Way — when placed end to end?

Okay, computing $p(10^n)$ is just a frivolous benchmark problem. Perhaps more interestingly, at least for number theorists, I generated over 22 billion new Ramanujan-type congruences (identities of the form $p(Ak+B) \equiv 0 \bmod m$ for all $k$) using the algorithm of Rhiannon Weaver. This greatly extends the computation of 76,065 congruences done by Weaver in 2001. An example of a new identity is

$$p(28995244292486005245947069 k + 28995221336976431135321047) \equiv 0 \; \operatorname{mod} \; 29$$

for all values of $k$.

Finding the 22 billion new congruences required evaluating the partition function of approximately 470,000 distinct values up to $n \approx 10^{13}$. This took approximately 150 CPU days distributed over 40-48 cores on a computer at the University of Warwick (access provided courtesy of Bill Hart).

The computation of $p(10^{19})$ took just less than 100 hours of CPU time and roughly 150 GiB of RAM, running on a single core. With 2-3 months of CPU time and 2 TB of swap space, it should be possible to compute the number of partitions of one sextillion ($10^{21}$) with the FLINT code, if anyone is up for the task (unfortunately, parallelization for a single $n$ is not supported, and would be difficult to implement to save more than a factor two).

There are actually three implementations of the partition function in FLINT: one for computing $p(0), p(1), \ldots p(n-1)$, another for computing the same set of values modulo a word-size integer, and finally a function for computing the isolated value $p(n)$. The first two are straightforward applications of FLINT’s fast power series arithmetic, and there is perhaps not that much to say about them (with several gigabytes of RAM, you can comfortably compute perhaps $10^6$ exact values or $10^9$ values modulo a small integer).

The computation of isolated values uses the Hardy-Ramanujan-Rademacher formula

$$p(n)=\frac{1}{\pi \sqrt{2}} \sum_{k=1}^\infty \sqrt{k}\, A_k(n)\,
\frac{d}{dn} \left(
\frac {1} {\sqrt{n-\frac{1}{24}}}
\sinh \left[ \frac{\pi}{k}
\sqrt{\frac{2}{3}\left(n-\frac{1}{24}\right)}\right]
\right)$$

where

$$A_k(n) = \sum_{0 \le m < k, (m,k) = 1} \,e^{ \pi i \left[ s(m,\, k) \;-\; \frac{1}{k} 2 nm \right]}$$

and $s(m,k)$ denotes a Dedekind sum. This is a numerical infinite series that has to be evaluated approximately. Truncating it appropriately and using a sufficiently high numerical precision gives an approximation that is guaranteed to round to the correct integer.

The FLINT implementation has complexity $O(n^{1/2+\varepsilon})$, which is quasi-optimal since $p(n)$ has about $n^{1/2}$ digits. Implementing the Hardy-Ramanujan-Rademacher formula with this complexity is rather complicated. Ostensibly, there are $n^{3/2}$ terms in the nested sums, and on top of that you need to work with numbers up to $O(n^{1/2})$ digits.

It was only after a lot of digging in the literature that I stumbled across a paper of A. L. Whiteman that gives identities for factoring the exponential sums $A_k(n)$ into short cosine products, reducing the number of terms. This uses quite a bit of modular arithmetic, and implementing it efficiently relies on the fast routines in FLINT for computing gcd, modular square roots, Legendre symbols, factorizations, etc. of word-size integers.

The numerical computations are done using a combination of MPFR arithmetic, some custom routines for high-precision transcendental functions, and machine precision arithmetic for small terms, with carefully managed precision across the whole computation.

Until now, the largest reported computations of $p(n)$ have involved $n$ around $10^9$. Some software certainly allowed going higher than $10^9$, but this was incredibly slow. My code makes it easy to go much higher on commodity hardware, the primary limitation being available memory. For example, computing $p(10^{16})$ with FLINT takes less than two hours on my laptop, using all the available 3 GiB of RAM (plus some swap space).

Compared to the previously best software (Mathematica and Sage), the FLINT code runs around 500 times faster for large $n$. For example $p(2^{32}-1)$ takes half a second in FLINT and around 200 seconds in both Sage and Mathematica.

The image is a loglog plot of the time needed to compute $p(n)$ using Mathematica 7 (green circles), Sage 4.7 (red triangles), and FLINT (blue squares). The thin dotted line indicates the slope of the trivial lower complexity bound $\Omega(n^{1/2})$ just for writing out the result. A quick extrapolation suggests that $p(10^{19})$ would take about a decade to compute with Mathematica and a million years with Sage (after patching Sage to allow $n$ larger than 32 bits).

More details about the FLINT partition function implementation (and the computational results) will be given in a forthcoming paper.

The running time is obviously $O(n)$ (assuming that $n$ fits in a single machine word so that arithmetic has constant cost). While elegant, this algorithm is useless in practice because even the most basic non-stupid primality test, trial division, only costs $O(n^{1/2})$ in the worst case.

Perhaps surprisingly, the complexity of primality testing using Wilson’s theorem can be brought down to about $O(n^{1/2})$ as well. The idea is to use fast polynomial arithmetic to compute the factorial faster than by the naive method. Assuming for simplicity that $n – 1$ is a perfect square, let $m = (n-1)^{1/2}$. We form the polynomial $P(x) = (x+1)(x+2) \cdots (x+m)$ and evaluate $P(x)$ simultaneously at the points $x = 0, m, \ldots m(m-1)$. Then $P(0) P(m) \cdots P(m(m-1)) = (n-1)!$. If $n – 1$ is not a perfect square, we choose $m = \lfloor n^{1/2} \rfloor$ and fill in the missing factors by repeated naive multiplication.

All the required operations can be done in roughly $O(n^{1/2})$ time using FFT-based polynomial multiplication and balanced subproduct trees. The complexity is actually slightly higher, about $O(n^{1/2} \log^3 n)$, and the constant overhead is huge, so the method is still considerably slower than trial division. But it is interesting to see how it performs in practice.

Since I recently implemented fast multipoint evaluation in FLINT, the fast factorial algorithm became easy to implement as well. In my repository, it is now enabled by default for computing factorials modulo an integer (n_factorial_mod2_preinv) when the input is large enough; the code is here. Here is how it compares to the naive algorithm for computing $(n-1)!$ modulo $n$:

The numbers agree reasonably well with theory (the naive algorithm does not quite take a constant multiple of $n$ time because the arithmetic is done slightly faster when several factors can be accumulated in a single word, and the ratio per order of magnitude for the fast algorithm is a bit larger than 4 and not $10^{1/2} \approx 3.16$, but that is reasonable considering the extra log factors in the complexity). Around 10¹⁵, my laptop is running out of memory. To compute larger factorials, one could choose $m$ smaller than the square root of the input, performing several multipoint evaluations. Choosing a fixed large $m$ (say 10⁷) gives an algorithm with $O(n)$ complexity, but a much smaller constant than the naive factorial algorithm.

Although we have achieved a factor 1000 speedup over the naive factorial algorithm and made Wilson’s theorem a feasible primality test for numbers as large as 15 digits without requiring special hardware or patience, it remains completely useless for practical purposes. Trial division is around 10,000 times faster (dividing by all integers up to $(10^{14})^{1/2}$ at 10 ns per iteration takes 0.1 seconds), and sophisticated algorithms are much faster still (the FLINT function n_is_prime tests primality of a number this size in about 7 μs, and n_factor factors it in about 500 μs).

I’ve imported the entries from the old blog. Unfortunately, the plugin I used messed up all <pre> text, so there are now a million broken code examples. I have fixed up the most recent half of the archive manually, but the oldest posts are still broken.

Not that I’ve been blogging that much recently, but you never know… it goes in waves.

In this blog post, I’m going to talk a bit about features I contributed in this version. With apologies to Sebastian Pancratz who wrote a whole lot of code as well — in particular, a new module for computing with p-adic numbers, and a module for rational functions! Bill Hart also implemented a faster polynomial GCD, which is a quite important update since GCD is crucial for most polynomial business. Anyhow…

Polynomial interpolation

I’ve added various functions for polynomial interpolation to the fmpz_poly module. In general, these can be used to speed up various computations involving integer or rational polynomials by mapping a given problem to (Z/nZ)[x], Z or even Z/nZ, taking advantage of fast arithmetic in those rings, and then via interpolation recovering a result in Z[x] or Q[x].

Firstly, there are some new Chinese Remainder Theorem functions for integer polynomials, allowing you to reconstruct an integer polynomial from a bunch of polynomials with coefficients modulo different primes. Straightforward code (the actual work is done by functions in the fmpz module), but useful to have. The CRT functions are used by the new modular GCD code.

There are also functions for evaluating an integer polynomial on a set of points, and forming the interpolating polynomial (generally with rational coefficients) given a set of points and the values at those points.

Finally, user-friendly functions for evaluation and interpolation at a power of two (Kronecker segmentation) have been added. The code for this is actually a very old part of FLINT, and possibly some of the most complicated code in the library (packing bits efficiently is surprisingly hard). The new functions just wrap this functionality, but take care of memory management and various special cases, so you can now just safely do something like:

fmpz_t z;
fmpz_init(z);

long bits = fmpz_poly_max_bits(poly) + 1; /* +1 for signs */
fmpz_poly_bit_pack(z, poly, bits);
fmpz_poly_bit_unpack(poly, z, bits);  /* recover poly */
fmpz_clear(z);

Apart from Kronecker segmentation, these functions are not currently asymptotically fast. Fast multi-modulus CRT for coefficient reconstruction is probably not all that important in most circumstances, because it’s more common to use evaluation-interpolation techniques for polynomials of large degree and small coefficients than the other way around. Nonetheless, polynomials with large coefficients do arise as well. For example, the vector Bernoulli number code in FLINT relies on fast CRT, and currently uses custom code for this.

Polynomial interpolation uses Lagrange interpolation with barycentric weights, with a few tricks to avoid fractions. This is all implemented using an O(n^2) algorithm, but the actual time complexity is higher due to the fact that the coefficients when working over integers usually will be large, around n! in magnitude.

Here are some timing examples, evaluating and recovering a length-n polynomial with +/-1 coefficients basically as follows:

x = _fmpz_vec_init(n);
y = _fmpz_vec_init(n);
fmpz_poly_init(P);
fmpz_poly_init(Q);
for (i = 0; i < n; i++)
    x[i] = -n/2 + i;
fmpz_poly_randtest(P, state, n, 1);
fmpz_poly_evaluate_fmpz_vec(y, P, x, n);
fmpz_poly_interpolate_fmpz_vec(Q, x, y, n);

The bits column below measures the largest value in y, which grows quite large despite the input polynomial having small coefficients:

n=8  eval=762 ns  interp=13 us  bits=8  ok=1
n=16  eval=3662 ns  interp=61 us  bits=42  ok=1
n=32  eval=29 us  interp=673 us  bits=-113  ok=1
n=64  eval=136 us  interp=4951 us  bits=-316  ok=1
n=128  eval=625 us  interp=45 ms  bits=-762  ok=1
n=256  eval=2500 us  interp=792 ms  bits=-1779  ok=1
n=512  eval=12 ms  interp=10 s  bits=-4089  ok=1

As you can see, the interpolation speed is not too bad for small n, but eventually grows out of control. How to do better?

Naive Lagrange interpolation is not optimal: it is possible to do n-point evaluation and interpolation in essentially O(n log² n) operations. Such algorithms do not necessarily lead to an improvement over the integers (you still have to deal with coefficient explosion), but they should win over finite fields. So the right solution will perhaps be to add polynomial evaluation/interpolation functions based on modular arithmetic.

Rational numbers and matrices

A new module fmpq is provided for computing with arbitrary-precision rational numbers. For the user, the fmpq_t type essentially behaves identically to the MPIR mpq_t type. However, an fmpq_t only takes up two words of memory when the numerator and denominator are small (less than 2⁶²), whereas an mpq_t always requires six words plus additional heap-allocated space for the actual number data.

The fmpq functions are a bit faster than mpq functions in many cases when the numerator and/or denominator is small. But the main improvement should come for vectors, matrices or polynomials of rational numbers, due to the significantly reduced memory usage and memory management overhead (especially when many entries are zero or integer-valued).

Some higher-level functionality is also provided in the fmpq module, e.g. for rational reconstruction. The functions for computing special rational numbers (like Bernoulli numbers) have also been switched over to the fmpq type. Another supported feature is enumeration of the rationals (using the Calkin-Wilf sequence or by height). Generating the 100 million “first” positive rational numbers takes 9.6 seconds done in order of height, or 2.6 seconds in Calkin-Wilf order.

FLINT actually does not use fmpq’s to represent polynomials over Q (fmpq_poly), and probably never will. The fmpq_poly module represents a polynomial over Q as an integer polynomial with a single common denominator, which is usually faster. The reason for adding the fmpq_t type is that it enabled developing the new fmpq_mat module, which implements dense matrices of rational numbers. For matrices, a common-denominator representation would be less convenient and in many cases completely impractical.

The new FLINT fmpq_mat module is very fast, or at least very non-slow. It is easy to find examples where it does a simple computation a thousand times faster than the rational matrices in Sage.

There’s not actually much code in the fmpq_mat module itself; it does almost all “level 3″ linear algebra (computations requiring matrix multiplication or Gaussian elimination) by clearing denominators and computing over the integers. This approach is in fact stolen shamelessly from Sage, but the functions in Sage are highly unoptimized in many cases. The code in Sage still wins for many sufficiently large problems as it has asymptotically fast algorithms for many things we do not (like computing null spaces). See the benchmarks page for more details.

I should not forget to mention that I’ve implemented Dixon’s p-adic algorithm for solving Ax = b for nonsingular square A. (I wish I had a good link for Dixon’s algorithm here, but sadly it doesn’t appear to be described conveniently anywhere on the web. The original paper is “Exact solution of linear equations using P-adic expansions“, if you have the means to get through the Springer paywall.)

This is now used both for solving over both Z and Q. The solver in FLINT is competitive with Sage (which uses IML+ATLAS) up to systems of dimension somewhere between perhaps 100 and 1000 (depending greatly on the size of the entries in the inputs and in the solution!). There’s much to do here — we should eventually have BLAS support in FLINT, which will speed up core matrix arithmetic, but there’s room for a lot of algorithmic tuning as well.

There are some other minor new matrix features as well… they can be found in the changelog.

Polynomial matrices

A new module (fmpz_poly_mat) is provided for dense matrices over Z[x], i.e. matrices whose entries are polynomials with integer coefficients. The available functionality includes matrix multiplication, row reduction, and determinants. Matrix multiplication is particularly fast, as it uses the Kronecker segmentation interpolation/evaluation technique described above. (A similar algorithm is provided for determinants, but it’s not really optimal as this point.)

The benchmarks page has detailed some detailed timings, so I won’t repeat them here — but generally speaking, the FLINT implementation is an order of magnitude faster than Sage or Magma for matrices of manageable size.

There’s much more to be done for polynomial matrices. Row reduction is implemented quite efficiently, but it’s too slow as an algorithm for many tasks such as computing null spaces of very large matrices. A future goal is to implement asymptotically fast algorithms (see the paper on x-adic lifting by Burçin Eröcal and Arne Storjohann for example).

A few of the formulas explicitly involve pi. Using those to calculate pi is rather circular (!), though a few of them could still be used for computing pi using numerical root-finding. In any case, most of the formulas are circular even when pi doesn’t appear explicitly since mpmath is likely using its value internally. In any further case, the majority of the formulas are not efficient for computing pi to very high precision (at least as written). Still, ~50 digits is no problem. Enjoy!

from mpmath import *
mp.dps = 50; mp.pretty = True

+pi
180*degree
4*atan(1)
16*acot(5)-4*acot(239)
48*acot(49)+128*acot(57)-20*acot(239)+48*acot(110443)
chop(2*j*log((1-j)/(1+j)))
chop(-2j*asinh(1j))
chop(ci(-inf)/1j)
gamma(0.5)**2
beta(0.5,0.5)
(2/diff(erf, 0))**2
findroot(sin, 3)
findroot(cos, 1)*2
chop(-2j*lambertw(-pi/2))
besseljzero(0.5,1)
3*sqrt(3)/2/hyp2f1((-1,3),(1,3),1,1)
8/(hyp2f1(0.5,0.5,1,0.5)*gamma(0.75)/gamma(1.25))**2
4*(hyp1f2(1,1.5,1,1) / struvel(-0.5, 2))**2
1/meijerg([[],[]], [[0],[0.5]], 0)**2
(meijerg([[],[2]], [[1,1.5],[]], 1, 0.5) / erfc(1))**2
(1-e) / meijerg([[1],[0.5]], [[1],[0.5,0]], 1)
sqrt(psi(1,0.25)-8*catalan)
elliprc(1,2)*4
elliprg(0,1,1)*4
2*agm(1,0.5)*ellipk(0.75)
(gamma(0.75)*jtheta(3,0,exp(-pi)))**4
cbrt(gamma(0.25)**4*agm(1,sqrt(2))**2/8)
sqrt(6*zeta(2))
sqrt(6*(zeta(2,3)+5./4))
sqrt(zeta(2,(3,4))+8*catalan)
exp(-2*zeta(0,1,1))/2
sqrt(12*altzeta(2))
4*dirichlet(1,[0,1,0,-1])
2*catalan/dirichlet(-1,[0,1,0,-1],1)
exp(-dirichlet(0,[0,1,0,-1],1))*gamma(0.25)**2/(2*sqrt(2))
sqrt(7*zeta(3)/(4*diff(lerchphi, (-1,-2,1), (0,1,0))))
sqrt(-12*polylog(2,-1))
sqrt(6*log(2)**2+12*polylog(2,0.5))
chop(root(-81j*(polylog(3,root(1,3,1))+4*zeta(3)/9)/2,3))
2*clsin(1,1)+1
(3+sqrt(3)*sqrt(1+8*clcos(2,1)))/2
root(2,6)*sqrt(e)/(glaisher**6*barnesg(0.5)**4)
nsum(lambda k: 4*(-1)**(k+1)/(2*k-1), [1,inf])
nsum(lambda k: (3**k-1)/4**k*zeta(k+1), [1,inf])
nsum(lambda k: 8/(2*k-1)**2, [1,inf])**0.5
nsum(lambda k: 2*fac(k)/fac2(2*k+1), [0,inf])
nsum(lambda k: fac(k)**2/fac(2*k+1), [0,inf])*3*sqrt(3)/2
nsum(lambda k: fac(k)**2/(phi**(2*k+1)*fac(2*k+1)), [0,inf])*(5*sqrt(phi+2))/2
nsum(lambda k: (4/(8*k+1)-2/(8*k+4)-1/(8*k+5)-1/(8*k+6))/16**k, [0,inf])
2/nsum(lambda k: (-1)**k*(4*k+1)*(fac2(2*k-1)/fac2(2*k))**3, [0,inf])
nsum(lambda k: 72/(k*expm1(k*pi))-96/(k*expm1(2*pi*k))+24/(k*expm1(4*pi*k)), [1,inf])
1/nsum(lambda k: binomial(2*k,k)**3*(42*k+5)/2**(12*k+4), [0,inf])
4/nsum(lambda k: (-1)**k*(1123+21460*k)*fac2(2*k-1)*fac2(4*k-1)/(882**(2*k+1)*32**k*fac(k)**3), [0,inf])
9801/sqrt(8)/nsum(lambda k: fac(4*k)*(1103+26390*k)/(fac(k)**4*396**(4*k)), [0,inf])
426880*sqrt(10005)/nsum(lambda k: (-1)**k*fac(6*k)*(13591409+545140134*k)/(fac(k)**3*fac(3*k)*(640320**3)**k), [0,inf])
4/nsum(lambda k: (6*k+1)*rf(0.5,k)**3/(4**k*fac(k)**3), [0,inf])
(ln(8)+sqrt(48*nsum(lambda m,n: (-1)**(m+n)/(m**2+n**2), [1,inf],[1,inf]) + 9*log(2)**2))/2
-nsum(lambda x,y: (-1)**(x+y)/(x**2+y**2), [-inf,inf], [-inf,inf], ignore=True)/ln2
2*nsum(lambda k: sin(k)/k, [1,inf])+1
quad(lambda x: 2/(x**2+1), [0,inf])
quad(lambda x: exp(-x**2), [-inf,inf])**2
2*quad(lambda x: sqrt(1-x**2), [-1,1])
chop(quad(lambda z: 1/(2j*z), [1,j,-1,-j,1]))
3*(4*log(2+sqrt(3))-quad(lambda x,y: 1/sqrt(1+x**2+y**2), [-1,1],[-1,1]))/2
sqrt(8*quad(lambda x,y: 1/(1-(x*y)**2), [0,1],[0,1]))
sqrt(6*quad(lambda x,y: 1/(1-x*y), [0,1],[0,1]))
sqrt(6*quad(lambda x: x/expm1(x), [0,inf]))
quad(lambda x: (16*x-16)/(x**4-2*x**3+4*x-4), [0,1])
quad(lambda x: sqrt(x-x**2), [0,0.25])*24+3*sqrt(3)/4
mpf(22)/7 - quad(lambda x: x**4*(1-x)**4/(1+x**2), [0,1])
mpf(355)/113 - quad(lambda x: x**8*(1-x)**8*(25+816*x**2)/(1+x**2), [0,1])/3164
2*quadosc(lambda x: sin(x)/x, [0,inf], omega=1)
40*quadosc(lambda x: sin(x)**6/x**6, [0,inf], omega=1)/11
e*quadosc(lambda x: cos(x)/(1+x**2), [-inf,inf], omega=1)
8*quadosc(lambda x: cos(x**2), [0,inf], zeros=lambda n: sqrt(n))**2
2*quadosc(lambda x: sin(exp(x)), [1,inf], zeros=ln)+2*si(e)
exp(2*quad(loggamma, [0,1]))/2
2*nprod(lambda k: sec(pi/2**k), [2,inf])
s=lambda k: sqrt(0.5+s(k-1)/2) if k else 0; 2/nprod(s, [1,inf])
s=lambda k: sqrt(2+s(k-1)) if k else 0; limit(lambda k: sqrt(2-s(k))*2**(k+1), inf)
2*nprod(lambda k: (2*k)**2/((2*k-1)*(2*k+1)), [1,inf])
2*nprod(lambda k: (4*k**2)/(4*k**2-1), [1, inf])
sqrt(6*ln(nprod(lambda k: exp(1/k**2), [1,inf])))
nprod(lambda k: (k**2-1)/(k**2+1), [2,inf])/csch(pi)
nprod(lambda k: (k**2-1)/(k**2+1), [2,inf])*sinh(pi)
nprod(lambda k: (k**4-1)/(k**4+1), [2, inf])*(cosh(sqrt(2)*pi)-cos(sqrt(2)*pi))/sinh(pi)
sinh(pi)/nprod(lambda k: (1-1/k**4), [2, inf])/4
sinh(pi)/nprod(lambda k: (1+1/k**2), [2, inf])/2
(exp(1+euler/2)/nprod(lambda n: (1+1/n)**n * exp(1/(2*n)-1), [1, inf]))**2/2
3*sqrt(2)*cosh(pi*sqrt(3)/2)**2*csch(pi*sqrt(2))/nprod(lambda k: (1+1/k+1/k**2)**2/(1+2/k+3/k**2), [1, inf])
2/e*nprod(lambda k: (1+2/k)**((-1)**(k+1)*k), [1,inf])
limit(lambda k: 16**k/(k*binomial(2*k,k)**2), inf)
limit(lambda x: 4*x*hyp1f2(0.5,1.5,1.5,-x**2), inf)
1/log(limit(lambda n: nprod(lambda k: pi/(2*atan(k)), [n,2*n]), inf),4)
limit(lambda k: 2**(4*k+1)*fac(k)**4/(2*k+1)/fac(2*k)**2, inf)
limit(lambda k: fac(k) / (sqrt(k)*(k/e)**k), inf)**2/2
limit(lambda k: (-(-1)**k*bernoulli(2*k)*2**(2*k-1)/fac(2*k))**(-1/(2*k)), inf)
limit(lambda k: besseljzero(1,k)/k, inf)
1/limit(lambda x: airyai(x)*2*x**0.25*exp(2*x**1.5/3), inf, exp=True)**2
1/limit(lambda x: airybi(x)*x**0.25*exp(-2*x**1.5/3), inf, exp=True)**2