machine_vectors.h

Loads 1, 2, 4 or 8 consecutive doubles. The vec2d versions are available on NEON and the generic backends, the vec8dz versions with AVX-512 only, and the others on all backends.

Stores the lanes of a at consecutive addresses starting at z. Availability as for the loads.

Loads and stores of floats. The vec16f versions are available on the generic backends only, the vec16fz versions with AVX-512 only, and the others on all backends.

Loads and stores of four words. All backends.

Same as vec4n_store_unaligned, but done as two 16-byte stores on AVX2. AVX2 and NEON.

Loads eight words. AVX2 and NEON.

vec4d_to_index converts lanes holding integers in [0, 2^{31}) to table indices, and vec4d_gather returns \{t[i_0], t[i_1], t[i_2], t[i_3]\}. All backends.

Returns a vector with every lane equal to a. Availability by type as for the loads.

Returns a vector of zeros. Availability by type as for the loads.

Returns a vector of ones. vec4d_one is available on all backends, vec1d_one on AVX2 and NEON.

Returns the vector with lanes a_0, a_1, \ldots in this order. vec4d_set_d4 is available on all backends, vec8d_set_d8 on AVX2 and NEON.

Returns lane i of a, for 0 \le i < 4 and 0 \le i < 8 respectively. vec4d_get_index is available on all backends, vec8d_get_index on AVX2 and NEON.

Prints a to standard output. vec4d_print prints the lanes in order as {a0, a1, a2, a3} with %f. vec4n_print prints them in hexadecimal from lane 3 down to lane 0, as [hi a3_a2_a1_a0 lo]. AVX2 only.

Returns a + b, a - b and ab. Availability by type as for the loads.

Returns a + b, a - b and ab in single precision. Availability by type as for the loads.

Returns a / b. vec4d_div is available on all backends, the others on AVX2 and NEON.

Returns -a and |a|, which flip and clear the sign bit (also of zeros and NaNs). The vec4d versions are available on all backends, the others on AVX2 and NEON.

Returns the smaller and the larger of a and b. When a lane is unordered (a NaN is involved), AVX2 and the generic backends return the lane of b, NEON returns a NaN, and vec1d_min and vec1d_max, which are fmin and fmax, return the operand that is not a NaN. The vec4d versions are available on all backends, the others on AVX2 and NEON.

Returns a/2, computed as a multiplication by 0.5. vec1d_half is available on AVX2 and NEON, vec4d_half on AVX2 only.

Subtracts in even lanes and adds in odd lanes: vec4d_addsub returns \{a_0 - b_0, a_1 + b_1, a_2 - b_2, a_3 + b_3\} and vec1d_addsub returns a - b. vec1d_addsub is available on AVX2 and NEON, vec4d_addsub on AVX2 only.

Returns the correctly rounded square root. All backends.

Rounds to an integer in the current rounding mode: to the nearest integer, ties to even, by default. On NEON, vec4d_round and vec8d_round always round to nearest with ties to even. vec4d_round is available on all backends, the others on AVX2 and NEON.

Returns \lfloor a \rfloor. vec8d_floor and vec16f_floor are available on the generic backends only, the z versions with AVX-512 only, and the others on all backends (vec2d_floor also on AVX2).

Returns ab + c and -ab + c. On AVX2, AVX-512 and NEON, and for vec1d and vec1f on every backend, the result is rounded once. On the generic backends the wider types compute a * b + c and c - a * b, which round twice unless the compiler contracts them. Availability by type as for the loads.

Single-precision versions, with the same rounding behaviour. Availability by type as for the loads.

Returns ab - c and -ab - c, rounded once except for vec4d_fmsub on the generic backends. NEON computes ab - c as -(c - ab), which differs from vec4d_fmsub_fused in the sign of an exact zero. vec4d_fmsub is available on all backends, the others on AVX2 and NEON.

Returns ab + c, ab - c and -ab + c rounded once on every backend, like fma(a, b, c), fma(a, b, -c) and fma(-a, b, c), including the sign of an exact zero. On AVX2 without FMA instructions they call fma lane by lane. All backends.

Returns the mask of the lanes where a_i = b_i, a_i \ne b_i, a_i < b_i, and so on. The predicates eq, lt, le, gt and ge are false when a lane of a or b is a NaN, and their negations ne, nlt, nle, ngt and nge are true. All backends.

Returns, in each lane, b_i if the sign bit of c_i is set and a_i otherwise, so a mask from a comparison selects b where the comparison holds. vec1d_blendv instead tests c >= 0, which differs for c = -0 (selects a) and for NaNs (selects b). vec4d_blendv is available on all backends, the others on AVX2 and NEON.

Returns the sign bits of the lanes as an integer, lane i giving bit i. All backends.

Returns whether every lane of a compares equal to the same lane of b with == (so +0 = -0, and a NaN is never equal). vec1d_same is available on AVX2 and NEON, the others on AVX2 only.

Bitwise operations on the 64-bit patterns of the lanes; vec4d_bit_andnot computes (~a) & b. All backends.

Operations on the lanes viewed as uint64_t bit patterns: a broadcast, addition and subtraction modulo 2^{64}, and logical shifts by 0 \le n < 64. All backends.

Returns \{a_0, b_0, a_2, b_2\} and \{a_1, b_1, a_3, b_3\}. All backends.

vec4d_permute_i_j_k_l returns \{a_i, a_j, a_k, a_l\}. Only these six permutations exist. All backends.

Viewing a and b as four 128-bit halves a_0a_1, a_2a_3, b_0b_1, b_2b_3 numbered 0 to 3, vec4d_permute2_i_j returns halves i and j: \{a_0, a_1, b_0, b_1\} and \{a_2, a_3, b_2, b_3\}. All backends.

An unpack followed by a permutation, fused where the backend allows: \{u_0, u_2, v_0, v_2\}, \{u_1, u_3, v_1, v_3\}, \{v_2, v_0, u_2, u_0\} and \{v_3, v_1, u_3, u_1\}. AVX2 and NEON.

The vec4n versions of the vec4d operations of the same names. AVX2 and NEON.

Returns \{a_3, a_2, a_1, a_0\}. AVX2 only.

Returns a vector with all lanes equal to a, or to zero. vec8n_set_n is available on AVX2 and NEON, the others on all backends.

Returns \{a_0, a_1, a_2, a_3\}. AVX2 and the generic backends; not NEON.

Returns a + b and a - b lane by lane, modulo 2^{\mathtt{FLINT\_BITS}}. All backends.

Bitwise and and or. vec8n_bit_and is available on AVX2 and NEON, the others on all backends.

Logical shifts of every lane by the same count 0 \le b < \mathtt{FLINT\_BITS}. vec8n_bit_shift_right is available on AVX2 only, the others on all backends.

Logical shift right by 32 bits. These are macros on AVX2 and functions on NEON.

Returns \{b_3, a_0, a_1, a_2\}: the lanes of a moved up by one, with the top lane of b entering at the bottom, as when passing a carry from one vector to the next. All backends.

Returns the 64-bit products of the low 32 bits of each lane, (u_i \bmod 2^{32})(v_i \bmod 2^{32}). AVX2 only.

Returns a_0 + a_1 + a_2 + a_3 modulo 2^{64}. AVX2 only.

Returns (a + b) \bmod n in [0, n) for a, b < n, with no condition on n. AVX2 and NEON.

The same for n < 2^{63}, which allows a cheaper test. AVX2 and NEON.

Converts between double lanes and ulong lanes holding integers in [0, 2^{52}), which are represented exactly in both. Outside this range the results are unspecified: most of these conversions work on the bit pattern of 2^{52}. AVX2 and NEON.

Returns an integer r \equiv ab (mulmod) or r \equiv -ab (nmulmod) modulo n, for integers a, b with |ab| < 2^{103} and 1 \le n < 2^{50}, given ninv = 1.0 / n. The result satisfies |r| \le n/2 + \eta|ab| with \eta = (1 + 2^{-53})^3 - 1 < 3.0001 \cdot 2^{-53}. In particular, if n < 2^{49} or fft_small_mulmod_satisfies_bounds(n) holds, then |ab| < 2n^2 gives |r| < n and |ab| < 4n^2 gives |r| < 3n/2. Larger products still give a correct representative, only a larger one. AVX2 and NEON.

Like mulmod and nmulmod, with the quotient rounded by round_mul. Requires |ab| < 2n^2 and n < 2^{50}. The result then satisfies |r| \le n/2 + \eta|ab| as for mulmod, and also |r| \le n/2 + 2.0001 \cdot 2^{-53}|ab| on the magic-constant path, so |r| < n if n < 2^{49} or fft_small_mulmod_satisfies_bounds(n) holds. With the magic constant, a product beyond 2^{51} n in absolute value gives a wrong result, possibly not an integer. With native rounding, and always on NEON, these functions are mulmod and nmulmod. AVX2 and NEON.

Returns ab rounded to an integer. With the magic constant (see FLINT_FFT_SMALL_ROUND_USE_NATIVE) this is the nearest integer to the exact product, with ties to even, and requires |ab| \le 2^{51}; a larger product gives a wrong value or a non-integer. With native rounding it is round(mul(a, b)), which rounds twice and has no size condition. AVX2 only.

Returns an integer r = a - qn \equiv a \pmod n, with q an approximation of a/n rounded to an integer, for an integer |a| < 2^{53} and an integer n \ge 4. The two functions are identical. The result satisfies |r| < n/2 + 2, hence |r| < n. On AVX2 the quotient is computed with round_mul, which with the magic constant needs |a| / n \le 2^{51}; this is why n \ge 4 is required. AVX2 and NEON.

Returns a \bmod n in [0, n) for an integer |a| < 2^{53} and an integer n \ge 4, by reduce_to_pm1no followed by reduce_pm1no_to_0n. On AVX2 with the magic constant, the vec4d and vec8d versions return n for the input -0.0. AVX2 and NEON.

Returns whether a_i \equiv b_i \pmod n in every lane, by comparing the results of reduce_to_0n, with the same conditions. vec1d_same_mod is available on AVX2 and NEON, vec4d_same_mod on AVX2 only.

Returns a \bmod n in [0, n) for a \in (-n, n), by adding n to negative lanes. The vec4d and vec8d versions on AVX2 test the sign bit, so they map -0.0 to n; the others return -0.0. AVX2 and NEON.

Returns a \bmod n in [0, n) for a \in [0, 2n). AVX2 only.

Returns a or a - n, whichever lies in (-n/2, n/2], for a \in [0, n]. vec1d_reduce_0n_to_pmhn is available on AVX2 and NEON, vec4d_reduce_0n_to_pmhn on AVX2 only.

Returns a, a - n or a + n, whichever lies in [-n/2, n/2], for |a| \le 3n/2 (in particular for a \in [-n, n]). AVX2 and NEON.

Sets C = AB, with the BLAS or the FLINT implementation according to flint_gemm_use_blas.

Sets C = AB with FLINT's own kernels, which are always available, handle any dimensions and leading dimensions without padding or alignment, and use up to flint_get_num_threads() threads of FLINT's thread pool.

Sets C = AB with cblas_sgemm or cblas_dgemm. Throws an exception if FLINT was built without BLAS.

Constructors and lane access, NEON only. vec4d_set_vec2d2 and vec8d_set_vec4d2 concatenate two halves, a giving the low lanes.

Arithmetic on two doubles, NEON only. vec2d_fmsub and vec2d_fnmsub are computed as negated fused operations, which the header notes may be slower.

Comparisons, masks and bit operations on two doubles, with the semantics of the vec4d versions. NEON only.

vec2d_unpacklo returns \{a_0, b_0\} and vec2d_unpackhi returns \{a_1, b_1\}; the vec8d versions act on each vec4d half. NEON only.

Modular arithmetic on two doubles, with native rounding and the semantics of the vec4d versions. NEON only.

Operations on one, two and eight words, with the semantics of the vec4n operations of the same names. NEON only.

Conversions for integers in [0, 2^{52}). Conversions to words round non-integers to nearest for vec2d and truncate for vec1d. NEON only.