Fast sigmoid algorithm
An answer to this question on Stack Overflow.
Question
The sigmoid function is defined as
S(t) = 1 / (1 + e^(-t))
(where ^ is pow)
I found that using the C built-in function exp() to calculate the value of f(x) is slow. Is there any faster algorithm to calculate the value of f(x)?
Answer
This question has significantly more detail including these benchmarked results:
name rms_error maxdiff time_us speedup samples
logistic_with_tanh 5.9496e-02 1.5014e-01 0.0393 0.5076 200000001
logistic_with_atan 3.9051e-02 9.6934e-02 0.0321 0.6211 200000001
logistic_with_erf 6.5068e-02 1.6581e-01 0.0299 0.6676 200000001
logistic_fexp_quintic_approx 1.2921e-07 5.9050e-07 0.0246 0.8118 200000001
logistic_product_approx_float128 8.7032e-04 1.7217e-03 0.0209 0.9523 200000001
logistic_with_exp_no_overflow 4.7660e-17 1.6653e-16 0.0198 1.0084 200000001
logistic_product_approx128 8.7032e-04 1.7211e-03 0.0164 1.2187 200000001
log_w_approx_exp_no_overflow128 8.7193e-04 1.7211e-03 0.0158 1.2640 200000001
logistic_with_sqrt 8.3414e-02 1.1086e-01 0.0146 1.3662 200000001
log_w_approx_exp_no_overflow16 6.9726e-03 1.4074e-02 0.0141 1.4114 200000001
log_w_approx_exp_no_overflow16_clamped 6.9726e-03 1.4074e-02 0.0141 1.4153 200000001
logistic_schraudolph_approx 1.5661e-03 8.9906e-03 0.0138 1.4497 200000001
logistic_with_abs 6.0968e-02 8.2289e-02 0.0134 1.4936 200000001
logistic_orig 0.0000e+00 0.0000e+00 0.0199 ------ 200000001