Open Source and Traditional Technical Computing

Transcription

1 Open Source and Traditional Technical Computing Alan Edelman Massachusetts Institute of Technology Scilabtec10 June 16, 2010

2 How should octave Compete/cooperate with

3 The 800 pound gorilla Hard to beat that first date experience Great numerical analyst Cleve Moler People think MATLAB s answer is right, when it s different Incredible documentation Years and years of experience Tons of functionality, but it s not the functionality

4 vs. the power of download, next, agree, next, next, next, install

5 Critical: trust in the numerics

6 but: Typical user: In grade school, math was either right or wrong. Getting it wrong meant: bad boy/bad girl. (Very shameful!) By extension, if a computer gets it wrong Shame, Bad computer! Bad programmer! Most of the world s users don t understand ill-conditioning, good/bad relative accuracy, forward/backward error. My personal hypothesis regarding the Pentium Bug of 1993: Bad division might have lead somebody to make money and somebody to lose money, but the affected parties were blissfully unaware!

7 Time feels right to harness Kahan

8 But none of these Tons of Numerical Analysis Classes Research from Classical & IEEE numerical analysis Algorithmic design Function Encyclopedias and Software Experience of libraries and computing environments LAPACK, MATLAB, Mathematica, Maple scilab, python, R Teach a young library writer what to do

9 Misconceptions There is a right and a wrong. In fact there are (poorly understood) tradeoffs. Computers are highly consistent: Programs are remarkably idiosynchratic

10 Back to Open source With open source, we can see and critique the algorithms. With private code we can only test

11 Math: Det(matrix of integers)=integer

12 a=sign(randn(27)),det(a) a = But is it right? ans =

13 We like sqrt(9)=+3 (not -3) What about acos(3)??? Python >>> from cmath import acos >>> from scipy import arccos >>> acos(3.0) j >>> arccos(3.0) j

14 Speaking of acos acos( ) (cmath)

15 Continuity is a function property, octave:> atan(1+i*2^64*[1 1+eps])) ans = Sloppy? not seen at a point Okay? tan(x+pi) is tan(x) after all so is it okay to return atan(x) + pi * any integer Principle: Continuity is required even if special handling is required on branch cuts, stratifications, etc. What about QR? Some people argue it s different, it s okay to just produce a Q and R such that QR is nearly A Nonsense, continuity is required!

16 Numerical Accuracy Statistics Function: erfinv (erfinv(erf)= Identity function ) >> erfinv([ ])*sqrt(2) ans = >> erfinv(1-eps) Exact: MATLAB: erfinv(1-eps - eps/2) is exactly and erfinv(1-eps + eps/2) is exactly condition number is O(1e14) Plain old inv : (inv(a) composed with A= Identity ) >> a=0; inv(a) Warning: Matrix is close to singular or badly scaled..

17 Possible Conclusions? Bad Algorithm: should have had a small forward error Bad MATLAB : should have warned us Good backward error so we need not worry? The user was already in extremely bad shape or extremely good shape anyway: Already been hit by a truck. (Think eigenvalues when pseudos!) Somehow won t matter. (Think preconditioning!) Problem is too ridiculous (but this attitude always comes back to haunt you )

18 Speaking of QR

19 Speaking of QR A=ones(4,4)

20 Continuity is a function property, not seen at a point: QR No different from atan Good enough to take atan+pi*(random integer?) Is it okay to take Q*diag(random(signs?))? There is an argument to avoid gratuitous discontinuity! Just like atan Many people take [Q,R]=qr(randn(n)) to get uniformly distributed orthogonal matrices! But broken because Q is only piecewise continuous. Backward error analysis has a subtlety. We guarantee that [Q,R] is a QR decomposition of a nearby matrix, not the QR decomposition of a nearby matrix. Even without roundoff, there may be no algorithm specified that gives THIS QR for the rounded output.

21 Embeddings log2(2^k)? log10(10^k)? sin(k*pi)? (double)^(integer)? Integers are embedded in reals Reals embedded in complexes 2d arrays embedded in nd arrays

22 Opinion I like the embedding concept But the answer should be right, continuous, and monotonic without side affects Terrible embedding: sort

23 >> x=[-1 1 2] x = >> sort(x) ans = >> x=[-1 1 2*i] x = i >> sort(x) ans = i >> 1 < (2*i) ans = 0 >> 1 > (2*i) ans = 1 >> max(1,i) ans = 1 >> min(1,i) ans = 1 >> max(i,1) ans = i >> min(i,1) ans = i Sorting >> x=[1-1 i]'; y=[1;-1;-i]; [x y x-y] ans = i i 0 >> [sort(x) sort(y)] ans = i i >> a=[ ]; roots(poly(a))' ans =

24 Friction First Element

25 Accuracy: Sine Function Extreme Argument sin(2^64)= One IEEE double ulp higher: sin(2^ )= over 651 wavelengths higher. 2^64 is a good test case Maple 10 evalhf(sin(2^64)); note evalf(sin(2^64)) does not accurately use 2^64 but rather computes evalf(sin(evalf(2^64,10))) Mathematica 6.0 Print[N[Sin[2^64]]]; Print[N[Sin[2^64],20]]; * MATLAB sin(2^64) sin(single(2^64)) Octave : sin(2^64) * python 2.5 numpy sin(2**64) ** R format(sin(2^64),digits=17) ** Scilab format(25);sin(2^64) *Mathematica and python have been observed to give the correct answer on certain 64 bit linux machines. **The python and R above numbers listed here were taken with Windows XP. The.3128 number was seen with python on a 32 bit linux machine. The correct answer was seen with R on a sun4 machine. It s very likely that default N in MMA, Octave, Python, and R pass thru to various libraries, such as LIBM, and MSVCRT.DLL. Notes: Maple and Mathematica use both hardware floating point and custom vpa MATLAB discloses use of FDLIBM. MATLAB and extra precision MAPLE and Mathematica can claim small forward error All others can claim small backward error Excel doesn t compute 2^64 accurately and sin is an error. Google gives Ruby on gives for Math.sin(2**64). On a sun4 the correct was given with Ruby. Relative Backward Error <= (2pi/2^64) = 3e-19 Relative Condition Number = abs(cos(2^64)dx/sin(2^64))/(dx/2^64) = 8e20 One can see accuracy drainage going back to around 2^21 or so, by powers of 2

26 Edge Case: Sine Function at Infinity Maple: sin(infinity); undefined evalf(sin(infinity)); Float(undefined) Mathematica: Sin[Infinity] Interval[{-1., 1.}] MATLAB: sin(inf) NaN Octave: Numpy: -1.#IND R: sin(inf) NaN (and a warning) Scilab: sin(%inf) Nan

27 Coverage: Erf Function Complex Support python/scipy is the only numerical package good Maple: evalf(erf(i)); I Mathematica: N[Erf[I]] I MATLAB: erf(i)??? Input must be real. Octave: error: erf: unable to handle complex arguments Python/Scipy: erf(1j) j R: pnorm(1i) Error in pnorm Non-numeric argument Scilab: erf(%i)!--error 52 argument must be a real

28 Scorecard Approach (sine)

29 Unique Opportunity Raise the bar for all functions: Linear Algebra! Elementary Functions Special Functions Hard Functions: Optimization/Diff Eqs Combinatorial and Set Functions What about 0-d, 1-d, n-d arrays, empty arrays Inf, Not Available (NaN) Types (integer, double, complex) and embeddings (real inside complex, matrices insidse n-d arrays) Methodology Work in Progress: Create a web experience that guides and educates users in the amount of time that it takes to say Wikipedia Futuristic?: The Semantic Web and Other Automations?

30 Computing Environments: what s appropriate? Good numerics? Who defines good? How do we evaluate? Seamless portability? Rosetta stone? Consumer Reports? Easy to learn? Good documentation? Proprietary or Free? Download, next, next, next, enter, and compute?

31 Star-P approach was a good start but needs more flexibility

32 Modern parallel computing needs to know so much

33 Developers want to do fancy things, but have high level functionality available

34 I expect original contributions to performance and parallelism in VHLL s (Very high level languages) well beyond the plug in and message passing models we ve had to date just don t know when