The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
39 views

avoid nan and inf in floating point multiplication and division

I have the following hypothesis: It is better to reorder mult and div operation in such a way that we can do our best effort to avoid nan, inf and operating with subnormals. The reorder should follow ...
4
votes
1answer
140 views

Refactoring numerical code for TDD and encapsulation

I am coming to terms with TDD, and the fact that I need to re-factor some code that I'm (re)writing. I am having a problem that I think is a classic conflict---TDD vs. encapsulation of private ...
0
votes
1answer
166 views

How to do scalar multiplication and matrix inverse when variables are of size 1000 bits?

I am doing arithmetic operations on really huge numbers. For example, I am given six variables, a_{11}, a_{12}, a_{21}, a_{22}, x_1, and x_2. Although the above are math terms, these six variables ...
1
vote
1answer
418 views

How is this number calculated?

I have numbers; A == 0x20000000 B == 18 C == (B/10) D == 0x20000004 == (A + C) A and D are in hex, but I'm not sure what the assumed numeric bases of the others are (although I'd assume base 10 ...
8
votes
4answers
1k views

Is OCaml any good for numerical analysis? [closed]

I'm currently using C and some FORTRAN to solve numerically systems of differential equations. I'm a bit fed up with both of these languages but I need to have some (rather) efficient code... I'm ...