The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
2answers
2k views

Can anyone explain representation of float in memory?

This is not duplicate question as I read the previous question. Can anyone help me in understanding how float values are stored in the memory . My doubt is here float values contain '.' (for example ...
6
votes
2answers
982 views

Solutions for floating point rounding errors

In building an application that deals with a lot of mathematical calculations, I have encountered the problem that certain numbers cause rounding errors. While I understand that floating point is not ...
5
votes
2answers
1k views

Performance and other issues with using floating point types in C++

Being interested in C++ performance programming there is one aspect I really have no clue about- and that is the implications of using floating point calculations vs doubles vs normal integer ...
3
votes
1answer
654 views

Dummy float values to force floating-point operations

It is common to use idioms such as: x/60.0 to force a floating-point division when x is an integer in languages which do not have distinct operators for integer and decimal division. Is this an ...
-2
votes
1answer
2k views

in Objective-C, value of type double/float can only be NAN, INFINITY, & normal number?

I knew double or float value can be not only normal value(-1.3, 0, 1.0, 2.3) but also NAN and INFINITY in Objective-C. Is there other special values except for NAN and INFINITY for double/float value ...
6
votes
2answers
951 views

How to remove the boundary effects arising due to zero padding in scipy/numpy fft?

I have made a python code to smoothen a given signal using the Weierstrass transform, which is basically the convolution of a normalised gaussian with a signal. The code is as follows: #Importing ...
40
votes
12answers
3k views

Is there an alternative to bits?

Is there an alternative to bits as the smallest unit of data? Something that won't be only 0 or 1, but actually hold many possible states in between? Wouldn't it be more natural to store floats like ...
0
votes
1answer
495 views

Conversion of a number from Single precision floating point representation to a Half precision floating point [closed]

I have a code where I have to work on Half precision floating point representation numbers. To achieve that I have created my own C++ class fp16 with all operators(arithmetic logical, relational) ...
27
votes
6answers
10k views

What causes floating point rounding errors?

I am aware that floating point arithmetic has precision problems. I usually overcome them by switching to a fixed decimal representation of the number, or simply by neglecting the error. However, I ...
26
votes
12answers
3k views

Why do you need float/double?

I was watching http://www.joelonsoftware.com/items/2011/06/27.html and laughed at Jon Skeet joke about 0.3 not being 0.3. I personally never had problems with floats/decimals/doubles but then I ...