steph_tsf wrote:Can you please describe a few juicy cases, or symptomatic cases, as I fear that it'll take some time before I fully understand how IEEE 754 floating point is dealing with denorm, nan, nil, invalid, etc.
Phew - a big subject indeed! So firstly, a couple of useful links...
For a gentle introduction to floating point representation,
this website is a good place to start, with links to more advanced concepts for those who need them. For exhaustive (and possibly exhausting!) detail, including mathematical proofs, I suggest
this extremely comprehensive article.
"Nil" we can disregard. It is an unrelated concept which, in the context of FS, only applies to Ruby (it has no fixed meaning; "unassigned variable", "no search result found", "data is unchanged", "feature disabled", etc. as indicated by documentation). As a brief summary of the others...
- Infinities are designed to work as closely as possible to mathematical theory - they are
not a kind of error condition, as some people seem to treat them. If an operation on infinity is theoretically defined then, within the bounds of precision, that will be the result; e.g. the comparison
(finite < infinity) will return true, and
(nonzero_finite / infinity) will return zero (there is no representation for infinitesimals).
- NaN ("Not a Number"; or sometimes '#IND' = "indeterminate") means that there is no theoretically defined result for the operation (assuming only real numbers, of course). NaNs propagate - that is, any operation on a NaN will always return another NaN. The idea is that testing for NaN after every single operation would waste CPU resources; propagation ensures that we can rely on being able to test for NaN only at the end of a sequence of operations. The obvious worst case is that a NaN within a feedback loop will propagate forever unless trapped.
- Denormal numbers are a direct consequence of the floating-point representation. Floats are stored as
(value * 2^exponent). By shifting the exponent, 'value' can be made to always have a most-significant bit that is one; so an extra bit of precision at the least-significant end is made available by assuming MSB = 1 and not storing it - this is "normalisation". However, this would limit the smallest number that can be represented when the exponent reaches the lowest available value. In this case the MSB = 1 restriction is relaxed, at the cost of greater CPU power being required to perform calculations on the "denormal". The lowest "normal" exponent is -126 for 32-bit floats, so these values are extremely small!
Denormals are such small numbers that in many cases we do not need to worry too much about them - for example; even a "professional" 24-bit soundcard cannot possibly give us such an input value. However, any process with exponential decay might easily produce them, especially if an input is not always present. For example; if the "send" to an echo effect with feedback or an IIR filter is turned off, many people are rather surprised to find that "no signal" suddenly consumes vastly more CPU power. Internally, there is a "signal", of course, but denormals are so small that no audio meter will show them, and they are most certainly not audible!
There are a couple of ways to deal with denormals. Firstly, we could test whether a number is within that range and truncate it to zero. Alternatively, a very small amount of noise could be added to the signal to ensure that following calculations rarely enter the denormal range (or the noise floor of analogue input components might provide this free-of-charge!) A variation on the second method is to add and then subtract a small DC offset, to push the exponent beyond the denormal range and deliberately force rounding - this is often the most economical method, and can be seen in a few of the modules you've used in your filter/compressor schemes.
