Page 2 of 3
Re: About value inaccuracy
Posted: Thu May 12, 2016 12:57 am
by martinvicanek
Oh, I see, the manual says it can only store floats, float arrays or strings. Hm, you could use strings to encode integers. Or use floats that have an exact machine representation so you can convert them to integers. If you can get away with integers in a range 0...127 (as your example seems to indicate) it should be doable.
Re: About value inaccuracy
Posted: Thu May 12, 2016 2:37 am
by Tronic
example:
@in_float.round(8)
output 100.10212/127.0
(@ins[0].round(8)*127.0).round(5) # >> 100.10212
Re: About value inaccuracy
Posted: Thu May 12, 2016 8:00 am
by martinvicanek
Divide by 128 instead?
Re: About value inaccuracy
Posted: Thu May 12, 2016 11:47 am
by tulamide
Hey Martin,
yes, 128 was one of the "low_bit_count"-dividers I used and it worked like a charme. But, as I said, for 128 values that results in a range of 0.0078125 - 1.0.
These normalized float values are used later on, and they need to be in the range 0.0-1.0, and using another normalizing brings up the "too-many-bits-for-single-precision" issue again.
Hey tronic,
I'm not sure if it will work, but I will give it a try!
EDIT: No, it doesn't work. It just leads to certain integer values not recognized at all. They are just left out. For example 32, 47, 49, 58, etc.
Re: About value inaccuracy
Posted: Thu May 12, 2016 3:09 pm
by KG_is_back
This method allows you to store up to 29bit unsigned integer as a float value in 0-1 range. Values lower than 2^24 will be stored as denormals, so it is a thing to consider. It may not be optimal solution, but it works...
In case you need to store more tan 29bits you have to split them into 29bit segments.
Code: Select all
def i_to_f01(n)
a=(n&(2**29-1)).to_s(2)
a.insert(-24,"0")
watch "bin",a+" "+a.length.to_s
a=a.to_i(2)
[a].pack("L").unpack("f")[0]
end
def f01_to_i(n)
a=[n].pack("f").unpack("L")[0]
a=a.to_s(2)
a[-24]=""
a.to_i(2)
end
EDIT: the highest value actually isn't 1 but 0.9999999403953552
Re: About value inaccuracy
Posted: Thu May 12, 2016 4:54 pm
by Tronic
tulamide wrote:Hey tronic,
I'm not sure if it will work, but I will give it a try!
EDIT: No, it doesn't work. It just leads to certain integer values not recognized at all. They are just left out. For example 32, 47, 49, 58, etc.
my solution not work for you?
Re: About value inaccuracy
Posted: Thu May 12, 2016 9:35 pm
by tulamide
@Tronic
Unfortunately it didn't work. As I described, some integers are simply not calculated.
@KG
That's a very interesting bitpacker. I will see if I can implement it.
@all
Many thanks for all your ideas so far! Don't hesitate to post if you came up with another!
Re: About value inaccuracy
Posted: Thu May 12, 2016 10:06 pm
by Tronic
tulamide wrote:@Tronic
Unfortunately it didn't work. As I described, some integers are simply not calculated.
mmm
Please, can you show me an not working example?
EDIT: OK, got it, not need anymore, sorry.
Re: About value inaccuracy
Posted: Fri May 13, 2016 8:47 am
by TheOm
You can just directly store an unsigned integer in the range [0, 1065353216] as a float and it will be between 0.0 and 1.0.
Conveniently, this is also the maximum possible number of values that you can store in a single precision float from 0.0 to 1.0 ((2^7 - 1) * 2^23 + 1).
Here is a ruby implementation for signed(range [-532676608, 532676608]) and unsigned ints:
I have also tested the correctness of this approach for all possible values.
Re: About value inaccuracy
Posted: Fri May 13, 2016 5:57 pm
by tulamide
I decided to go with the solution provided by KG and TheOm. It does not fulfill all criteria (the range 0.0-1.0 is fixed to 0-1065353216, but I need a smaller integer range), but at least the integers will keep intact and can be saved via preset, and calculating a new float range from integers is not too cpu heavy and can be done on request.
Thanks for all the help! I hope this also showed the issue and that it is more complex than thought.