Decimal values and math - Inconsistent results
Bigsquatch
Posts: 216
In working on my home system I have my OTA channels stored as DOUBLEs in an array. When performing operations on them to get the multicast part of the channel I am getting unreliable results.
When I run this code the ouput consistently produces the following results:
Notice the when testVar is 2.2, 6.3 or 8.3 the result is exactly as expected but in the rest of the cases it is not. This wouldn't surprise me too much if I was working with floating point values of varying precision but this kind of inconsistency with numbers with a single decimal place has me stymied.
Anyone care to test my code and confirm/debunk my findings?
I'm leaning now towards storing my presets in a structure with the channel and the multicast values stored separately as integers. That will certainly avoid this problem but in the mean time this issue is of interest to me regardless of how I solve the problem.
DEFINE VARIABLE
VOLATILE DOUBLE tv_channels[] = { 2.2, 6.2, 6.3, 8.2, 8.3, 11.2, 19.2, 23.2 }
DEFINE_EVENT
BUTTON_EVENT[dvUI,0]
{
PUSH:
{
STACK_VAR INTEGER x
STACK_VAR INTEGER wholeNum
STACK_VAR INTEGER decimalTimesTen
STACK_VAR DOUBLE testVar
STACK_VAR DOUBLE decimal
FOR(x=1; x <= 8; x++)
{
testVar = tv_channels[x]
wholeNum = TYPE_CAST(testVar)
decimal = TYPE_CAST(testvar - wholeNum)
decimalTimesTen = TYPE_CAST(decimal * 10)
SEND_STRING 0, "'testVar = ',FTOA(testVar)"
SEND_STRING 0, "'decimal = ',FTOA(decimal)"
SEND_STRING 0, "'decimalX10 = ',ITOA(decimalTimesTen)"
}
}
}
When I run this code the ouput consistently produces the following results:
Line 1 (16:14:45):: testVar = 2.2 Line 2 (16:14:45):: decimal = 0.2 Line 3 (16:14:45):: decimalX10 = 2 Line 4 (16:14:45):: testVar = 6.2 Line 5 (16:14:45):: decimal = 0.2 Line 6 (16:14:45):: decimalX10 = 1 Line 7 (16:14:45):: testVar = 6.3 Line 8 (16:14:45):: decimal = 0.3 Line 9 (16:14:45):: decimalX10 = 3 Line 10 (16:14:45):: testVar = 8.2 Line 11 (16:14:45):: decimal = 0.2 Line 12 (16:14:45):: decimalX10 = 1 Line 13 (16:14:45):: testVar = 8.3 Line 14 (16:14:45):: decimal = 0.3 Line 15 (16:14:45):: decimalX10 = 3 Line 16 (16:14:45):: testVar = 11.2 Line 17 (16:14:45):: decimal = 0.2 Line 18 (16:14:45):: decimalX10 = 1 Line 19 (16:14:45):: testVar = 19.2 Line 20 (16:14:45):: decimal = 0.200001 Line 21 (16:14:45):: decimalX10 = 2 Line 22 (16:14:45):: testVar = 23.2 Line 23 (16:14:45):: decimal = 0.200001 Line 24 (16:14:45):: decimalX10 = 2
Notice the when testVar is 2.2, 6.3 or 8.3 the result is exactly as expected but in the rest of the cases it is not. This wouldn't surprise me too much if I was working with floating point values of varying precision but this kind of inconsistency with numbers with a single decimal place has me stymied.
Anyone care to test my code and confirm/debunk my findings?
I'm leaning now towards storing my presets in a structure with the channel and the multicast values stored separately as integers. That will certainly avoid this problem but in the mean time this issue is of interest to me regardless of how I solve the problem.
0
Comments
DEFINE_EVENT //BUTTON_EVENT[vTEST,0] BUTTON_EVENT[vTEST,0] { PUSH: { STACK_VAR INTEGER x STACK_VAR INTEGER wholeNum STACK_VAR INTEGER testVar STACK_VAR INTEGER decimal FOR(x=1; x <= 8; x++) { testVar = TYPE_CAST(float_test_channels[x] * 10); wholeNum = testVar / 10; decimal = testVar % 10; SEND_STRING 0, "'testValue = ',FTOA(float_test_channels[x])" SEND_STRING 0, "'testVar = ',ITOA(testVar)" SEND_STRING 0, "'wholeNum = ',ITOA(wholeNum)" SEND_STRING 0, "'decimal = ',ITOA(decimal)" } } }decimalTimesTen = decimal * 10.0
instead of
decimalTimesTen = TYPE_CAST(decimal * 10)
Paul
For your channels as the 'decimal' component is really just a convient way to represent the two parts of the channel address your idea of using a structure with two individual integer values sounds like a great solution.
I doubt that is happening here. He isn't going to see relative errors with these values. Good link though, all programmers run into this issue eventually.
Paul
It will throw a compile error without the type cast. Keep in mind this is just code to see if I could get reliable results from floating point math.
Now I understand that there will be rounding errors when working with floating point numbers but I honestly expected numbers with a single decimal value to be usable when doing mathmatical operations on them.
The point of this post was to find out if floating point math in Netlinx is really that unusable. And, if so, to also give a heads up to anyone who hasn't been down that road from wasting their time trying to make it work.
Yes I see that you called the variable decimalTimesTen but defined it as an integer so that was confusing. Try declaring decimalTimesTen as a double as well and it will compile. You can cast it to an integer right before printing if needed or use ftoa instead.
They are perfectly usable, you just have to be careful when converting from integers to floats/doubles and know how casting can result in data loss.
Its certainly not a waste of time to understand floating point numbers and Netlinx works fine with them. But in the case of TV stations, I always use strings since there is no reason to use numbers.
Paul
More than likely, you'll need to wind up:
1) converting each digit into an ascii representation or some other kind of value to send to the tuner / display via serial
2) pulsing each digit individually in an IR sequence
Doing a simple atoi() on each character is by far easier (though maybe more expensive) than some round about mathematical equation, but then again - maybe not!
I like this way of dealing with it if the values can be changed at runtime from a touchpanel. A structure with a major and minor component of the channel number will simplify things quite a bit.
The other option is to store all of the values as strings.
Paul
If you need something like a major and minor number to pulse two IR channels, why not use the XCH command?
/me is probably missing something
I've stored and retrieved favorites different ways in dozens of different projects at my previous job doing residential AMX. Not once did I use floats and since this was my own system to do with as I please I thought "why not?" My thinking was that floats are the data type that most closely represents the data being stored. Now I have my answer.