precision of double data

Jul 23, 2013 at 8:55 PM
Hi Keith,

it would be very kind if you can comment on this issue that i observe. say i have in the excel cell numbe like .34567. When i read the number in i observe in the c++ debugger the number has changed to .34566999999999999. This i guess is a precision issue between excell and c++ but is there some flag which can be switched on/off to automatically handle the issue(which is read the data as is maybe padding with zeros at the end).

Jul 23, 2013 at 9:01 PM
This is due to how floating point numbers are represented. See for the gory details.
In particular