Your personal information you provide will be transfered and stored as encrypted data.
You have the ability to update and remove your personal information.
You consent to our cookies if you continue to use this website.
Allow cookies for
Necessary Cookies Necessary Cookies cannot be unchecked, because they are necessary for our website to function properly. They store your language, currency, shopping cart and login credentials.
Analytics Cookies We use google.com analytics and bing.com to monitor site usage and page statistics to help us improve our website. You may turn this on or off using the tick boxes above.
Marketing Cookies Marketing Cookies do track personal data. Google and Bing monitor your page views and purchases for use in advertising and re-marketing on other websites. You may turn this on or off using the tick boxes above.
Social Cookies These 3rd Party Cookies do track personal data. This allows Facebook, Twitter and Pinterest integration. eg. shows the Facebook 'LIKE' button. They will however be able to view what you do on our website. You may turn this on or off using the tick boxes above.
I was trying to read numeric value from an excel file.I used the function "cell.getNumericCellValue()" to read the column. Some times the value is fetched correctly and some times it returns a high precision value( ie, some times the value 118.474 is fetched as 118.4739999999), while I require the exact value. The format type of the column in the excel is "custom".
Can any one please help me on this?
vain-ego pkBRONZE Member Lambretta Fanatic 4,997 posts Location: United Kingdom
Posted: 118.4739999999 ...considering excel uses better calculations than myself I'd say that figure was most likely to be correct when 118.474 may look logically correct.
I'm no mathematician nor excel user.
Deamon_LlamaBRONZE Member member 78 posts Location: Melbourne, Australia
Posted: I can show mathematically that 118.4739999999 is actually equal to 118.474, but I don't think that will be particularly useful, sorry
--- No statements are true unless they can be proven scientifically. ---
YakumoSILVER Member veteran 1,237 posts Location: Oxfordshire, United Kingdom
Posted: are you always reading cell.getNumericCellValue into a variable with a datatype that can hold the full value? and are all the fields your reading from definitely formatted correctly?
same problem is discussed here though you havn't actually said what language you're programming/scripting in...
Blinded by Hyperlights, please donate generously
Mr ChutneySILVER Member Tosser 1,712 posts Location: Bristol,UK
Posted: Nor indeed why you are posting this on a poi forum?!