BASIC INT rounding issue
Rick Murray (539) 13840 posts |
I believe it is VisualBasic. Therefore, one would need to see how the variables were declared to know what type they are supposed to be. There’s no “As Integer” or % suffix, for example.
I’m at work so haven’t tested, but how about I don’t know if VB supports zero having a sign. Given that it was lacking a basic binary shift, I’d be surprised if it did…but again it may depend upon the underlying type (as in “maybe if it is FP”).
Ah, but the question is, does -0 equal 0? Given there’s no explicit input and output type in the function definition, I think VB normally defaults to variant (which is sort of the same as “whatever”). Which, of course, might not have been the correct behaviour either. All just goes to show, something that seems simple can be…complicated. That example code, however, is still abysmal. |
Graeme (8815) 106 posts |
Absolutely. The original post in this discussions shows some of the complications of floating point. Have a go at this program in BASIC: x = 9000000000 : REM 9 zeros y = x+1 IF x=y THEN PRINT "Equal" ELSE Print "NotEqual" Look at the code. Obviously x does not equal y (which is x+1) and so the text “NotEqual” will be printed. And against your expectations, standard BASIC prints “Equal”. It is not a bug. It is a quirk of floating point. |
Clive Semmens (2335) 3276 posts |
Quite instructive to add:
PRINT y Might be worth trying it with various different settings of @%. Supplementary question: how many zeroes does it take to cause this effect in BASIC? How many in BASIC64? I wouldn’t even call it a quirk: I’d call it a pretty obvious consequence of finite precision arithmetic. At this precision, 1/9000000000 = 0 … Try this:
|