1. John Fauss
  2. PowerBuilder
  3. Thursday, 26 December 2019 23:00 PM UTC

In addition to the well-understood data types of Int, Long, Decimal(n), Real and ULong, the DataWindow column object supports a column type of "Number". What is the range of values a column of type Number can hold? What is the precision (maximum number of decimal places) of a DataWindow column of type Number? Please note I am not referring to the range of values of a PowerScript variable...those are well documented.

John Fauss

Michael Kramer Accepted Answer Pending Moderation
  1. Friday, 27 December 2019 03:23 AM UTC
  2. PowerBuilder
  3. # 1

Hi John, My experience is that NUMBER in DataWindow buffer behaves like DOUBLE (15-16 digits precision). It may differ but I highly doubt there is much difference. This has consequences >>>

  • Number datatype in DataWindow buffer loses precision at approx 15 digits because it handles Double.
  • Precision of Double = ~ 52 bits = 15-16 digits.
  • Max LongLong value = 63 bits = 19 digits.
  • Max precision of Decimal28 digits.

I recommend:

  1. Use decimal when data is "decimal" in nature (ex: US or EU GDP)
  2. Use decimal when data is integer value of 48+ bits.
  3. PowerScript :: Copy LongLong value to Decimal{0} variable before doing arithmetic.
    I experienced PowerScript implicitly promote LongLong value to Double despite what PB docs state.

HTH /Michael

 

Comment
  1. John Fauss
  2. Saturday, 28 December 2019 00:49 AM UTC
As usual, you are a treasure trove of information and recommendations, Michael. Many thanks for sharing!
  1. Helpful
There are no comments made yet.
Chris Pollach @Appeon Accepted Answer Pending Moderation
  1. Friday, 27 December 2019 00:36 AM UTC
  2. PowerBuilder
  3. # 2

Hi John;

  The DW "number" data type can be either Decimal, Double, Integer, Long or Real. The numeric range for each of those is documented under the "Standard" data types in the PB help.

Regards ... Chris

Comment
  1. John Fauss
  2. Saturday, 28 December 2019 00:46 AM UTC
Thanks for the information, Chris!
  1. Helpful
There are no comments made yet.
  • Page :
  • 1


There are no replies made for this question yet.
However, you are not allowed to reply to this question.