Subscribe via RSS
Toggle Submenu
  1. Categories
  2. Recent
  3. Tags
View Replies (1)

Resolved How Decimal works...

  1. How-to
0
Votes
Undo
  1. CJ Lai
    CJ Lai
  2. PowerBuilder
  3. Monday, 21 March 2022 21:29 PM UTC

Hi all

I came across a script at work and am a bit confused by how the data type of Decimal works.

Here is what I understand

- If a variable, say ldec_test, is declared as Decimal (7, 2), it would have a value like 12345.67 (for example).  Is that correct?

- If the variable is declared as Decimal(2), it would have a value format like ######.12 (only 2-digit decimal point). Is that correct?

- When the variable (declared as decimal(2)) has a value $123.456 assigned, would the final value be $123.45, or $123.46 by default? Does Decimal truncate or round it to the nearest penny?

 

I hope my question makes sense.

Thanks

Powerbuilder CPU usage is very high
RestClient error -16
Accepted Answer
John Fauss
John Fauss Accepted Answer Pending Moderation
  1. Monday, 21 March 2022 22:03 PM UTC
  2. PowerBuilder
  3. # Permalink

Hi, CJ -

First, you cannot declare a PB Decimal variable like Decimal{7,2}. Only a single numeric, the number of decimal places, is permitted.

Second, if you include the optional number of decimal places in the declaration of the variable, that's how PB keeps track of its value. If you assign a variable declared as Decimal{2} with the value 123.456, the value that get assigned is 123.46 (it rounds). You can easily test this, inspect the value using the PB Debugger, convert it to a string value without including the format, such as 

Decimal{2} ldec_money = 123.456
MessageBox("Try It!",String(ldec_money))

And the message will be displayed as 123.46.

HTH, John

Comment
Load more comments
Matt Balent
  1. Matt Balent
  2. Monday, 28 March 2022 18:38 PM UTC
Using dec{0} can be useful in situations where a long just isn't big enough (i.e., large key value for db tables).
  1. Helpful 1
Olan Knight
  1. Olan Knight
  2. Monday, 28 March 2022 23:19 PM UTC
And of course we now have LongLong data types for those pesky types of issues.

  1. Helpful
John Fauss
  1. John Fauss
  2. Tuesday, 29 March 2022 01:11 AM UTC
True, but a LongLong variable can store only 18 digits faithfully (almost a full 19 digits), which, while significantly more than what a Long or ULong can hold, it is not always sufficient. Even more critically, a recent, different post raised the possibility that LongLong arithmetic may not be reliable:

https://community.appeon.com/index.php/qna/q-a/calculation-errors-with-longlong-datatype
  1. Helpful
There are no comments made yet.
Responses (1)


There are replies in this question but you are not allowed to view the replies from this question.