Subscribe via RSS
Toggle Submenu
  1. Categories
  2. Recent
  3. Tags
View Replies (1)

Resolved does anyone know the limit for a numeric computed column?

  1. How-to
0
Votes
Undo
  1. Miguel Leeuwe
    Miguel Leeuwe
  2. PowerBuilder
  3. Friday, 11 June 2021 14:52 PM UTC

I'm having to calculate the initial and end coordinates to draw a trend line (linear regression line).

To do so, I put the values of my points in a graph in a datastore and one of the computed columns is X2. 

The only problem I'm having, is that my X column is a long value. X2 would be X * X and I'm getting an overflow.

Does anyone know what the maximum numeric datatype is for a computed column?

regards

 

 

SDI? Behavior when run from IDE vs from the RUN co...
Menu Item Event
Accepted Answer
John Fauss
John Fauss Accepted Answer Pending Moderation
  1. Friday, 11 June 2021 15:22 PM UTC
  2. PowerBuilder
  3. # Permalink

Hi, Miguel -

What value for "X" are you getting an overflow error with? In a quick test, I entered the 10-digit value 2000000000 and a computed field that squares the value works OK.

Have you tried to change the column's datatype in the DW to number instead of long?

Comment
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Friday, 11 June 2021 15:55 PM UTC
Hi John,

I've changed the value to decimal(0) and now it works ok.

I was having a long value before and I was getting -241......

It seems that the computed column bases it's datatype on the values you use within its expression, which seems logical in a way.

Thanks!

  1. Helpful
John Fauss
  1. John Fauss
  2. Friday, 11 June 2021 16:14 PM UTC
Super! I'm glad to hear changing the column's datatype worked for you. Have a great weekend!
  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Friday, 11 June 2021 16:24 PM UTC
I will (if England looses from Croatia), same to you!
  1. Helpful
There are no comments made yet.
Responses (1)


There are replies in this question but you are not allowed to view the replies from this question.
We use cookies which are necessary for the proper functioning of our websites. We also use cookies to analyze our traffic, improve your experience and provide social media features. If you continue to use this site, you consent to our use of cookies.