1. Miguel Leeuwe
  2. PowerBuilder
  3. Friday, 11 June 2021 14:52 PM UTC

I'm having to calculate the initial and end coordinates to draw a trend line (linear regression line).

To do so, I put the values of my points in a graph in a datastore and one of the computed columns is X2. 

The only problem I'm having, is that my X column is a long value. X2 would be X * X and I'm getting an overflow.

Does anyone know what the maximum numeric datatype is for a computed column?

regards

 

 

Accepted Answer
John Fauss Accepted Answer Pending Moderation
  1. Friday, 11 June 2021 15:22 PM UTC
  2. PowerBuilder
  3. # Permalink

Hi, Miguel -

What value for "X" are you getting an overflow error with? In a quick test, I entered the 10-digit value 2000000000 and a computed field that squares the value works OK.

Have you tried to change the column's datatype in the DW to number instead of long?

Comment
  1. Miguel Leeuwe
  2. Friday, 11 June 2021 15:55 PM UTC
Hi John,

I've changed the value to decimal(0) and now it works ok.

I was having a long value before and I was getting -241......

It seems that the computed column bases it's datatype on the values you use within its expression, which seems logical in a way.

Thanks!

  1. Helpful
  1. John Fauss
  2. Friday, 11 June 2021 16:14 PM UTC
Super! I'm glad to hear changing the column's datatype worked for you. Have a great weekend!
  1. Helpful
  1. Miguel Leeuwe
  2. Friday, 11 June 2021 16:24 PM UTC
I will (if England looses from Croatia), same to you!
  1. Helpful
There are no comments made yet.


There are replies in this question but you are not allowed to view the replies from this question.