1. Amit De Picciotto
  2. PowerBuilder
  3. Monday, 16 September 2024 13:44 PM UTC

Dear all,

I have heavy use of datastore array within an NVO and since it's batch operation I try to release memory, with very limited 

It's something like that:

u_nvo_test uo_test
datastore lds_try//simple "external" data object with three fields and three rows of strings
long ll_ndx
uo_test = Create u_nvo_test

'
for ll_ndx = 1 to 3000
 uo_test.test[ll_ndx] = Create datastore
 uo_test.test[ll_ndx].SetTransObject(SQLCA)
 uo_test.test[ll_ndx].dataobject = 'd_try'

next
for ll_ndx = 1 to 3000
 uo_test.test[ll_ndx].reset()
 uo_test.test[ll_ndx].dataobject = ''

 destroy uo_test.test[ll_ndx]
next
destroy uo_test
GarbageCollect ( )

On the above script, the memory go up to approx. 130 MB, later, on the 2nd loop release about 30 MB

How can I release more memory? How can i avoid this memory leakage?

PB 2022 R2

Many Thanks!

 

Chris Pollach @Appeon Accepted Answer Pending Moderation
  1. Monday, 16 September 2024 13:53 PM UTC
  2. PowerBuilder
  3. # 1

Hi Amit;

  Have you tried a custom PB.INI with the following setting ...

[Data Window]

UseHwnd=no

Regards ... Chris 

Comment
There are no comments made yet.
mike S Accepted Answer Pending Moderation
  1. Monday, 16 September 2024 14:01 PM UTC
  2. PowerBuilder
  3. # 2

i was doing a similar process that used arrays of datastores.

  • fix 1 was to use garbagecollect.  runs out of memory easily.  If you are not actually running out of memory, then just keeping it this way is an option.
  • fix 2 was to create my own recycle nvo to create/destroy the datastores.  instead of destroying the arrays of datastores, it resets them and saves them in the nvo and then uses them again rather than create new ones.  this worked well (and was faster) however that only works if you are doing a process with large arrays in a loop.  easy to implement.  still ran out of memory on a very very large process it was running.  the nvo has 3 functions - recycle of a single datastore, recycle for an array of datastores (these reset the datastores and stores them on a heap), and a create that returns a datastore either off the datastore heap, or if none, creates a new datastore.
  • refactoring:   I restructured things into a nvo with just 2 datastores. the main datastore was basically my array of datastores - i had an extra field/column that tracked which virtual datastore it was (index of the array).  The 2nd datastore tracked each array and some meta information.    It is at least 10x faster than the nvo recycle, and uses a fraction of the memory.  lot more work to do as it ended up being pretty much being an in-memory database written in powerbuilder.

 

Comment
There are no comments made yet.
Amit De Picciotto Accepted Answer Pending Moderation
  1. Monday, 16 September 2024 14:19 PM UTC
  2. PowerBuilder
  3. # 3

Thanks Chris, for the answer.

Gave it a try with and without the key on the PB.INI looks pretty much the same:

Memory consumption: 

with INI key:
start: 25
Pick: 147.8
End: 115.2

Without key:
Start: 25.1
Pick: 147.9
End: 115.0

Comment
  1. mike S
  2. Monday, 16 September 2024 14:57 PM UTC
"UseHwnd=no" is to not use windows handles (which you can run out of) - nothing to do with memory. you should have that set anyway if you use a lot of datastores.
  1. Helpful
  1. Chris Pollach @Appeon
  2. Tuesday, 17 September 2024 11:45 AM UTC
Hi Mike ;

When allocated, Windows handles take a bit more memory as these O/S control blocks get allocated. They can also slow down DS performance quite a bit.

Regards ... Chris
  1. Helpful
  1. mike S
  2. Wednesday, 18 September 2024 14:49 PM UTC
its a dumb setting in any case - it should be no by default.



I know there are some reports of printing issues, but they seem rather specific enough for the datastore to be reset with a handle at runtime by pb automatically if they really are required (otherwise fix it).
  1. Helpful
There are no comments made yet.
John Fauss Accepted Answer Pending Moderation
  1. Monday, 16 September 2024 17:48 PM UTC
  2. PowerBuilder
  3. # 4

Hi, Amit -

If the "test" instance array of DataStore object references is an unbounded array, then I suggest you also try resetting that instance array with another unbounded (and unused) array of DataStore object references, then destroy the NVO, then perform GarbageCollect().

I would hope/expect nearly all of the memory utilized for the array would then be freed.

For example:

DataStore lds_empty_array[]

   .
   .
   .
   destroy uo_test.test[ll_ndx]
Next
uo_test.test = lds_empty_array
Destroy uo_test
GarbageCollect()

Best regards, John

Comment
There are no comments made yet.
Amit De Picciotto Accepted Answer Pending Moderation
  1. Tuesday, 17 September 2024 07:01 AM UTC
  2. PowerBuilder
  3. # 5

John,

Thanks for the reply, I tried that, assign an empty datastore array to the data store array on the NVO that just destroyed, but it did not released any memory:

for ll_ndx = 1 to 3000
 uo_test.test[ll_ndx].reset()
 uo_test.test[ll_ndx].dataobject = ''

 destroy uo_test.test[ll_ndx]
next
uo_test.test = lds_empty
destroy uo_test
GarbageCollect ( )

 

Actually all the lines after the Next, did not released any memory

 

I made additional test and tried not to use the loop...next to destroy the data store array, as well, assign the empty data store array did not helped, but i noticed that the GarbageCollect ( ) have similar effect on the memory, if i don't use the for next loop to destroy all datastores

 

I need to find the way how to release the memory, why power builder keep using so much memory after the Destroy?

 

Many thanks

Amit

Comment
  1. mike S
  2. Tuesday, 17 September 2024 13:20 PM UTC
PB does its own memory management. it won't release all the memory it no longer uses - it goes into a heap to be reused. the engineers could go into detail of how it work. The point is there is no magic setting to give you all that memory back.



if you are not running out of memory then do nothing. If you are (or think you might) then you need to use an alternative
  1. Helpful 3
  1. John Fauss
  2. Tuesday, 17 September 2024 15:10 PM UTC
Excellent point, Mike!

It's also important to recognize that PB will reclaim memory used for an object only when there exists no variables that reference the object, but that is likely not what is happening here in this simple example.
  1. Helpful
  1. Marc Wietscher
  2. Tuesday, 17 September 2024 21:14 PM UTC
Hi Amit,

you could try to call

Yield()

or even

DO WHILE Yield(); LOOP

after destroying the datastore at the end of the loop. That might give the internal garbage collector the opportunity to do its job.



Best regards,

Marc
  1. Helpful 1
There are no comments made yet.
Amit De Picciotto Accepted Answer Pending Moderation
  1. Wednesday, 18 September 2024 14:19 PM UTC
  2. PowerBuilder
  3. # 6

Dear all,

Many thanks for all the great ideas!

As mentioned, I checked all the options (I did not tried the brilliant idea of 'recycle NVO' as well the refactoring since it change my SW architecture, I just wanted to find rapid solution).

 

All the tries indicate that unfortunately,  PB clean only approx. 30%, it just stay on the heap, as explained.

 

BTW, I found that irritate and destroy all datastore on the array, is almost equal to call of GarbageCollect ( )

 

Once again, thanks all for the help, for now, i will implement the 30% release on memory.

Thanks!

Comment
There are no comments made yet.
  • Page :
  • 1


There are no replies made for this question yet.
However, you are not allowed to reply to this question.