1. Steen Jakobsen
  2. PowerBuilder
  3. Monday, 27 November 2017

Hi all,

 

The code below is eating up memory. (This example is eating 50 MB in 3 minutes)

GarbageGollect() does not work

 

The only 2 solutions are:

1. destroy and create datastore

2. MyDataStore.DataObject = "some dw"     and then  myDataStore.SetTransObject(MyTransaction)

But this is just not possible in our application.

What can I do ? (I'm in deep trouble)

Thanks in advance 

// Steen

-----------------------------

DataStore lds
lds = Create DataStore
lds.DataObject = "d_abc"
lds.SetTransobject(SQLCA)
 
int li_x
long ll_f
For li_x = 1 To 500
    lds.Retrieve('some id')
    For ll_f = 1 To lds.RowCount()  // 700 rows
         lds.SetItem(ll_f,"mycolname",'some data') // memory leak here!!
    Next
Next
MessageBox("","before destroy")
Destroy lds
 
// memory released
 
----------------------------
Who is viewing this page
Armeen Mazda Accepted Answer Pending Moderation
2
Votes
Undo

Guys,

There are reasonable workarounds and arguably better ways of writing the code.  PowerBuilder product is composed of many millions of lines of code that we didn't write and have to be very careful how we change so our customer's mission critical apps don't break when they upgrade, many of which are Fortune 500 companies. 

Plus we need to balance our R&D resources between fixing Sybase's old bugs that customers have tolerated (i.e. worked around) over the years with bringing long overdue features to market.  Hope you guys take a moment to look at this from our side as well that we are trying to do the best we can for PowerBuilder customers and their projects.

But even with that said we aren't passing the buck and have fixed or are currently releasing fixes for a number of such legacy bugs that Sybase ignored, such as bug ID # 150, 173, 186, 190, 199, 219, 234, 237, 256, 309, 314, 316, 341, 354, 381, 415, 437, 509, 535, 589, 717, 819, 896, 990, 1037, 1153, 1233, 1345, 1360, 1392, 1693, 1718, 1729, 1745, 1757, 1783, 1814, 1825, 1856, 1982, 1999, 2278, 2443, 2467, 2576, 2743, 2753, 2834, 3077, 3504, 3513.

Thanks,

Armeen Mazda
CEO, Appeon

Comment
Yes Armeen, you are very right on this. I might seem negative sometimes, but really do appreciate the effort Appeon is doing and the customer service is "unseen". Maybe it's because we have been suffering so much time with Sybase and SAP and now that there's a positive direction, we have become a bit demanding.

regards
  1. Miguel Leeuwe
  2. Thursday, 7 November 2019
Also, I now realize that this bug also happens in 12.6. Many more like this will show up, as now users want things to be fixed.

Before Appeon, we'd have to re-think twice before reporting a bug, as many times it would not "go anywhere".
  1. Miguel Leeuwe
  2. Thursday, 7 November 2019
Thanks for your understanding, and we do sympathize with what customers went through during the Sybase days. We are doing our best to wash the bad taste out of people's mouth and rebuild goodwill with the PB community. For what its worth, if we introduce a regression bug (i.e. didn't exist in PB 12.6) it will be rare that we don't fix it because that code is usually would be written by us, and that's why in the support portal we have a specific bug classification called "Sybase (legacy) bug".

But even with that said we aren't passing the buck and have fixed or are currently releasing fixes for a number of such legacy bugs that Sybase ignored, such as bug ID # 150, 173, 186, 190, 199, 219, 234, 237, 256, 309, 314, 316, 341, 354, 381, 415, 437, 509, 535, 589, 717, 819, 896, 990, 1037, 1153, 1233, 1345, 1360, 1392, 1693, 1718, 1729, 1745, 1757, 1783, 1814, 1825, 1856, 1982, 1999, 2278, 2443, 2467, 2576, 2743, 2753, 2834, 3077, 3504, 3513.

The exception to this generalization would obviously be any third-party stuff that is in PowerBuilder, such as the RichTextEdit control (made by Text Control). For third-party stuff we rely on the third-party to fix. But Text Control is very good company, and in fact they are rolling out a 64-bit version with many new features that we hope will make it into PB 2019 R3.
  1. Armeen Mazda
  2. Thursday, 7 November 2019
There are no comments made yet.
  1. Thursday, 7 November 2019
  2. PowerBuilder
  3. # 1
John Fauss Accepted Answer Pending Moderation
0
Votes
Undo

Has anyone tried replacing the hundreds of SetItem calls performed inside of the inner loop with a single statement outside of the inner loop that sets the data values in a column for all rows at once?

Integer li_x
Long ll_rowcount, ll_f
String ls_columndata[], ls_emptyarray[]
DataStore lds

lds = Create DataStore
lds.DataObject = "d_abc"
lds.SetTransObject(SQLCA)

For li_x = 1 To 500
   // Flush, then populate the DataStore.
   lds.Reset()
   ll_rowcount = lds.Retrieve('some_id')
   if ll_rowcount > 0 then
      // Clear column data array, then load it with new column data values.
      ls_columndata = ls_emptyarray
      
      
For ll_f = 1 To ll_rowcount
         ls_columndata[ll_f] = 'some data'
      Next
     
      // Replace column data in the DataStore for all rows at once.

      lds.Object.mycolname.Primary.Current = ls_columndata
      // or
      //lds.Object.mycolname[1,ll_rowcount] = ls_columndata
   end if
Next

MessageBox("","Before destroy")
Destroy lds

Disclaimer: I have NOT tested the above code, and I do not claim it will solve the memory utilization issue. However, syntactically it looks like it should work correctly. The technique is documented in the DataWindow Reference, section 4.3.1 "Syntax for one or all data items in a named column".

HTH,
John

Comment
Nice one, Yes this should work. I do have "mixed" experience with DATA though. Especially when getting full DATA and storing it in an any variable and - when user decides to not save - setting the primary buffer back using this stored any variable. I would sometimes get "chinese characters" in string fields. I should have some trivial explanation (somewhere) of SAP somewhere explaining why this sometimes happens.
  1. Miguel Leeuwe
  2. Friday, 8 November 2019
There are no comments made yet.
  1. Thursday, 7 November 2019
  2. PowerBuilder
  3. # 2
Miguel Leeuwe Accepted Answer Pending Moderation
0
Votes
Undo

Hi Steen,

I think that 3 possible workarounds were suggested when you reported bug 535.

One of them would be to do "

lds.resetupdate() //to release memory

If your MyDatastore would be a standard user object datastore, like n_ds of the pfc's, maybe a possible solution would be to do a ResetUpdate() in the RetrieveStart() event of the ancestor class of MyDataStore.

Just an idea, don't know if it would really work, this workaround.

regards,

Comment
I've tested my proposed solution and it works!

I've added these 2 lines of code to the retrievestart event of n_ds:



this.ResetUpdate()

return AncestorReturnValue



What is weird is that it does NOT work when doing the same thing in the retrieveEND event.

So my next move is to make sure all of my datatores are created as n_ds instead of datastore.

regards
  1. Miguel Leeuwe
  2. Thursday, 7 November 2019
There are no comments made yet.
  1. Thursday, 7 November 2019
  2. PowerBuilder
  3. # 3
Miguel Leeuwe Accepted Answer Pending Moderation
0
Votes
Undo

This is the won't-fix bug I'm referring to, as Steen J. reported this before as a bug:

https://www.appeon.com/standardsupport/search/view?id=535

 

Comment
Ah, now I see that Yakov already mentioned this bug too..
  1. Miguel Leeuwe
  2. Thursday, 7 November 2019
There are no comments made yet.
  1. Thursday, 7 November 2019
  2. PowerBuilder
  3. # 4
Roland Smith Accepted Answer Pending Moderation
0
Votes
Undo

The Retrieve function returns the number of rows so you should get it into a variable and use that in the for-next.

The RowCount function is being executed every time through the loop.

 

Comment
Hi Roland,

You're right, but this code is just meant as a how to reproduce the bug, is my guess. The problem of the memory leak won't be fixed by replacing the rowcount().
  1. Miguel Leeuwe
  2. Thursday, 7 November 2019
There are no comments made yet.
  1. Thursday, 7 November 2019
  2. PowerBuilder
  3. # 5
Miguel Leeuwe Accepted Answer Pending Moderation
0
Votes
Undo

Well, Appeon's answer on this reported bug has been clear: they won't fix it as they don't know how much other stuff they might break in doing so.

"Wow", is all I can say !

Comment
There are no comments made yet.
  1. Thursday, 7 November 2019
  2. PowerBuilder
  3. # 6
yakov werde Accepted Answer Pending Moderation
0
Votes
Undo

Hello

Seems like we are encountering the same issue in multiple places when processing large data sets.   This is a likely cause for multiple reported failures.  Since "loop retrieve, setitem, update" is a common processing pattern code refactoring is not practical.  I saw the case https://www.appeon.com/standardsupport/search/view?id=535

Was a fix implemented in R3 or 2019?

 

Thanks

yakov.werde@iongroup.com

Comment
Does 64bit help?
  1. Roland Smith
  2. Wednesday, 6 November 2019
Hi Roland, Thanks for the suggestion.

We ran the production code against a large customer dataset in 2017 R2 64 bit. Same abrupt termination results. Seems like the flaw is not address memory related.

We migrated to Appeon 2019 and ran the production code against a large customer dataset . Same abrupt termination results.

We are estimate at least about 2 months of work to refactor all the suspect code routine, with the expectation (but not an absolute guarantee) that using a different algorithm will avoid the ABEND.

Sure wish engineering could address the runtime issue!



Yakov
  1. yakov werde
  2. Thursday, 14 November 2019
There are no comments made yet.
  1. Wednesday, 6 November 2019
  2. PowerBuilder
  3. # 7
Chris Pollach Accepted Answer Pending Moderation
0
Votes
Undo

Hi Steen;

    Can you try the following ....

  1. Create a PB.INI file in your App's work folder
  2. Add the following to the INI ...

[DataStore Behavior]
UseHwnd=no

Let me know if that helps!

Regards ... Chris

 

Comment
Hi Dan;



      Basically, this stops the PBVM from allocating very expensive control blocks and MS-Window handles for DataStores. The PBVM treats the DS like a DW Control and thinks that you might make it visible at some point - so it allocates graphic resources for the DS. The above setting tells the PBVM to ignore creating these resources as the DS will never be visible. So not only do you save MS-Window resources, it also makes your PB Apps run much faster where DS use is concerned.



    BTW: This fix goes way back to the PowerSoft days of PB 7 / 8.



Regards ... Chris



 

  1. Chris Pollach
  2. Tuesday, 28 November 2017
Hi Chris,



Can the DataStore object still print if it has no Windows handles? /Michael

  1. Michael Kramer
  2. Tuesday, 28 November 2017
Hi Michael;



  A great question and ... Yes, it can!  



Regards ... Chris

  1. Chris Pollach
  2. Tuesday, 28 November 2017
There are no comments made yet.
  1. Monday, 27 November 2017
  2. PowerBuilder
  3. # 8
Roland Smith Accepted Answer Pending Moderation
0
Votes
Undo

Try getting the rowcount into a long first instead of having it part of the For statement.

Comment
There are no comments made yet.
  1. Monday, 27 November 2017
  2. PowerBuilder
  3. # 9
Govinda Lopez Accepted Answer Pending Moderation
0
Votes
Undo

Hi Steen,


Based on your example code, I infer that, you are using a nested "FOR" because you are doing some more processing in it. But, if this wasn't the case, I would suggest to fill that column in the query during the RETRIEVE. (ie: SELECT a, b, mycolname = 'some data' FROM mytable WHERE a BETWEEN 1 AND 500). That way you may be able to avoid the use of a FOR cycle of 350,000 iterations.

On the other hand, if you find that a particular part of the function is causing this, then I would recommend you open a support ticket for this case. It may be something with the internal processing of the info.

I hope this helps and please do share your results here.


Regards,

Comment
Hi Govinda,



Thanks for your quick reply :-)



 



The sample code is a simple way to repro the issue.



The application has 5000 objects and cannot be easily rewritten to workaround the bug.



using the sample code the issue is 100% visible. To my mind a huge memory management bug.



I will open a ticket. ;-)



 



Thanks - Steen



 

  1. Steen Jakobsen
  2. Monday, 27 November 2017
Thank you Steen,



 



I will review your ticket. Meanwhile, have you used this on other PowerBuilder versions as well? If so, did this happen there too?



 



 



Regards,

  1. Govinda Lopez
  2. Monday, 27 November 2017
There are no comments made yet.
  1. Monday, 27 November 2017
  2. PowerBuilder
  3. # 10
  • Page :
  • 1


There are no replies made for this question yet.
However, you are not allowed to reply to this question.