Subscribe via RSS
Toggle Submenu
  1. Categories
  2. Recent
  3. Tags
View Replies (8)

Resolved Bizarre Performance issues with 250K rows

  1. How-to
0
Votes
Undo
  1. Richard Carrier
    Richard Carrier
  2. PowerBuilder
  3. Wednesday, 15 May 2019 18:59 PM UTC

We are having a strange issue. Users are retrieving a data window with about 70 columns and before you ask, yes they need all these for tax research. Also, the number of records that must be retrieved is around 250K and yes, they need all these records.

On my laptop , using the latest Appeon, it loads fine and I have 16gb ram. In task manager I can see that after all rows are loaded and sitting in my screen, I used about 500mg of memory.

My user has 8gb ram, when they run it from the executable deployed, memory goes up to 1500MG then GPF.

 

However, if they load an older version of the app. that is pre appeon from 2015, (same columns, same # of rows) but older DLL's, its runs in 5 minutes, uses normal memory and works perfect.

 

Users are now complaining that our new builds are the cause of this not working.

 

Any help or suggestions appreciated. 

 

I'm using the latest SNC driver but there are some connect string parameters I'm looking at. But this is just a real irritation

 

Thanks

Richard

Display of PDF from blob
How-to? Incremental number in group header
Responses (8)
Richard Carrier
Richard Carrier Accepted Answer Pending Moderation
  1. Tuesday, 21 May 2019 22:55 PM UTC
  2. PowerBuilder
  3. # 1

I did not know about the trimspaces = 1

 

I do have an update

We added  PB.ini file

We use rows to disk

I had a yield() that I removed that was worthless

It seems to be working now with 280K rows. The minute I turn off rows to disk, it blows up past 1500mg.

 

THANK YOU for all the help !!!

Comment
There are no comments made yet.
Chris Pollach @Appeon
Chris Pollach @Appeon Accepted Answer Pending Moderation
  1. Tuesday, 21 May 2019 21:25 PM UTC
  2. PowerBuilder
  3. # 2

Hi Richard;

  I am glad that the "Rows to Disk" feature worked for you ... at least as a workaround for now!  :-)

BTW: In your SNC connection ... did you set the DBParm filed to ensure that the PB DB Client driver stripped off trailing spaces? The DBParm setting would be  ... "TrimSpaces=1". If you have lots of VarChar or large CHAR fields in your SQL Query, there could be a "ton" of memory consumption on the PB App side if your DBMS is sending lots of trailing spaces in the result set. This would in turn, cause memory havoc on the PB App's side when processing a super large result set. Just a thought!

Regards ... Chris

 

Comment
There are no comments made yet.
Richard Carrier
Richard Carrier Accepted Answer Pending Moderation
  1. Monday, 20 May 2019 21:24 PM UTC
  2. PowerBuilder
  3. # 3

Chris, should I get the C++ dlls that appeon says are in the powerbuilder shared or from the website they also mention.

They are different versions and dates

Comment
Chris Pollach @Appeon
  1. Chris Pollach @Appeon
  2. Tuesday, 21 May 2019 21:31 PM UTC
Hi Richard ... yes, the C++ Microsoft DLL's, Java DLL's and some 3rd party packaged DLL's change quite often across major PB releases. The best practice (IMHO) is to deploy these DLL's that match the PB IDE release that you are using. You are also correct .. these DLL's normally reside in the "Shared" PB installation folder.

Regards ... Chris
  1. Helpful
There are no comments made yet.
Richard Carrier
Richard Carrier Accepted Answer Pending Moderation
  1. Monday, 20 May 2019 20:40 PM UTC
  2. PowerBuilder
  3. # 4

OK so this has become a raging rats nest.

 

  • I completely redid the build, every single file in from Appeon and they are all from the same build.
  • We turned on rows to disk, on all my computers, it's slower but uses less ram and works

On the customers Dell with 16gb of ram, it flys past the 348mb that my machines used to load the data, does to 1500mb and crashes with the below message. Also, I'm using the PB SNC PROVIDER='SQLNCLI11

My next option is to go backwards to the older SNC provider that was in 12.5

 

Any help is really appreciated

Faulting application name: akoreftplus.exe, version: 3.9.4.0, time stamp: 0x5c4c46f1

Faulting module name: pbdwe170.dll, version: 17.2.0.1880, time stamp: 0x5c4c47f4

Exception code: 0xc0000005

Fault offset: 0x0024c375

Faulting process id: 0x4240

Faulting application start time: 0x01d50f2c16edff01

Faulting application path: C:\Program Files (x86)\AKOREFTPLUS\akoreftplus.exe

Faulting module path: C:\Program Files (x86)\AKOREFTPLUS\pbdwe170.dll

Report Id: d7de2369-9d89-4d34-8a80-6f63d77d7c06

Faulting package full name:

Faulting package-relative application ID:

 

Comment
Armeen Mazda @Appeon
  1. Armeen Mazda @Appeon
  2. Monday, 20 May 2019 22:29 PM UTC
Can you provide reproducible test case to our support team to diagnose?
  1. Helpful
David Peace (Powersoft)
  1. David Peace (Powersoft)
  2. Tuesday, 21 May 2019 10:04 AM UTC
Hi

Is your application crashing when you retrieve on the DW or later when you are doping some processing?



Is the DW you are using displaying the data for the user? or just for processing, would it be appropriate to use a datastore instead. I'm thinking that the UI rendering is possibly the cause of your RAM usage. Just a thought.



Cheers

David
  1. Helpful
There are no comments made yet.
Olan Knight
Olan Knight Accepted Answer Pending Moderation
  1. Thursday, 16 May 2019 23:12 PM UTC
  2. PowerBuilder
  3. # 5

Can you write the data into a FILE, then have the code open that file? Excel comes to mind.

Can you use ROWS AS NEEDED/DISPLAY AS NEEDED in the DW?

 

Olan

 

Comment
There are no comments made yet.
Chris Pollach @Appeon
Chris Pollach @Appeon Accepted Answer Pending Moderation
  1. Thursday, 16 May 2019 20:43 PM UTC
  2. PowerBuilder
  3. # 6

Hi Richard;

  Are you using the PB SNC native DB driver or ODBC's SNC feature?

Regards ... Chris

Comment
There are no comments made yet.
Armeen Mazda @Appeon
Armeen Mazda @Appeon Accepted Answer Pending Moderation
  1. Wednesday, 15 May 2019 21:04 PM UTC
  2. PowerBuilder
  3. # 7

Try turning off accessibility feature of Windows OS to see if related to this?

Comment
Chris Pollach @Appeon
  1. Chris Pollach @Appeon
  2. Thursday, 16 May 2019 00:27 AM UTC
Hi Richard;

Armeen's suggestion is a good one but often hard to enact as many corporate PC's are locked down from doing this MS-Windows alteration. Especially, on end user PC's. ;-)

You can do this for your PB App by creating your own PB.ini in your application's folder and then adding the following section:

[Data Window]

ACCESSIBILITY=0

Note: If you create an EXE and deploy this to a test or user PC. Make sure that your custom PB.ini file goes with the EXE and is placed in the same folder.

HTH

Regards ... Chris
  1. Helpful
There are no comments made yet.
Chris Pollach @Appeon
Chris Pollach @Appeon Accepted Answer Pending Moderation
  1. Wednesday, 15 May 2019 19:14 PM UTC
  2. PowerBuilder
  3. # 8

Hi Richard;

 Yes, 250K of Unicode datum will probably do your 8G of RAM PC App user in with an out of memory condition (a la GPF).  :-(

You now have two options:

1) Compile your App into a 64Bit EXE.

2) Try changing your DWO to use the "Rows to Disk" option.

   DW Painter => Rows => Retrieve Options => Rows To Disk

HTH

Regards ... Chris

 

 

Comment
Load more comments
Chris Pollach @Appeon
  1. Chris Pollach @Appeon
  2. Wednesday, 15 May 2019 19:59 PM UTC
PS: Double check that you have the correct PB run-time version & build that it matches the same as your PB IDE environment. A mismatch there can have very weird consequences / side effects.
  1. Helpful
Richard Carrier
  1. Richard Carrier
  2. Wednesday, 15 May 2019 21:46 PM UTC
That is our next step.

I will let you know
  1. Helpful
Richard Carrier
  1. Richard Carrier
  2. Thursday, 16 May 2019 20:16 PM UTC
Chris, the version that is failing is written using the latest Appeon available, the version that works was written using SAP Powerbuilder 12.5 in 2015.



  1. Helpful
There are no comments made yet.
  • Page :
  • 1


There are no replies made for this question yet.
However, you are not allowed to reply to this question.
We use cookies which are necessary for the proper functioning of our websites. We also use cookies to analyze our traffic, improve your experience and provide social media features. If you continue to use this site, you consent to our use of cookies.