• You are here:  
  • Home
  • Q&A
  • Q&A
  • PowerBuilder
  • Size limit while exporting data from datawindow/datastore to plain JSON

Subscribe via RSS
Toggle Submenu
  1. Categories
  2. Recent
  3. Tags
View Replies (2)

Resolved Size limit while exporting data from datawindow/datastore to plain JSON

  1. Issue
1
Votes
Undo
  1. Sivaprakash BKR
    Sivaprakash BKR
  2. PowerBuilder
  3. Friday, 25 August 2023 08:23 AM UTC

Hello,

Using PB 2022 R2 build 2819

Need to know whether any limit is there to export data from a datawindow / datastore to JSON.

Command used to export:

ls_json = dw_1.ExportJson(False)

Data in the above datawindow is retrieved by entering a period.  Data is retrieved for the given period.  If we give a whole year, then we get empty string in ls_json [ when I try to write the json to a file, I see an empty file, with no content ].  If the period is shorter, then we get a perfect JSON string.  

For one year, there are 117345 rows with 21 columns.  

Since this works for a shorter period, I suspect there should be some limit.  

Any idea?

Happiness Always
BKR Sivaprakash

 

  1. JSON
  2. PowerBuilder (Appeon)
SharedObjectGet debugger crash
How to build incrementally using command-line tool...
Responses (2)
Peter Pang @Appeon
Peter Pang @Appeon Accepted Answer Pending Moderation
  1. Tuesday, 29 August 2023 02:01 AM UTC
  2. PowerBuilder
  3. # 1

Hi BKR,

ls_json = dw_1.ExportJson(False)
The above statement is subject to two limitations:
1. The ExportJson function is implemented through the JSONGenerator object, and JSONGenerator has one limitation: It is not supported to use this object to process large data (20 MB or 100,000 data rows can be considered as large data based on our tests), because this object will call a third-party library which is memory intensive.
2. The length of String variable (ls_json) is limited, which may be deviated by different machines. My local machine can reach 40M size under 32bit.

BTW: If you are running 64bit, the above 2 restrictions will be relaxed a lot.


Best Regards,
Peter

Comment
There are no comments made yet.
Miguel Leeuwe
Miguel Leeuwe Accepted Answer Pending Moderation
  1. Friday, 25 August 2023 08:25 AM UTC
  2. PowerBuilder
  3. # 2

I think at least we should get some kind of error message when we're out of bounds.

Comment
Chris Pollach @Appeon
  1. Chris Pollach @Appeon
  2. Wednesday, 30 August 2023 18:46 PM UTC
Maybe wrapping this in a TRY..CATCH would capture that runtime issue?

FWIW: I wonder though if using the DBMS itself for large data requests like this would be the better way to handle that anyway?

For example: https://docs.devart.com/data-pump/exporting-data/export-to-json.html

(regardless of the export file type). ;-)

  1. Helpful
There are no comments made yet.
  • Page :
  • 1


There are no replies made for this question yet.
However, you are not allowed to reply to this question.