Subscribe via RSS
Toggle Submenu
  1. Categories
  2. Recent
  3. Tags
View Replies (8)

Resolved Use C# for Replace() function

  1. How-to
2
Votes
Undo
  1. Miguel Leeuwe
    Miguel Leeuwe
  2. PowerBuilder
  3. Tuesday, 8 September 2020 01:50 AM UTC

Hi,

TIP:

I don't know if anyone has a similar function, so this is just a tip.

We have a global function "f_str_replace()", which uses a loop with the powerbuilder Pos() and Replace() functions to replace all occurrences of an "old" piece of string within a string and replaces them with a "new" piece of string.

Now that - with pb2019 r2 - it's way easier to call a c# function in a DLL, I decided to replace that function with the Replace() of C#. It no longer needs a loop to find and replace all "old" strings. It's a single call to Replace() in .Net.

This makes the replace between 500 and 600 times faster on my machine. (this is very much depending on how big the strings are and what you might be replacing with what).

We use this replace function in lots of places, so it's pretty useful.

We instantiate the .Net caller object in a global variable, so it only has to be done once.

(If anyone's interested and needs source code, just let me know).

regards

MiguelL

 

BTW: I tried to see if .net's Substring() function is also faster than Powerbuilder's Mid() function, but that was a big disappointment. Powerbuilder's Mid() function is waaaaay faster than the .net Substring() function. Probably due to all kinds of checks with exceptions that .net does before actually performing the substring().

 

how to change the image of a picture control to a ...
Serena CM build tool and PB 12.6 - no images are s...
Responses (8)
Miguel Leeuwe
Miguel Leeuwe Accepted Answer Pending Moderation
  1. Saturday, 4 June 2022 15:26 PM UTC
  2. PowerBuilder
  3. # 1

Hi all,

I'm sorry for the prolongued comments on this thread, but I just came to some conclusions:

I've done a test using .Net 6.0 instead of .net framework 4.51 and the times are the same. (Great!). Interesting enough, compiling the DLL for 'any cpu' makes the replace slower. Compiling it for x86, shows a performance twice as fast (0.14 seconds instead of 0.29 seconds for 10 loops).

I've done some more test with a larger initial string () (616400 characters) and here are the results:

These results are based on a single loop: Loops = 1. 

NOTE: If you want to try it yourself, don't forget to 'Unblock' in the Properties of the DLL file as shown in an image elsewhere on this page.

f_str_replace: 352.703 seconds. (point as decimal separator)

PFC- Mid() (of_globalReplace): 203.922 seconds

Roland's BlobMid() solution: 134.453 seconds.

Dotnet_replace C# DLL: 0.015 seconds.

To be fair, I'm doing some extreme replacing here: I replace all 'e' characters with 'abcdefg', which might not be something you do in normal life.

Well, I think this is enough for just now, just remember that for shorter strings with few replacements, powerbuilder's Replace() function works great and might even be the fastest solution.

(I've attached my .Net 6 solution, you'll need the .Net 6.0 SDK / Runtime installed to run the sample).

regards,

MiguelL

 

 

 

 

Attachments (1)
SampleAppReplaceNet6.zip
Comment
Load more comments
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Sunday, 5 June 2022 02:45 AM UTC
Hi Mark,

It's an interesting point! I have used (for example) Oracle's Replace / regexp_replace function many times, but only to replace existing data in a database, which I think is pretty fast!

The (search and) replace that I'm thinking of with all of this, is more generic: change contents of files read from disk, dwSyntax, etc.

. You could try to do something like "SELECT Replace(:ls_myVar, ...) .... FROM DUAL;", maybe using dynamic SQL, but it's going to be up to a limited size before needing to work with a Blob variable. Once you need UPDATEBLOB / SELECTBLOB things would slow down considerably.

Also when working with a DB Replace, you're working over a network connection to your database (unlees you run it locally on your own PC). That would also slow down the process. So, it depends on what you're doing I suppose.



best regards,

MiguelL

  1. Helpful
Mark Goldsmith
  1. Mark Goldsmith
  2. Sunday, 5 June 2022 16:23 PM UTC
No question the network could come into play and of course not all networks are the same. A couple of my customers have a number of processes that run via scheduled tasks on the server and so the network would not come into play in those scenarios. I was just curious though how it would compare, primarily to the non-dot-net approaches, since those approaches turned out to be so much slower. Thanks again Miguel!

Regards,

Mark
  1. Helpful
Benjamin Gaesslein
  1. Benjamin Gaesslein
  2. Tuesday, 7 June 2022 10:17 AM UTC
Never apologize for providing valuable info, please. :)
  1. Helpful
There are no comments made yet.
Steen Jakobsen
Steen Jakobsen Accepted Answer Pending Moderation
  1. Tuesday, 31 May 2022 16:20 PM UTC
  2. PowerBuilder
  3. # 2

GREAT WORK!

Comment
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 31 May 2022 16:35 PM UTC
Glad you like it, Thanks!
  1. Helpful
There are no comments made yet.
Benjamin Gaesslein
Benjamin Gaesslein Accepted Answer Pending Moderation
  1. Tuesday, 31 May 2022 13:58 PM UTC
  2. PowerBuilder
  3. # 3

Cool thingy, Miguel.

Out of curiosity, you mention possible plans to implement an overflow check. Have you ever tried to provoke a string overflow using it? Both PB and .Net strings are limited by 32bit memory constraints so you'd need to get a string longer than one billion characters. PB help states a maximum string length of 1,073,741,823. .Net objects can't be bigger than 2GB so @2bytes per char, that's also the maximum a .Net string can hold. An out of memory exception will probably be thrown when you get near that limit but at this point we're juggling sizeable ~2GB variables so I'm guessing there will be some performance issues long before that.

I can personally see no way that this will be an issue in practice unless you do it on purpose. :)

Comment
Load more comments
Brad Mettee
  1. Brad Mettee
  2. Tuesday, 31 May 2022 18:14 PM UTC
For starters, test the search string length against the replace string length. If search len >= replace len, then no overflow can occur and you can safely replace and return the string.

If replace is longer, let the C# code do the replacing, then test string length before returning it to PB.

The few extra steps will prevent any overflows and shouldn't increase execution time enough to matter.

  1. Helpful 1
Benjamin Gaesslein
  1. Benjamin Gaesslein
  2. Wednesday, 1 June 2022 06:30 AM UTC
The C# code can also hit the limit and will raise an exception, though. PB strings and .Net strings have the same size constraints. I guess you could catch OutOfMemoryException in C#.
  1. Helpful 1
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Wednesday, 1 June 2022 07:28 AM UTC
Hi Brad and Benjamin,

I like the advice. As far as C# hitting the limit, I don't think that would be any problem at all. Powerbuilder will hit its limit way sooner with their funny way of working with their own memory stack/ pb heap. Maybe not when creating a 64 bit executable, I have never tried that. Just try reading a 600MB video file with for example FileReadEx(). You might have loads of memory available in windows, but there's a point where PB just doesn't anticipate the amount needed.

Correct me if I'm wrong,

regards and thanks again!

MiguelL
  1. Helpful
There are no comments made yet.
Miguel Leeuwe
Miguel Leeuwe Accepted Answer Pending Moderation
  1. Tuesday, 31 May 2022 11:57 AM UTC
  2. PowerBuilder
  3. # 4

NOTE:

I've been trying to use this in pb2021 to do a demo and I got an error when debugging signalerror:

An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.

(Regretfully I don't show this error when running the current code).

It turns out you have to edit the properties of the DLL and mark/activate the "unblock" checkbox.

First close the PB IDE !!

Then when running again, things work correctly.

regards,

MiguelL

Comment
There are no comments made yet.
Miguel Leeuwe
Miguel Leeuwe Accepted Answer Pending Moderation
  1. Tuesday, 8 September 2020 16:14 PM UTC
  2. PowerBuilder
  3. # 5

Hi,

Okay, I've made a sample application. See attached.

There's:
- an editmask in which you can set the amount of loops for the replace to occur.
- 2 Buttons on the left and 2 buttons on the right.
- one wide button below: When you have clicked on a button on the left and after that on a button on the right, you then can use the wide button below "compare string result ... " to see if the results of replacing have been the same or if something has gone wrong and there's a difference. They should all result in the same and I think that's the case.

So I've done a small test with 10 loops:
4th place: Slowest is my old f_str_replace() (uses Powerbuilder's Replace() ): 26.7 seconds
3rd place: A bit less slow is the PFC function (uses Powerbuilder's Mid() ): 16.5 seconds
2nd place: Roland's string class (uses Powerbuilder's BlobMid() ): 10.4 seconds
1st place: Fastest is the .Net Replace() function (DLL is in the zip): 0.016 seconds

The difference is really HUGE: It's pretty logical too, in .Net it's simply a single call to its Replace function, no loops or position functions needed.

What's missing is to build checks in to not overflow the return string. If my original string is almost the maximum allowed string length and I'm then going to replace "e" with "a much longer string", than that check is definitely necessary. I think the only function that has some kind of check on that is the one in Roland's stringclass.

I should also add a try ... catch .. to the .net version and check for overflow. (That will probably also make it a bit slower).

I hope it serves anyone.

regards,

MiguelL

Attachments (1)
SampleAppReplace.zip
Comment
Load more comments
Steen Jakobsen
  1. Steen Jakobsen
  2. Tuesday, 31 May 2022 16:19 PM UTC
LOVE IT!!! THANK YOU Miguel Leeuwe





//Steen
  1. Helpful 1
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 31 May 2022 16:30 PM UTC
Hi Roland,

For most cases your BlobMid() solution is really a lot faster than using Replace() or even Mid(). You do make an interesting point here. When I have some time I'll play around with it.

Thanks!
  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Saturday, 4 June 2022 13:43 PM UTC
Hi Roland,

I did a little test and passed the as_oldstring by Reference to the ReplaceAll() function. Instead of copying it to ls_oldstring, I ommited ls_oldstring adn used the parameter itself directly.

I did a test with an initial string with a length of 616,400 characters. With my test, ByRef was only half a second faster: Current method took 134.453 seconds and byRef took 133.921 seconds. Not really worth it, since when using ByRef, you will also loose the contents of your original string (in case you want to re-use it). So if you don't want the original string to change, you'd still have to do a copy before calling ReplaceAll() and then afterwards re-assign the original value.

Regards
  1. Helpful
There are no comments made yet.
Andreas Mykonios
Andreas Mykonios Accepted Answer Pending Moderation
  1. Tuesday, 8 September 2020 14:25 PM UTC
  2. PowerBuilder
  3. # 6

Hi.

Well, I don't use my implementation of this function. But I do use PFC's imlpementation.

n_cst_string has the following function:

of_GlobalReplace function

 Description

Replaces all occurrences of one string within another string.

 

I haven't tested to see if .net replace is faster than of_globalreplace.

Andreas.

Comment
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 8 September 2020 14:33 PM UTC
Good point! That function uses Mid(), which should be waaaay faster than Replace().

I'll build it in in my little sample app.
  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 8 September 2020 14:36 PM UTC
But only if you first called of_setGlobalReplaceMethod( CST_GLOBALREPLACE_NEW ).

If not, it will use the "legacy" replace() function.

regards.
  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 31 May 2022 16:31 PM UTC
( using Mid() instead of Replace() is almost twice as fast in my little test ).
  1. Helpful
There are no comments made yet.
Olan Knight
Olan Knight Accepted Answer Pending Moderation
  1. Tuesday, 8 September 2020 14:08 PM UTC
  2. PowerBuilder
  3. # 7

Hey, Miguel -

Can you please show us an example, including the function API calls, along with whatever else is needed to make such a global function - and then use it?

Some of us have never used a .NET call in our life!  :O


Thanks,

Olan

Comment
Load more comments
Olan Knight
  1. Olan Knight
  2. Tuesday, 8 September 2020 14:17 PM UTC
That's awesome, but way more than I need! :) But hey, I'll take it and say "Thank You"!
  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 8 September 2020 18:15 PM UTC
YW Olan! At least you get the full picture now :)
  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 8 September 2020 18:17 PM UTC
BTW: I created the nvo using the DLL Import Tool. I'm not sure if I mentioned that.

The typical setup is to create the nvo as a global and then just call it's functions anywhere.
  1. Helpful
There are no comments made yet.
mike S
mike S Accepted Answer Pending Moderation
  1. Tuesday, 8 September 2020 13:26 PM UTC
  2. PowerBuilder
  3. # 8

I'm curious - have you compared it to using blob for your string functions?

https://community.appeon.com/index.php/qna/q-a/n-cst-string-of-globalreplace-function-slow

based on what you are saying, the .net version should still be faster than the blob style of string processing.

 

 

and i agree that the .net inclusion with 19R2 is another game changer from appeon.

Comment
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 8 September 2020 14:20 PM UTC
Hi MIke,

Haven't tried it yet, but you make an interesting point, so I'm going to try that now.

We already are using Roland's n_stringClass in some places.

I'll let you know about the results, or even better, I'll build it in in the sample I'm making for Olaf.

regards

  1. Helpful
Miguel Leeuwe
  1. Miguel Leeuwe
  2. Tuesday, 8 September 2020 18:00 PM UTC
Hi,

the Replace of .Net absolutely rules when it comes to replacing all occurrences. If it's just a single replace of a single occurrence, we can just use the PB function, seems to be even slightly faster in some cases.
  1. Helpful 1
There are no comments made yet.
  • Page :
  • 1


There are no replies made for this question yet.
However, you are not allowed to reply to this question.