WMI thread issue

hi,

Our Windows application server (2022) keeps going down with below error message. We have our PB 2022 R2 application hosted on this server..

Windows Management Instrumentation has stopped WMIPRVSE.EXE because a quota reached a warning value. Quota: ThreadCount Value: 259 Maximum value: 256 WMIPRVSE PID: 53864 Providers hosted in this process: C:\Windows\System32\wbem\WmiPerfClass.dll, C:\Windows\System32\wbem\WmiPerfClass.dll

the error implies Application Queries using the WMI process ...that it has a Thread limit of 256 .... and the error implies it found 259

We dont know whats causing this as we didnt had this issue before we migrated to PB 2022 R2.. The Oracle version we are using is 19c in case anybody needs to know. This is affecting our production and always bringing primary server down and inaccessible by Infrastructure team as well.. only workaround is to reboot server so far.

Would appreciate any advise to fix the same.

Thanks,

Joy

Armeen Mazda @Appeon Accepted Answer Pending Moderation

Wednesday, 13 November 2024 14:20 PM UTC
PowerBuilder
# 1

How do you know the PB upgrade is causing the issue? PB itself doesn't have any dependency on this and like you said you didn't call Windows API to use this service. So it could just be a coincidence it is misbehaving now. I would suggest you involve somebody from Citrix or Microsoft support to determine why this Windows OS feature is misbehaving.

Comment

Joy Gupta
Wednesday, 13 November 2024 16:35 PM UTC

Hi Armeen,

My intention was not to say that PB upgrade is causing the issue, but more if this was a new feature or something as we never had this issue before. We migrated in april 2024 but issue was seen from 7/31.. My Infra team is also trying to decipher as to what is causing the issue. If required we will engage with Windows support and Citrix support as well. Yesterday's issue co-incided with our Database going down as well. The error seemed to point to application making too many queries thats why i was asked to look at the application layer. My dev team investigated but said they never heard or used it for anything thus i came to you all for guidance.

Helpful 0

Armeen Mazda @Appeon
Wednesday, 13 November 2024 16:45 PM UTC

Hi Joy, Thanks for the clarification. We did not do anything with WMI in PB 2022. So it must be some other application or configuration you did that has enabled WMI. You might want to try disabling WMI and see how that impacts things... just as a troubleshooting step.

As far as your database going down, it is possible there is some coding issue in your PowerBuilder app that is overloading your database. Sometimes coding issues go undetected until exactly the moon and sun align and then blows things up. I say this because gain I think it is coincidence there is database issue with the PB 2022 upgrade. We have not made changes in how DB connections/transactions work. You can review the new features guide by version to understand exactly what changes Appeon has made between your previous version and PB 2022: https://docs.appeon.com/pb/whats_new/index.html

One thing you might want to do is check if any configurations have changed or if you upgraded the Oracle client at the time you did the PB 2022 upgrade. It could be configuration change or upgrade to Oracle client causing the issue?

Helpful 0

There are no comments made yet.

John Fauss Accepted Answer Pending Moderation

Tuesday, 12 November 2024 21:05 PM UTC
PowerBuilder
# 2

Hi, Joy -

From the limited info you have supplied, it looks as if your application is querying Windows Management Instrumentation (WMI) services to obtain performance information. Can you verify this? If so, what information is being queried and how often does the app issue the query/queries to WMI?

The app is executing on a Windows server? Was the previous version of the app also running on the same Windows Server 2022 machine? Has anything changed besides the PB version used to produce the application?

Are multiple users running the app on the server simultaneously? If so, how many?

Best regards, John

Comment

Load more comments

Joy Gupta
Wednesday, 13 November 2024 12:56 PM UTC

Forgot to answer this, the application is accessible via Citrix platform and has 50 concurrent users at any time. its a worldwide used application.

Helpful 0

John Fauss
Wednesday, 13 November 2024 14:38 PM UTC

In light of what you have stated, I agree with Armeen that the PB app is unlikely to be the cause of this problem. Fifty concurrent users on one Citrix server sounds like a lot, but I'm not Citrix-savvy. Is there a regularly-scheduled maintenance window for this server, where all users could be logged out and the server restarted, say, weekly or bi-weekly?

Helpful 0

Joy Gupta
Wednesday, 13 November 2024 16:45 PM UTC

John,

We have a citrix farm under which there are many application servers serving many different environments. The issue in question is coming on only 2 Production servers. Citrix farm has 100 concurrent licences so we are good on number of users accessing it as its a worldwide used application. All servers are patched regularly and rebooted as well in after hrs window. These servers are also monitored as well so if they go down someone would take action immediately but this issue is hanging the server so unless admin tries to login to the server, they wont know it has gone down or there is a user complain that they are not able to log on as logon is terminating.

I am taking all the advise from all of you and also working with infra team to get to bottom of this. Will share if we find the resolution because right now its manual monitoring and rebooting is all we can do.