Hi John,

It seems to be a bug. Please open a Support Ticket for this issue (at https://www.appeon.com/standardsupport) and also attach a simple PB app to the ticket as a test case.

Many thanks in advance!

Regards,

Tom Jiang

An auto-retry might be a useful feature, not in your case since you are using it for credit cards.

Hi Tom,

Did you find if this is a httpclient bug in PB2017R3. We are planning to make some changes in Timeout in our application to not wait longer at client side. We are running PB2017R3 build 1858.

Please confirm.

Thanks, Vipin

Hi John,

Did you get this fixed?

Thanks, Vipin

Vipin, we opened a support ticket for this bug with Appeon.  To my knowledge it has not been fixed yet.

Our workaround is to extend our timeout beyond the API's timeout.

So if the API times out at 60 seconds, we set our timeout at 65 seconds to ensure no duplicate requests are posted.  Not ideal, but we can live with it while the issue gets addressed.

Thanks,

John

Might be worth looking into if they have a REST API as well.

John- Did a bug get submitted? I'm not finding one via the Standard Support portal.

I think we're seeing this -- or a very similar -- problem, PB 2017 R3. In our case it's a restful service and we're posting to an https (basic authentication) service.

I have a question about your scenario though: If I'm understanding correctly, the client times out after 15 seconds as expected .. but the service STILL receives 2 additional calls over the next 30 seconds? So even though the flow of execution has returned to you, there's still some background process running and issuing retries?  If so, that's terrifying...

I still have a lot or testing to do, but here's what I think I'm seeing so far:

In our case, in the debugger it steps into the SendRequest() function and comes out 90 seconds later with a "-1" error (timeout was set to 100 seconds). Meanwhile, during those 90 seconds the receiving service gets 3 duplicate requests exactly 30 seconds apart.

This is a service we use to submit orders, so we end up placing three identical orders -- not good!

In our case I believe the service (or something in between -- we have a complex environment with a middleware layer in between), seems to be timing out after exactly 30 seconds. In SoapUI at 30 seconds I get "org.apache.http.NoHttpResponseException: The target server failed to respond" which is a different error than the one you get when SoapUI itself times out. My theory is that the HTTPClient responds to this exception by retrying three times before returning "-1" (not "-2" as would be expected for timeout).

I'm not sure I understand the theory behind your proposed workaround of increasing the client timeout, and in fact I was going to try the opposite of what you suggest: I was hoping that setting the HTTPClient timeout to LOWER than the service timeout would cause the client to give up before getting and responding to that first "NoHttpResponse" exception and thus avoid retries. But what you're describing makes it sound like the retries may be unavoidable...

I've had problems getting HTTPClient to honor the timeout we set, but will be experimenting with that next. Any additional advice or clarity would be appreciated!

Hi Jeff, I did open Bug 2826 for this issue.

If we set the HTTPClient timeout greater than the web service timeout, everything is handled gracefully.  The service returns a failed response and we're able to handle in our TRY/CATCH logic.

If the HTTPClient timeout is less than the server timeout, it will keep retrying the request at the specified client interval until it gets a response from the web service.  

From your example, it sounds like your HTTPClient is set to a 30 second timeout so it keeps reissuing the call every 30 seconds (which is the bug we're seeing).

Note that we only see this when it's an HTTPS request; we didn't observe this behavior when sending an HTTP request.  Since we're dealing with credit card data, HTTP is not an option.  So we had to use the workaround of having the server timeout before the client to prevent the possibility of duplicate (or triplicate) calls.

Thanks,

John

Ugh, well shortening the timeout does indeed do what you've just described: With timeout set to 20 seconds, it didn't actually emerge from SendRequest() for 60 seconds, after sending three requests. (It did return a -1 return code -- is that what you're seeing?)

But whereas setting a long timeout works for you, it doesn't for me, and I think that's because of something specific to the server I'm talking to.

In your case setting a long timeout allows the Client to receive a proper timeout message from the server that it processes as a failure and doesn't cause it to retry. But the way our server times out is by sending this weird disconnect message after 30 seconds which also causes HTTPClient to retry three times.

So what's the weird thing my server is doing?

I used Wireshark while testing with SoapUI to determine that there was no network traffic between client and server during those 30 seconds, and that at exactly 30 seconds the server replied and that's when SoapUI failed. Unfortunately, because this is https, the traffic is encrypted so I can't actually see what the server is sending at 30 seconds, I just know SoapUI interprets is as a "NoHttpResponseException". But at least this appears to rule out something on the client causing the 30 second timeout.

A little bit of research on the internet indicates that http servers like Apache often have an "idle session timeout" which defaults to 30 seconds. To conserve connection pool resources. if a session is active but idle for 30 seconds, it hangs up on the client to return the connection to the pool, and that comes back as a NoHttpResponseException.

I believe I read that HTTPClient is a wrapper over Microsoft's WinHTTP API? I poked around the documentation a bit and didn't find any reference to auto-retry functionality like this. Did the Appeon developers add it themselves? Either way, for a GET or PUT, retrying might be fine, but for a POST a retry is a terrible idea, and having no control over this behavior puts me between a rock and a hard place.

As I can tell I'm left with NO workaround at all within the HTTPClient object, and will need to pursue either giving up on HTTPClient and using some other HTTP object, or trying to get configuration changes (which will effect the whole enterprise and may raise concerns about server resources) made at the server, to accommodate PowerBuilder.

I have not tested the same scenario with the RestClient; our current web service is SOAP based so until it's rewritten I'm locked into the HTTPClient.  

If/when time permits I'll write a simple REST service to try to mimic the same test scenario and report back.

I have two code examples that may be of use:

This one uses the WinHTTP API functions:

http://www.topwizprogramming.com/freecode_winhttp.html

This one uses the Msxml2.XMLHTTP COM control:

http://www.topwizprogramming.com/freecode_xmlhttp.html

It is coded to expect XML but that could be removed easily.

Dear Everyone, 

The HTTPClient had implemented an auto-retry feature which must be causing this issue. The developer is analyzing the issue and will provide a solution as soon as possible. 

Regards,

Tom Jiang

FYI our issue was resolved with a workaround provided by Appeon support.

In our case, setting SecurityProtocol to "5" (TLS 1.2) solved the problem.

When HTTPClient has a specific protocol set for its "securityprotocol" attribute, the retries no longer occur and the object reliably times out at the specified interval.

Apparently this is a bug in how the object implements automatic detection of the security protocol when the default value, "0", is not changed. The default represents "Automatic" and according to the documentation "It will detect and use the secure protocol in the following order: TLS 1.2, TLS 1.1, SSL 2.0/SSL 3.0/TLS 1.0."

So apparently the bug is that it is misinterpreting the server disconnect that occurs in our environment after 30 seconds as a security protocol failure which caused it to retry again with the next protocol.

The only remaining (minor) issue is that although the SendRequest() now times out at exactly the specified number of seconds, the return code is still "-1" (General Error), not "-4" as documented for a Timeout. But it looks like we can reliably infer a timeout by measuring how long it takes for the SendRequest() to return the error -- if the -1 is returned after a duration that's within a half-second of the timeout setting, it probably timed out.