Hello:

We've recently upgraded to PB 2022 R2 2828, and one of our developers has two branches that are crashing during the full build, yet he has submitted four others that build just fine.  We have 9 other developers whose branches are working seamlessly.  Our SCC is Git.  We are compiling to 64 bit.

In the IDE if you do the RMB Full Build it does not crash, if you use the Project it crashes.  I've evaluated the project and it is the same on the branches that build successfully.  Do you have any suggestions as to how I can go about finding the code that is causing the issue?  

This is what the Reliability History gives us:

Description
Faulting application name: pbautobuild220.exe, version: 22.1.0.2828, time stamp: 0x65313be4
Faulting module name: PBSHR.dll, version: 22.1.0.2828, time stamp: 0x6531453b
Exception code: 0xc0000005
Fault offset: 0x001297b0
Faulting process id: 0xab4
Faulting application start time: 0x01da73d88fc63ed9
Faulting application path: C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\pbautobuild220.exe
Faulting module path: C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBSHR.dll
Report Id: 1e251ece-5f70-4dc8-9251-e8d56fd10fa8
Faulting package full name:
Faulting package-relative application ID:

The first error in the Event Viewer is :

Application: pbautobuild220.exe
CoreCLR Version: 6.0.2623.60508
.NET Version: 6.0.26
Description: The process was terminated due to an unhandled exception.
Exception Info: exception code c0000005, exception address 69CA97B0
Stack:

The next error in the Event Viewer:

Faulting application name: pbautobuild220.exe, version: 22.1.0.2828, time stamp: 0x65313be4
Faulting module name: PBSHR.dll, version: 22.1.0.2828, time stamp: 0x6531453b
Exception code: 0xc0000005
Fault offset: 0x001297b0
Faulting process id: 0xab4
Faulting application start time: 0x01da73d88fc63ed9
Faulting application path: C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\pbautobuild220.exe
Faulting module path: C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBSHR.dll
Report Id: 1e251ece-5f70-4dc8-9251-e8d56fd10fa8
Faulting package full name:
Faulting package-relative application ID:

Then there is this error (.wer file) (of which there were 3 others like this generated within 5 seconds):

Version=1
EventType=APPCRASH
EventTime=133546520191648675
ReportType=2
Consent=1
UploadTime=133546520344651424
ReportStatus=524388
ReportIdentifier=2a1d1a5a-eacc-422e-8c20-5f9beda680fb
IntegratorReportIdentifier=1e251ece-5f70-4dc8-9251-e8d56fd10fa8
Wow64Host=34404
Wow64Guest=332
NsAppName=pbautobuild220.exe
AppSessionGuid=00000ab4-0002-0048-d93e-c68fd873da01
TargetAppId=W:0006aed39009b32b694d7404d71990e796ef00000408!0000c9019293f4fb87e7c45b0534e27392b192d0f9e6!pbautobuild220.exe
TargetAppVer=2023//10//19:14:23:32!33311!pbautobuild220.exe
BootId=4294967295
ServiceSplit=11002852
TargetAsId=114161
IsFatal=1
EtwNonCollectReason=1
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=pbautobuild220.exe
Sig[1].Name=Application Version
Sig[1].Value=22.1.0.2828
Sig[2].Name=Application Timestamp
Sig[2].Value=65313be4
Sig[3].Name=Fault Module Name
Sig[3].Value=PBSHR.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=22.1.0.2828
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=6531453b
Sig[6].Name=Exception Code
Sig[6].Value=c0000005
Sig[7].Name=Exception Offset
Sig[7].Value=001297b0
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=10.0.19045.2.0.0.256.48
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=2f78
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=2f78a8cbca46fff25388ef5da0c24863
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=1979
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=19793605beeac4112c0b77ea147bd3ac
UI[2]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\pbautobuild220.exe
LoadedModule[0]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\pbautobuild220.exe
LoadedModule[1]=C:\WINDOWS\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\WINDOWS\System32\KERNEL32.DLL
LoadedModule[3]=C:\WINDOWS\System32\KERNELBASE.dll
LoadedModule[4]=C:\WINDOWS\System32\USER32.dll
LoadedModule[5]=C:\WINDOWS\System32\win32u.dll
LoadedModule[6]=C:\WINDOWS\System32\GDI32.dll
LoadedModule[7]=C:\WINDOWS\System32\gdi32full.dll
LoadedModule[8]=C:\WINDOWS\System32\msvcp_win.dll
LoadedModule[9]=C:\WINDOWS\System32\ucrtbase.dll
LoadedModule[10]=C:\WINDOWS\System32\ADVAPI32.dll
LoadedModule[11]=C:\WINDOWS\System32\msvcrt.dll
LoadedModule[12]=C:\WINDOWS\System32\sechost.dll
LoadedModule[13]=C:\WINDOWS\System32\RPCRT4.dll
LoadedModule[14]=C:\WINDOWS\System32\bcrypt.dll
LoadedModule[15]=C:\WINDOWS\System32\SHELL32.dll
LoadedModule[16]=C:\WINDOWS\System32\ole32.dll
LoadedModule[17]=C:\WINDOWS\System32\combase.dll
LoadedModule[18]=C:\WINDOWS\System32\SHLWAPI.dll
LoadedModule[19]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\MSVCP140.dll
LoadedModule[20]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\VCRUNTIME140.dll
LoadedModule[21]=C:\WINDOWS\System32\IMM32.DLL
LoadedModule[22]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBDevProxy.dll
LoadedModule[23]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\ijwhost.dll
LoadedModule[24]=C:\Program Files (x86)\dotnet\host\fxr\6.0.26\hostfxr.dll
LoadedModule[25]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\hostpolicy.dll
LoadedModule[26]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\coreclr.dll
LoadedModule[27]=C:\WINDOWS\System32\OLEAUT32.dll
LoadedModule[28]=C:\WINDOWS\System32\bcryptPrimitives.dll
LoadedModule[29]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Private.CoreLib.dll
LoadedModule[30]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\clrjit.dll
LoadedModule[31]=C:\WINDOWS\SYSTEM32\kernel.appcore.dll
LoadedModule[32]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Runtime.dll
LoadedModule[33]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Threading.dll
LoadedModule[34]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Runtime.Extensions.dll
LoadedModule[35]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Runtime.InteropServices.dll
LoadedModule[36]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Collections.NonGeneric.dll
LoadedModule[37]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Runtime.CompilerServices.VisualC.dll
LoadedModule[38]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Text.Json.dll
LoadedModule[39]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Memory.dll
LoadedModule[40]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Text.Encoding.Extensions.dll
LoadedModule[41]=C:\WINDOWS\SYSTEM32\icu.dll
LoadedModule[42]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Numerics.Vectors.dll
LoadedModule[43]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Runtime.CompilerServices.Unsafe.dll
LoadedModule[44]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Collections.dll
LoadedModule[45]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Collections.Concurrent.dll
LoadedModule[46]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Text.Encodings.Web.dll
LoadedModule[47]=C:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App\6.0.26\System.Runtime.Intrinsics.dll
LoadedModule[48]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBCMP.dll
LoadedModule[49]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBSHR.dll
LoadedModule[50]=C:\WINDOWS\System32\COMDLG32.dll
LoadedModule[51]=C:\WINDOWS\System32\shcore.dll
LoadedModule[52]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBVM.dll
LoadedModule[53]=C:\WINDOWS\System32\CRYPT32.dll
LoadedModule[54]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBDWE.dll
LoadedModule[55]=C:\WINDOWS\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.19041.3636_none_c0df324c38bbc0ce\COMCTL32.dll
LoadedModule[56]=C:\WINDOWS\SYSTEM32\WINSPOOL.DRV
LoadedModule[57]=C:\WINDOWS\SYSTEM32\WSOCK32.dll
LoadedModule[58]=C:\WINDOWS\System32\WS2_32.dll
LoadedModule[59]=C:\WINDOWS\WinSxS\x86_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.19041.3996_none_d954cb49e10154a6\gdiplus.dll
LoadedModule[60]=C:\WINDOWS\SYSTEM32\MSIMG32.dll
LoadedModule[61]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBUIS.dll
LoadedModule[62]=C:\WINDOWS\SYSTEM32\VERSION.dll
LoadedModule[63]=C:\WINDOWS\SYSTEM32\oledlg.dll
LoadedModule[64]=C:\WINDOWS\SYSTEM32\UIAutomationCore.DLL
LoadedModule[65]=C:\WINDOWS\SYSTEM32\UxTheme.dll
LoadedModule[66]=C:\WINDOWS\SYSTEM32\OLEACC.dll
LoadedModule[67]=C:\WINDOWS\SYSTEM32\WINMM.dll
LoadedModule[68]=C:\WINDOWS\SYSTEM32\PROPSYS.dll
LoadedModule[69]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\pbresource.dll
LoadedModule[70]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\pbjson.dll
LoadedModule[71]=C:\WINDOWS\SYSTEM32\windows.storage.dll
LoadedModule[72]=C:\WINDOWS\SYSTEM32\Wldp.dll
LoadedModule[73]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\LibPSCDPEnv.dll
LoadedModule[74]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\pbcrypt.dll
LoadedModule[75]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PbCompile.dll
LoadedModule[76]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBORC.dll
LoadedModule[77]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBDEV.dll
LoadedModule[78]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBSYS.dll
LoadedModule[79]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBLIB.dll
LoadedModule[80]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBAPL.dll
LoadedModule[81]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBCSI.dll
LoadedModule[82]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBDTS.dll
LoadedModule[83]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBDWP.dll
LoadedModule[84]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBEAS.dll
LoadedModule[85]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBSCR.dll
LoadedModule[86]=C:\WINDOWS\SYSTEM32\WTSAPI32.dll
LoadedModule[87]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBUDO.dll
LoadedModule[88]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBGShare.dll
LoadedModule[89]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBSQL.dll
LoadedModule[90]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBDPP.dll
LoadedModule[91]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\libjcc.dll
LoadedModule[92]=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\PBWED.dll
LoadedModule[93]=C:\WINDOWS\SYSTEM32\ACTIVEDS.dll
LoadedModule[94]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\PBDPL.dll
LoadedModule[95]=C:\WINDOWS\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.9672_none_d08f9da24428a513\MSVCR80.dll
LoadedModule[96]=C:\WINDOWS\SYSTEM32\adsldpc.dll
LoadedModule[97]=C:\WINDOWS\System32\WLDAP32.dll
LoadedModule[98]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\libjutils.dll
LoadedModule[99]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\libjtml.dll
LoadedModule[100]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\nlwnsck.dll
LoadedModule[101]=C:\Program Files (x86)\Appeon\Common\PowerBuilder\Runtime 22.1.0.2828\libsybunic.dll
LoadedModule[102]=C:\Program Files (x86)\Appeon\PowerBuilder 22.0\IDE\MainViewInterop.dll
OsInfo[0].Key=vermaj
OsInfo[0].Value=10
OsInfo[1].Key=vermin
OsInfo[1].Value=0
OsInfo[2].Key=verbld
OsInfo[2].Value=19045
OsInfo[3].Key=ubr
OsInfo[3].Value=4046
OsInfo[4].Key=versp
OsInfo[4].Value=0
OsInfo[5].Key=arch
OsInfo[5].Value=9
OsInfo[6].Key=lcid
OsInfo[6].Value=1033
OsInfo[7].Key=geoid
OsInfo[7].Value=244
OsInfo[8].Key=sku
OsInfo[8].Value=48
OsInfo[9].Key=domain
OsInfo[9].Value=1
OsInfo[10].Key=prodsuite
OsInfo[10].Value=256
OsInfo[11].Key=ntprodtype
OsInfo[11].Value=1
OsInfo[12].Key=platid
OsInfo[12].Value=10
OsInfo[13].Key=sr
OsInfo[13].Value=0
OsInfo[14].Key=tmsi
OsInfo[14].Value=222574753
OsInfo[15].Key=osinsty
OsInfo[15].Value=3
OsInfo[16].Key=iever
OsInfo[16].Value=11.3636.19041.0-11.0.1000
OsInfo[17].Key=portos
OsInfo[17].Value=0
OsInfo[18].Key=ram
OsInfo[18].Value=16384
OsInfo[19].Key=svolsz
OsInfo[19].Value=79
OsInfo[20].Key=wimbt
OsInfo[20].Value=0
OsInfo[21].Key=blddt
OsInfo[21].Value=191206
OsInfo[22].Key=bldtm
OsInfo[22].Value=1406
OsInfo[23].Key=bldbrch
OsInfo[23].Value=vb_release
OsInfo[24].Key=bldchk
OsInfo[24].Value=0
OsInfo[25].Key=wpvermaj
OsInfo[25].Value=0
OsInfo[26].Key=wpvermin
OsInfo[26].Value=0
OsInfo[27].Key=wpbuildmaj
OsInfo[27].Value=0
OsInfo[28].Key=wpbuildmin
OsInfo[28].Value=0
OsInfo[29].Key=osver
OsInfo[29].Value=10.0.19041.4046.amd64fre.vb_release.191206-1406
OsInfo[30].Key=buildflightid
OsInfo[30].Value={2F4769CE-9307-4C93-B075-ECF286013FCA}.201
OsInfo[31].Key=edition
OsInfo[31].Value=Professional
OsInfo[32].Key=ring
OsInfo[33].Key=expid
OsInfo[34].Key=fconid
OsInfo[35].Key=containerid
OsInfo[36].Key=containertype
OsInfo[37].Key=edu
OsInfo[37].Value=0
File[0].CabName=WERInternalMetadata.xml
File[0].Path=WER43B8.tmp.WERInternalMetadata.xml
File[0].Flags=327683
File[0].Type=5
File[0].Original.Path=\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER43B8.tmp.WERInternalMetadata.xml
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=Appeon PowerBuilder 2022 R2
AppPath=C:\Program Files (x86)\Appeon\PowerBuilderUtilities 22.0\pbautobuild220.exe
NsPartner=windows
NsGroup=windows8
ApplicationIdentity=A298B35507A56A20565F5C958C501653
MetadataHash=-1325942284

Thank you.

MJ