IIS-10 worker processes crashing - reporting-services

We recently upgraded our server O/S from Win2012 to Win2019. While on 2012, in IIS, we had configured a Virtual Directory called PDFReader. The physical path was a filestream directory which contains a series of PDFs, which is essentially just a UNC share.
The directory was being called by an SSRS report & its sole purpose was to call the PDF from filestream and render it in SSRS.
We used a simple virtual directory when we were on Windows 2012 which worked fine. But we noticed that the app pool kept crashing on 2019. So that led to me convert it to an application because that way, I could assign an individual app pool to it & manage the rapid fail protection settings as a band-aid for now until we could identify the underlying issue. That has been getting us by.
I tried a few things such as changing the application pool identity to LocalSystem, then a privileged domain user w/SQL Filestream access. I also tried changing the .NET CLR Version to "No Managed Code" and the pipeline from Integrated to Classic. The latter seemed to slow the failures slightly, but nevertheless the problem persists.
Here's the stacktrace which we consistently see every few minutes as users frequently pull up PDFs.
Faulting application name: w3wp.exe, version: 10.0.17763.1, time stamp: 0xa7ccada2
Faulting module name: nativerd.dll, version: 10.0.17763.1, time stamp: 0x84a81d7a
Exception code: 0xc0000005
Fault offset: 0x0000a8f0
Faulting process id: 0x25cc
Faulting application start time: 0x01d921151b65f242
Faulting application path: C:\Windows\SysWOW64\inetsrv\w3wp.exe
Faulting module path: C:\Windows\System32\inetsrv\nativerd.dll
Report Id: 88535cc3-8716-40f7-bfab-1f53f9c77b92
Faulting package full name:
Faulting package-relative application ID:
Fault bucket 1192683016347496699, type 1
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: w3wp.exe
P2: 10.0.17763.1
P3: a7ccada2
P4: nativerd.dll
P5: 10.0.17763.1
P6: 84a81d7a
P7: c0000005
P8: 0000a8f0
P9:
P10:
There are a number of PDFs that are called which one might consider large, over 20MB with a handful of them over 100mb.
As for the user experience, since I implemented the rapid fail protection, the user doesn't run into the 503 errors anymore, requiring me to start the stopped app pool. Occasionally, when the worker process dies & hands off to the next one spawned, it's a minor delay.
Any help or insight would be appreciated.
Thanks,
Mike
Converted virtual directory to application.
Created/assigned individual app pool to application.
Changed rapid fail protection settings to allow for 25 failures in 10 minutes instead of the 5/5 out of the box.
Changed application pool identity to LocalSystem... Problem persisted. So I changed to a privileged domain user/local admin with proper SQL filestream rights, same behavior.
Ensured nativerd.dll was present in system directory, no discernible permissions issues.
Looked into MaxWebConfigFileSizeInKB registry key associated with nativerd.dll per a Microsoft KB article, but I don't see that key present in the registry.
Ran DebugDiag analysis with this summary result:
In w3wp__PDFReader__PID__160__Date__06_15_2022__Time_08_39_35AM__249__Second_Chance_Exception_C0000005.dmp the assembly instruction at nativerd!INDEXED_ARRAY_LIST::FindEntry+18 in c:\Windows\System32\inetsrv\nativerd.dll from Microsoft Corporation has caused an access violation exception (0xC0000005) when trying to read from memory location 0x00000038 on thread 6

Related

Getting Assemble script failed error building application in Redhat Openshift

I'm new to Redhat OpenShift and trying to deploy node application (with angularjs + mysql) and running into build issues.
Using openshift console created node application and in advanced options pointed to the private repository and linked configured secret (ssh key to private repository).
My build is failing with "Assemble script failed". Pasting the logs as below (from console - obfuscated private keys and values).
Not sure if I'm missing some configurations. Appreciate help on this.
Cloning "ssh://username#bitbucket.org/username/my-app.git" ...
Commit: xxxxxxxxxxxxxxxxx (Fixed readme)
Author: Name <email>
Date: Wed Sep 6 19:50:59 2017 -0700
Pulling image "docker-
registry.default.svc:5000/openshift/nodejs#sha256:0000000000000" ...
---> Installing application source
---> Building your Node application from source
Current git config
url.https://github.com.insteadof=git#github.com:
url.https://.insteadof=ssh://
url.https://github.com.insteadof=ssh://git#github.com
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=ssh://username#bitbucket.org/username/my-app.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
---> Installing dependencies
---> Using 'npm install -s --only=production'
error: build error: non-zero (13) exit code from docker-
registry.default.svc:5000/openshift/nodejs#sha256:0000000000000
Pls note that my source code is hosted on private repository and per log above it appears openshift is able to access the repository and download the source code.
Thanks Graham for the pointer. I recreated the application and this time in advanced options on the web console selected 1 GB (from default of 500 mb) after which my build worked fine.

Windows 8.1 universal app crashes on Windows 10

My app is already published in the Store, it was built on Windows 8.1 and it perfectly works on 8.1 (desktop and tablet). However it crashes on Windows 10 just after I navigate from page that has a WebView control (though this might not be the reason). I looked at windows logs (eventvwr.msc) and found out the crash has happened in a system library.
Has anybody faced problems with Windows 8.1 apps running on Windows 10?
UPDATE 1: information from system's event log
Faulting application name: ....
Faulting module name: twinapi.appcore.dll, version: 10.0.10240.16397, time stamp: 0x55af0938
Exception code: 0xc000027b
Fault offset: : 0x0001cab9
Faulting process id: 0xe30
Faulting application start time: 0x01d0daabb7ab6d89
Faulting application path: C:\Program Files\WindowsApps.....
Faulting module path: C:\Windows\System32\twinapi.appcore.dll
Faulting package full name: ....
Faulting package-relative application ID: App
UPDATE 2
With help of this article Debugging a Windows 8.1 Store App Crash Dump I've cracked crash dump and found out the problem method in my code. That method has code that captures a screenshot of a web page with webView.CapturePreviewToStreamAsync(memoryStream); and saves it to local file. I suspect webview causes the crash because its core should differ from one on windows 8.1 (is it based on MS Edge instead of IE?).

Harmon.ie for SharePoint Exception on Init - Registry Access

I have a user who has successfully installed Harmon.ie to the computer, but the log file is showing an "Exception on Init" error. The complete error is below:
Starting Harmonie
2015-01-16 06:48:57,172 INFO [1] Mainsoft.Harmony.AddIn.HarmonyAddIn - OnConnection
2015-01-16 06:48:57,747 INFO [1] Mainsoft.Harmony.AddIn.HarmonyAddIn - Load Harmonie version: 3.4.0.9975 Outlook Version 14.0.0.7105 X86 bit
2015-01-16 06:48:57,748 ERROR [1] Mainsoft.Harmony.AddIn.HarmonyAddIn - Exception on Init
System.Security.SecurityException: Requested registry access is not allowed.
at System.ThrowHelper.ThrowSecurityException(ExceptionResource resource)
at Microsoft.Win32.RegistryKey.OpenSubKey(String name, Boolean writable)
at Mainsoft.Harmony.Utils.RedemptionCleaner.CleanOrphanRegistryKeys()
at Mainsoft.Harmony.AddIn.HarmonyAddIn.Init()
The Zone of the assembly that failed was:
MyComputer
Does anyone know the registry key that is trying to be accessed?
Harmon.ie needs to access at the stage reported in the stack traces you have originally attached the following keys under HKEY_CURRENT_USER\SOFTWARE\Classes
"Redemption.Addin",
"Redemption.AddressLists",
"Redemption.MAPIFolder",
"Redemption.MAPITable",
"Redemption.MAPIUtils",
"Redemption.RDOSession",
"Redemption.SafeAppointmentItem",
"Redemption.SafeContactItem",
"Redemption.SafeCurrentUser",
"Redemption.SafeDistList",
"Redemption.SafeInspector",
"Redemption.SafeJournalItem",
"Redemption.SafeMailItem",
"Redemption.SafeMeetingItem",
"Redemption.SafePostItem",
"Redemption.SafeReportItem",
"Redemption.SafeTaskItem",
"Mainsoft.Harmony.OutlookSidebar.TaskPaneSidebarAx"
Outlook should run under an account with sufficient privileges to read\write under HKEY_CURRENT_USER.
By the way, you are running a quite old version of Harmon.ie, I suggest you download and install the latest 4.5 version from harmon.ie web site.
--- Jean
This error indicates the user did not have sufficient permission to access this registry key.
Do you reproduce it when running Outlook as admin? (please try also to install as admin)
--- Jean

Play TypeSafe Activator fails to start - IllegalArgumentException "Failed to download new template catalog properties"

Moving from play 2.2.x to latest activator last night. Downloaded minimal 1.2.10, extracted it in program file (x86)\typesafe... and put the directory into the system path variable. cloned my repository, and when i executed activator run it downloaded the required modules and my app is up and running. All great so far. run works!
Then I tried to create a new app, and activator fails, with the following trace:
Checking for a newer version of Activator (current version 1.2.10)...
... our current version 1.2.10 looks like the latest.
Found previous process id: 9632
FOUND REPO = activator-local # file:////C:/Program%20Files%20(x86)/Typesafe/activator-1.2.10-minimal/repository
Play server process ID is 9760
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /127.0.0.1:8888
[info] a.e.s.Slf4jLogger - Slf4jLogger started
[WARN] [10/30/2014 10:47:13.972] [default-akka.actor.default-dispatcher-2] [ActorSystem(default)] Failed to download new template ca
talog properties: java.lang.IllegalArgumentException: requirement failed: Source file 'C:\Users\admin\.activator\1.2.10\templates\in
dex.db_6e0565f0c8826b17.tmp' is a directory.
[ERROR] [10/30/2014 10:47:13.972] [default-akka.actor.default-dispatcher-2] [akka://default/user/template-cache] Could not find a te
mplate catalog. (activator.templates.repository.RepositoryException: We don't have C:\Users\admin\.activator\1.2.10\templates\cache.
properties with an index hash in it, even though we should have downloaded one
activator.templates.repository.RepositoryException: We don't have C:\Users\admin\.activator\1.2.10\templates\cache.properties with a
n index hash in it, even though we should have downloaded one
at activator.cache.TemplateCacheActor.preStart(TemplateCacheActor.scala:184)
at akka.actor.Actor$class.aroundPreStart(Actor.scala:470)
at activator.cache.TemplateCacheActor.aroundPreStart(TemplateCacheActor.scala:25)
at akka.actor.ActorCell.create(ActorCell.scala:580)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
I've taken a look at several similar issues on SO and elsewhere. I've deleted .activator directory and retried, I've tried this process from behind a proxy and not, as well as offline (surely offline should work!), but it consistently gives the above error. activator ui gives the same error. I'm stuck and any suggestions would be appreciated. (Edit. tried with full activator download, rather than minimal, and I get the same error.)
Look for reasons it might be impossible to create or access 'C:\Users\admin.activator\1.2.10\templates\in
dex.db_6e0565f0c8826b17.tmp' ... maybe a permissions issue?
The failed check is for "is a directory" but that also fails if it just doesn't exist or can't be accessed.

System.Security.SecurityException being thrown on initializing StructureMap bootstrapper

I am running a .NET 4.0 web application locally using the Visual Studio Development Server (built in web server with VS2010), and for the last couple months, my StructureMap bootstrapper file has worked perfectly.
I'm using StructureMap 2.6.1
I have not changed the Bootstrapper file or the Web.config file, and suddenly, I'm getting this strange error when trying to start up my web application.
Here is the error being thrown from the website:
it's a bit tough to read. It says:
Security Exception
Description: The application attempted to perform an operation not allowed by the security policy. To grant this application the required permission please contact your system administrator or change the application's trust level in the configuration file.
Exception Details: System.Security.SecurityException. Request Failed.
here is the code in my bootstrapper file:
public class BootStrapper
{
public static void ConfigureStructureMap()
{
ObjectFactory.Initialize(x =>
{
x.AddRegistry<ProductRegistry>();
});
}
}
public class ProductRegistry : Registry
{
public ProductRegistry()
{
For<IFirmRepository>()
.Use<FirmRepository>().Ctor<string>("connectionString").Is(ConfigurationManager.ConnectionStrings["FeesAndFlows"].ConnectionString);
For<ICryptographyService>()
.Use<Rijndael>();
For<IUserRepository>()
.Use<UserRepository>().Ctor<string>("connectionString").Is(ConfigurationManager.ConnectionStrings["FeesAndFlows"].ConnectionString);
For<IAuthenticationService>()
.Use<AuthenticationService>();
For<ILogger>()
.Use<DatabaseLogger>();
}
}
The error is being thrown on this line:
x.AddRegistry<ProductRegistry>();
I've already tried adding each of these lines to my Web.config file, one at a time, and they didn't fix the problem:
<trust level="Full" />
and
<securityPolicy>
<trustLevel name="Full" policyFile="internal"/>
</securityPolicy>
Does anyone have any ideas or might have heard of a problem like this? It's pretty important, b/c without StructureMap starting up correctly, my entire application will not run.
Thanks,
Mike
UPDATE:
Ok, so it appears my problem is local. Other developers here can download the code, and runs it just fine on their local machines. Weird. They bootstrap StructurMap just fine and all instances are resolved...
Any ideas on why just my machine can't bootstrap StructureMap when running in debug mode locally for my web project?
Are you running the code from a network drive by any chance (ie is your documents folder redirected onto a network drive)? Are you in an enterprise environment running on a domain?
If the former it's likely that the code is running in the intranet security context. If the former isn't true but the later is, then its quite possible that a network administrator has changed the enterprise wide CAS policy.
Okay, so this was the problem. The StructureMap.dll was blocked by Win 7. I don't know how is become blocked or where it became blocked, but apparently, when I downloaded the StructureMap.zip file onto my system, the .zip file was blocked, which in turn, led to all the items extracted from the .zip file being blocked as well.
Every time I unblocked it, it went back to blocked when I tried to run the web app.
The way I fixed it was to go back to the original .zip file, unblock it, extract it, and then replace my StruectureMap.dll reference with one that was not blocked.
Insane.
I don't even know WHAT causes file to suddenly become blocked or what process in Windows 7 determines what file(s) should be blocked, but this strange operating system "feature" cost me a day's worth of work.
I had the exact same issue at the exact same place, ObjectFactory.Initialize:
Server Error in '/X.ServiceHost' Application.
Inheritance security rules violated while overriding member:
'StructureMap.StructureMapException.GetObjectData(System.Runtime.Serialization.SerializationInfo,
System.Runtime.Serialization.StreamingContext)'. Security
accessibility of the overriding method must match the security
accessibility of the method being overriden.
Description: An unhandled exception occurred during the execution of
the current web request. Please review the stack trace for more
information about the error and where it originated in the code.
Exception Details: System.TypeLoadException: Inheritance security
rules violated while overriding member:
'StructureMap.StructureMapException.GetObjectData(System.Runtime.Serialization.SerializationInfo,
System.Runtime.Serialization.StreamingContext)'. Security
accessibility of the overriding method must match the security
accessibility of the method being overriden.
Getting the latest StructureMap package from NuGet (2.6.4.1) fixed the issue.
StructureMap NuGet Package