Coldfusion server error - Corrupt form data: premature ending

[PeteComcar]

I've got a problem with our live server. Occasionally a page will not load, and the following error is displayed in plain black text on a white background:

Server Error
The server encountered an internal error and was unable to complete your request
Application server is busy. Either there are too many concurrent requests

If I look in the error log for the website, at the time of the error using the following command:

grep "12:17:33" error_log

lines such as this are returned:

[Tue May 03 12:17:33 2011] [notice] jrApache[14352: 21512] returning error page for JRun too busy or out of memory

Then if I search in the Coldfusion cfserver.log I find these errors, a few seconds before hand:

05/03 12:17:29 error Corrupt form data: premature ending
java.io.IOException: Corrupt form data: premature ending
at com.oreilly.servlet.multipart.MultipartParser.<init>(MultipartParser.java:177)
at com.oreilly.servlet.multipart.MultipartParser.<init>(MultipartParser.java:99)
at coldfusion.filter.FormScope.fillMultipart(FormScope.java:190)
at coldfusion.filter.FusionContext.SymTab_initForRequest(FusionContext.java:369)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:33)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:53)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126)
at coldfusion.CfmServlet.service(CfmServlet.java:200)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

05/03 12:17:29 error (JRun Service: ProxyService [jrun.servlet.jrpp.JRunProxyService@500b675]) JRunPRoxyServer.invokeRunnable:
java.lang.IllegalStateException
at jrun.servlet.JRunResponse.getWriter(JRunResponse.java:205)
at jrun.servlet.JRunResponse.sendError(JRunResponse.java:597)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:328)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

java.lang.IllegalStateException
at jrun.servlet.JRunResponse.getWriter(JRunResponse.java:205)
at jrun.servlet.JRunResponse.sendError(JRunResponse.java:597)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:328)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

I don't really know what to do from here.

If the page is refreshed then it loads without error. I don't know how to replicate the error, it doesn't seem to happen on any particular page or at any articular time.

I've tried setting:

Maximum number of simultaneous Template requests: 100

in the Coldfusion administrator, I've done that based on Steven Erat's post: http://www.talkingtree.com/blog/index.cfm/2005/3/11/ContextSwitchingBad

But I'm still getting errors. Would increasing this to a higher value help?

And then there's the "premature form ending" part of the longer Java error, I'm not sure if that's related and don't understand what it means anyway.

I've also looked through my access logs, and ruling out any images or js files that get called at the time of any errors, I've got a variety of CFM pages that return a 503 http status, and some of them are CFM pages that get returned with a content type of Javascript, and are never requested after a form post.

So in summary, I'm getting some pretty blunt error messages, and I can reproduce them at present, and the page loads fine if I just hit F5, and I'd like to know if anyone can help me to understand the source of the problem.

Oh and one last thing, we're running a RHEL 5 server with CF9, and we've upgraded recently from CF7. Before the upgrade this error was not happening.

[cfwebdeveloper]

Make sure the CF Admin settings from CF7 carried over to CF9.

[PeteComcar]

All my settings were carried over, that wasn't the problem.

The issue seems to have been resolved, although I can't put my finger on why.

I installed FusionReactor to try and better monitor the server while the error was happening, and in doing so had to update my JVM and restart CF9. Since then, I've not had the error.

My guess is the newer JVM, although I suppose it might have been the CF9 reboot, perhaps some settings I'd made hadn't take effect yet and that kicked them in. I very much doubt installing FusionReactor fixed it, but I really can't say that it didn't.

[cfwebdeveloper]

Yeah, keep an eye on it. If it comes back let us know. I've seen this happen but without a real solution.