Error loading large JSON files using Scala Play Framework 2 - json

Question

I'm trying to use Apache Bench to load test a group of large (4MB each) JSON requests. When running with a large file and many concurrent requests I get the following error:
Exception caught in RequestBodyHandler java.nio.channels.ClosedChannelException: null
Here is my ab command:
ab -p large.json -n 1000 -c 10 http://127.0.0.1:9000/json-tests
If I run this with no concurrency and only 10 requests it works fine. Increasing the number of requests or the concurrency causes this error to occur over and over.
My controller currently has no logic in it:
def addJsonTest = Action {
Ok("OK")
}
Here is the full error:
[error] play - Exception caught in RequestBodyHandler
java.nio.channels.ClosedChannelException: null
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.setInterestOps(AbstractNioWorker.java:506) [netty-3.9.3.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker$1.run(AbstractNioWorker.java:455) [netty-3.9.3.Final.jar:na]
at org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:40) [netty-3.9.3.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:372) [netty-3.9.3.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:296) [netty-3.9.3.Final.jar:na]
This is just using Play in development mode, is there any setup or configuration for having Play handle multiple large requests?
Thanks!

Answer 1

You need to do it reactively, with Iterators
val iteratee = Iteratee.foldM[Array[Byte], Either[Result, String]](Right("start")) { case (str, bytes) =>
Future.successful(Left(Ok))
}
val parser = BodyParser(rh => iteratee)
def eatDust = Action(parser) { req =>
Ok
}
See these links.
https://www.playframework.com/documentation/2.2.x/Iteratees
Play 2.x : Reactive file upload with Iteratees