Sorry for the quick version-up. Version 0.1.1 added non-blocking I/O, but was using Dispatch without configuring the threadpool used for HTTP connections. This caused issues on Heroku where there is a 256 combined thread + process limit for 1x dynos (512 for 2x dynos), whereby Java OOM “unable to create new native thread” errors would be thrown.

0.2.1 adds:

  • Configuration of Actor HTTP client on ScraperActor instantiation
    • Notably: HTTP client ExecutorService thread pool

Relevent info:

Comments