Last week, I decided to take a stab at learning Scala macros. I had played around with macros when I wrote Scheme for a living (yes, believe it or not, these places exist…and existed long before Clojure made Lisp hip again), but the complexity of Scala’s macros always put me off (if you don’t believe me, check out the example given in the offical docs for a simple print macro).

In Scala, things are not so simple, but with the introduction of quasiquotes and some refinements brought by Scala 2.11, things are smoother. Still, for a guy like me, the documentation was both sparse and DRY. Since I learn best when I’m actively engaged in building something, I decided to try writing the run-of-the-mill unless-when macros in Scala.

This post aims to summarise my journey towards implementing unless-when and hopefully along the way make Scala macros accessible, at least at an introductory level, for Most People. There are already a few Scala macro blog posts out there but another one can’t hurt.

A couple days ago, I released v0.1.3 of Schwatcher, which introduces the ability to monitor events on file paths using a composable Rx Observable interface. “What does that even mean and why should you care?” is what this blog post tries to answer.

The original version of Schwatcher allowed you to tell a MonitorActor what callback you want to fire when a certain type of event happened on a file path. This is fine and there are people out there using it in production as is. The limitation to this approach is that (at least by default), the events are difficult to treat as data and thus difficult to compose.

With Rx, we turn file path events into an asynchronous stream/channel. Essentially, you tell a RxMonitor object what path and event type you want to monitor and when an event happens, it will get pushed into its observable (the stream). You can then choose to filter, map, or fold over this data stream, creating new data streams. If you wish to cause side-effects, you can add one or more observers to these data streams.

Version 0.1.3 of Schwatcher has been released.

This version brings a new Observable interface that exposes a “stream” (or channel) of EventAtPaths that can be composed. Using this interface, you no longer need to register callbacks - you simply register paths and get notifications for events on them either by subscribing to the Observable or by composing.

For more information on how to use Observables (especially how they compose in awesome ways), checkout the Rx homepage

Example usage:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import com.beachape.filemanagement.RxMonitor
import java.io.{FileWriter, BufferedWriter}

import java.nio.file.Paths
import java.nio.file.StandardWatchEventKinds._

val monitor = RxMonitor()
val observable = monitor.observable

val subscription = observable.subscribe(
  onNext = { p => println(s"Something was modified in a file mufufu: $p")},
  onError = { t => println(t)},
  onCompleted = { () => println("Monitor has been shut down") }
)

val desktopFile = Paths get "/Users/lloyd/Desktop/test"

monitor.registerPath(ENTRY_MODIFY, desktopFile)

Thread.sleep(100)

//modify a monitored file
val writer = new BufferedWriter(new FileWriter(desktopFile.toFile))
writer.write("Theres text in here wee!!")
writer.close

// #=> Something was modified in a file mufufu: /Users/lloyd/Desktop/test

// stop monitoring
monitor.stop()

// #=> Monitor has been shut down

Relevant links: - Github page with how to install and example usage - Release page

Goal

Create a backported Ruby 1.9.3p484 Ubuntu package. For the purposes of this post, we will assume that you are OK with not creating a signed package (required if uploading to launchpad). If you wish to upload to launchpad (for example to share your backport with the world), it may be useful for you to check out this site after reading through this guide.

Motivation

At my day job, I had to upgrade Ruby on very old production boxes that were running Oneiric AMD64. Upgrading the OS was a chore that would be happening later but we needed to start using the latest Ruby as soon as possible.

Unfortunately, there is no Ruby 1.9.3p484 Debian package readily available for Oneiric, not even as a backport.

Version 0.0.7 of Schwatcher has been released.

Changes:

  • No longer has external logging library dependencies. Thanks crdueck for the suggestion.
  • No longer uses any mutable members for extra thread-safety

Relevent info:

Version 0.0.4 of Schwatcher has been released.

Changes:

  • No longer uses Akka Agent to hold CallbackRegistry (thanks crdueck). This should result in a small performance increase because of more ‘direct’ memory access inside MonitorActor.
  • Refactored testing for better coverage and maintainability
  • Scala 2.10.3 support in testing

Relevent info:

For the most part, I’m a very happy Heroku user. The platform allows me deploy my apps, be they Java, Scala or Ruby-based without having to think/worry about infrastructure, which is amazing. They also allow me to, for the most part, do this for free. For the most part, I love it, and so do many others. That said, sometimes you do run into problems that cause you to lose lots of time debugging … in the wrong direction.

Yesterday, after adding non-blocking I/O to my Metascraper library, I load tested my deployed application and found it fatally crashing with java.lang.OutOfMemoryError: unable to create new native thread errors. Not Good

Sorry for the quick version-up. Version 0.1.1 added non-blocking I/O, but was using Dispatch without configuring the threadpool used for HTTP connections. This caused issues on Heroku where there is a 256 combined thread + process limit for 1x dynos (512 for 2x dynos), whereby Java OOM “unable to create new native thread” errors would be thrown.

0.2.1 adds:

  • Configuration of Actor HTTP client on ScraperActor instantiation
    • Notably: HTTP client ExecutorService thread pool

Relevent info: