The ClojureWerkz Blog

News and updates about ClojureWerkz projects

Mailer 1.2.0 Is Released

TL;DR

ClojureWerkz Mailer is an ActionMailer-inspired mailer library for Clojure.

1.2.0 is a minor feature release.

Changes Between 1.1.0 and 1.2.0

Improved Template Rendering Exceptions

Template rendering exceptions now have a better error message.p

Contributed by Lei.

Mailer is a ClojureWerkz Project

Mailer is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Cassaforte, a Clojure client for Cassandra built around CQL
  • Monger, a Clojure MongoDB client for a more civilized age
  • Welle, a Riak client with batteries included
  • Neocons, a Clojure client for the Neo4J REST API

and several others. If you like Mailer, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

About the Author

Michael on behalf of the ClojureWerkz Team.

Validateur 2.2.0 Is Released

TL;DR

Validateur is a functional validations library inspired by Ruby’s ActiveModel. Validateur 2.2 is a minor feature release.

Changes Between 2.1.0 and 2.2.0

nested

nested is a new validator runner for nested attributes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
(require '[validateur.validation :refer :all])

(let [v (vr/nested :user (vr/validation-set
                            (vr/presence-of :name)
                            (vr/presence-of :age)))
        extra-nested (vr/nested [:user :profile]
                                (vr/validation-set
                                 (vr/presence-of :age)
                                 (vr/presence-of [:birthday :year])))]
  (v {})
  ;= {[:user :age] #{"can't be blank"}
      [:user :name] #{"can't be blank"}}
  (v {:user {:name "name"}})
  ;= {[:user :age] #{"can't be blank"}}
  (extra-nested {:user {:profile {:age 10
                                  :birthday {:year 2004}}}})
  ;= {}
  (extra-nested {:user {:profile {:age 10}}})
  ;= {[:user :profile :birthday :year] #{"can't be blank"}}

Contributed by Sam Ritchie.

validate-by

validate-by is a new validator function. It returns a function that, when given a map, will validate that the + value of the attribute in that map is one of the given:

1
2
3
4
5
6
(require '[validateur.validation :refer :all])

(validation-set
   (presence-of :name)
   (presence-of :age)
   (validate-by [:user :name] not-empty :message \"Username can't be empty!\"))

Contributed by Sam Ritchie.

Full Change Log

Validateur change log is available on GitHub.

Validateur is a ClojureWerkz Project

Validateur is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Monger, a Clojure MongoDB client for a more civilized age
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Cassaforte, a Clojure Cassandra client built around CQL
  • Neocons, a client for the Neo4J REST API
  • Welle, a Riak client with batteries included
  • Quartzite, a powerful scheduling library

and several others. If you like Validateur, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

About The Author

Michael on behalf of the ClojureWerkz Team

Mailer 1.1.0 Is Released

TL;DR

ClojureWerkz Mailer is an ActionMailer-inspired mailer library for Clojure.

1.1.0 is a minor feature release.

Changes Between 1.0.0 and 1.1.0

Support for Alternative Email Bodies

build-email and deliver-email now take extra set of template, data, content-type for alternative email body. This is useful for supplying alternative plain-text body in addition to main HTML body of the message.

1
2
3
(build-email {:from "Joe The Robot", :to ["ops@megacorp.internal" "oncall@megacorp.internal"] :subject "Hello!"}
             "templates/html_hello.mustache" {:name "Joe"} :text/html
             "templates/hello.mustache" {:name "Joe"} :text/plain)

Contributed by Lei.

Mailer is a ClojureWerkz Project

Mailer is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Cassaforte, a Clojure client for Cassandra built around CQL
  • Monger, a Clojure MongoDB client for a more civilized age
  • Welle, a Riak client with batteries included
  • Neocons, a Clojure client for the Neo4J REST API

and several others. If you like Mailer, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

About the Author

Michael on behalf of the ClojureWerkz Team.

Quartzite 1.3.0 Is Released

Quartzite is a Clojure DSL on top of the Quartz scheduler.

1.3.0 is a minor release that contains minor improvements and upgrades dependencies.

Changes Between Quartzite 1.2.0 and 1.3.0

Clojure 1.6 by Default

The library now depends on Clojure 1.6.

clj-time upgraded to 0.8.0

clj-time dependency has been upgraded to version 0.8.0.

Changes between Quartzite 1.1.0 and 1.2.0

Clojure 1.3 is No Longer Supported

Quartzite requires Clojure 1.4+ as of this version.

clj-time upgraded to 0.6.0

clj-time dependency has been upgraded to version 0.6.0.

New Functions

  • clojurewerkz.quartzite.scheduler/get-currently-executing-jobs returns a set of currently executing jobs.
  • clojurewerkz.quartzite.scheduler/currently-executing-job? returns true there is a running job for a given key.

Change Log

Quartzite change log is available on GitHub.

Quartzite is a ClojureWerkz Project

Quartzite is part of the group of libraries known as ClojureWerkz, together with

  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB driver for a more civilized age
  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Welle, a Riak client with batteries included
  • Neocons, a client for the Neo4J REST API

and several others. If you like Quartzite, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

@michaelklishin on behalf of the ClojureWerkz Team

CHash 1.1.0 Is Released

TL;DR

Chash is a Clojure consistent hashing library ported from riak_core.

Changes Between 1.0.x and 1.1.0

Clojure 1.6 by Default

The library now depends on Clojure 1.6.

Change Log

Chash change log is available on GitHub.

Chash is a ClojureWerkz Project

Chash is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Cassaforte, a Clojure Cassandra client built around CQL
  • Monger, a Clojure MongoDB client for a more civilized age
  • Welle, a Riak client with batteries included
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Neocons, a client for the Neo4J REST API
  • Quartzite, a powerful scheduling library

and several others. If you like Chash, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Metrics-clojure 2.1.1 Is Released

TL;DR

metrics-clojure is a Clojure façade around Coda Hale’s Metrics library, originally developed by Steve Losh.

metrics-clojure is not a ClojureWerkz project but we help maintain it and consider it to be a very valuable library.

Changes Between 2.1.0 and 2.1.1

Ring Extension Now Respects User-provided Registry

The Ring extension now respects user-provided registries instead of always using the default one.

Contributed by David Smith.

Changes Between 2.0.x and 2.1.0

Ring Extension Supports Multiple Registries and Options

It is now possible to use metrics.ring.expose with a custom registry:

1
2
3
(require '[metrics.ring.expose :refer [expose-metrics-as-json]])

(expose-metrics-as-json ring-app "/ops/metrics" registry {:pretty-print? true})

JVM Instrumentation Extension

The project now also has a JVM instrumentation extension that covers:

  • Number of threads and their states
  • GC stats, heap, off heap memory
  • File descriptors

To enable full instrumenting, use

1
2
3
(require '[metrics.jvm.core :refer [instrument-jvm]])

(instrument-jvm metric-registry)

Contributed by John Cowie (ThoughtWorks).

timers/mean Returns Value (Not Rate)

metrics.timers/mean now returns mean value (not rate) of the timer.

Contributed by Steve Miner.

Ring Extension Updated for 2.0

The Ring extension is now updated for Metrics core 2.0 API.

Contributed by John Cowie (ThoughtWorks).

Changes Between 1.1.0 and 2.0.0

Metrics Registries

metrics-clojure 1.x maintained a metrics registry in a dynamic var. This approach makes the library a little easier for beginners but also much harder to use in more sophisticated cases, e.g. in concurrent applications or those that use a Component-like approach to program structure.

As such, metrics-clojure 2.0+ makes metrics registry a required explicit argument to most functions in the API:

1
2
3
4
5
6
7
8
9
10
(require '[metrics.meters :as meters])

;; with 1.x
(meters/rate-mean)
(meters/mark! 10)

;; with 2.0
(let [m (meters/meter ["test" "meters" "test-rate-mean-update-multiple"])]
  (meters/rate-mean m)
  (meters/mark! m 10))

The library maintains a default registry in metrics.core/default-registry which tries to keep the 1.x API as functional as possible but using your own registry is encouraged.

To instantiate a registry, use metrics.core/new-registry:

1
2
3
(require '[metrics.core :as mtr])

(mtr/new-registry)

See GH #19 for discussion.

defgauge Restricted to Functions Only

In metrics-clojure 1.x, metrics.gauges/defgauge could accept a function or a bunch of forms (body). In 2.0, it only accepts a function. This is in part due to the new API structure but also make the API more straightforward and works much better with explicit registry management now advocated by the library.

Nanoseconds Precision in Timers

Metrics 3.0 uses nanoseconds precision in timers.

Upgrade to Metrics 3.0

Metrics 3.0 is now used internally by the library.

Clojure 1.3 No Longer Supported

Clojure 1.3 is no longer supported by the library.

Full Change Log

metrics-clojure change log is available on GitHub.

About the Author

Michael on behalf of the metrics-clojure Team.

How to Make Your Open Source Project Really Awesome, Part 2

TL;DR

This is a continuation to How to Make Your Open Source Project Really Awesome.

In addition to the things recommended in the first part, do the following:

  • Talk to your biggest users
  • Make it easy to contribute
  • Publish releases often

How To Make Your Open Source Project Really Awesome, Part 2

As ClojureWerkz approaches its 3rd birthday, it’s time to expand our (relatively) popular blog post from early 2013, How to Make Your Open Source Project Really Awesome.

The focus of this part is on how to help your users help you, the maintainer. Very few successful open source projects are one man shows. The more people contribute, the better the project can get and less time it will take on your end to maintain it well.

Talk to Your Biggest Users

If your project is reasonably licensed and you maintain it well, eventually there will be people using it in commercial software. Some of them will be one man shops, others are giant public corporations. Most will lie in-between. It’s still possible to identify some key users: those that really push your project to the limits. Their engineers will be filing the most issues, will turn up on your mailing list more often than others, etc.

You need to identify those users and talk to them. Ask them what they do and don’t like, what they find missing, why they decided to use your project over an alternative.

Not only this kind of feedback is the best roadmap for your project, you will build some connections along the way. With ClojureWerkz, we are very fortunate to have commercial users among the companies we admire. Not only that, we now know some of their engineering team members. It’s never a bad position to be in.

Make It Easy to Contribute

We’ve touched on this in the first part. Time to expand on this key subject a little bit.

Identify Low Hanging Fruit Issues

Most contributors start with small issues: contribute a documentation fix here, a tiny feature there, a few extra failing test cases for this issue. Few of your users will jump in and contribute a major feature from the get-go. There are fairly objective reasons for this:

  • It takes time to get familiar with a codebase
  • People are more confident contributing to the projects where they “know” one of the maintainers
  • Contributing major features requires having some experience with the project

All of this takes time. Making it painless for your users to become contributors is perhaps the most important thing you can do. Except for the smallest projects, it usually separates the projects that will die from those that will march on even if you, the author, completely lose interest or ability to work on it.

Here’s a trick we’ve been trying with some of our projects (e.g. Elastisch): label GitHub issues with low-hanging fruit. When someone asks how she can help you, point the person to this label. We’ve seen this work very well for possibly the most widely used Clojure project out there: Leiningen.

At the time of writing, Leiningen has 217 contributors, which is not a small number by any stretch (the most popular ClojureWerkz projects have something between 25 and 40).

Document Development Setup

Many projects have some kind of development setup that needs to be performed before you can work on it. It can be a database of some kind running locally, or an env variable set to a particular value. For example, testing the Elastisch requires setting an env variable that tells the test suite what’s the name of the local ElasticSearch cluster should be.

On top of that, there can be multiple ways to run the tests, multiple test suites, OS-specific hacks necessary, VM/compiler version requirements, etc. Document as much of this process as you can. It should be dead obvious how to set things up for development.

If you have a specific VCS workflow (e.g. development happens on the branch devel and master is only used for stable releases), document this as well. Add a CONTRIBUTING.md file to the repo to make this more visible.

Not doing so will result in frustration for potential contributors. Don’t know about you but I’m not very motivated to contribute to a project that makes it frustrating.

Don’t Be an A-hole

This is a subject of a blog post of its own but be a decent person on the Internet (yes, it’s possible!). Be respectful. Point out various issues in contributed pull request in a reasonably polite way.

Nobody ever will contribute to a project maintained by an a-hole.

Publish Releases Often

This applies to both bug fix releases and development milestone versions. For ClojureWerkz projects, we often publish a new bug fix release when only 1 or 2 bugs were fixed. It’s easy for us to do and may save some pointless work for the users affected by the bugs.

With Leiningen, it takes a couple of commits (version changes) and 2 lines in the terminal to push a new release:

1
2
lein do pom, jar
scp push target/[jar] clojars@clojars.org:

With Leiningen 2.4, it can be as easy as a single command, lein release.

It is also an important thing to do for development milestones. Added a couple of neat features to your project? Publish a new beta release. There will be users who were dying to get their hands on this feature, so much so that they’re willing to run a beta in production just to not spend days reinventing the wheel.

As always, state release stability clearly in the change log and release notes.

This helps avoid another common mistake (applicable primarily to JVM languages): forcing your users to use snapshot releases. Snapshot releases make repeatable builds pretty much impossible and some tools (e.g. Leiningen) won’t allow non-snapshot releases depend on snapshot ones.

Give Credit Where Due

It’s always a good idea to give your contributors some credit by mentioning them in the change log. Even better, have a “Thank You, Contributors” section in your release announcement. Few things are as motivating as getting recognized for your work.

Final Thoughts

Just like it is not terribly hard to make your project approachable, it is also not that hard to make it contributor-friendly. Having a healthy number of contributors (even if it’s just 2-4) ensures that the project will live on even if you no longer have the time (or interest) maintain it. For example, I have a project that I haven’t contributed any code to since 2008. It is still being used and improved by other people.

Maybe more importantly, making your project contributor-friendly will introduce you to a lot of people and may open a lot of doors to you in the software industry, at least in engineering.

About the Author

Michael on behalf of the ClojureWerkz team.

Elastisch 2.1.0-beta4 Is Released

TL;DR

Elastisch is a battle tested, small but feature rich and well documented Clojure client for ElasticSearch. It supports virtually every Elastic Search feature and has solid documentation.

2.1.0-beta4 is a preview release of Elastisch 2.1 which introduces several new features, primarily in the native client.

Changes between Elastisch 2.1.0-beta3 and 2.1.0-beta4

Aggregations Support in the Native Client

Native client now has support for aggregations.

The API is the same as in the REST client.

Note that ElasticSearch 1.2 has 26 aggregations. Currently only the most commonly used ones are supported but support for more types will be added eventually. The supported types are:

  • Avg
  • Max
  • Min
  • Sum
  • Stats
  • Extended stats
  • Cardinality, value count
  • Percentiles
  • Histogram
  • Date Histogram
  • Range
  • Date Range
  • Terms
  • Missing
  • Global

Multi-Search Support in the Native Client

Native client now has support for multi-search.

The API is the same as in the REST client except that the functions are in the clojurewerkz.elastisch.native.multi.

Extra Options on Upserts

clojurewerkz.elastisch.native.document/upsert now accepts a map of extra options, e.g. parent document ID:

1
(doc/upsert conn index-name index-type id doc {:parent parent-id})

Terms Query Helper

clojurewerkz.elastisch.query/terms is a newly added alias for clojurewerkz.elastisch.query/term when used with a collection.

Contributed by Martin Klepsch.

Remove Alias Now Works in Native Client

Bug fixed in native client for removing aliases from indices and improved inline documentation. See aliases in the guide.

GH issue: #98.

Changes between Elastisch 2.1.0-beta2 and 2.1.0-beta3

Highlighting Support in Native Client

Native client now supports (most of the) highlighting features the REST client does:

1
2
3
4
5
6
(require '[clojurewerkz.elastisch.native.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])

(doc/search conn index type
            {:query (q/query-string :query "software" :default_field "summary")
             :highlight {:fields {:summary {}}}})

Changes between Elastisch 2.1.0-beta1 and 2.1.0-beta2

Per Connection clj-http Options in REST Client

It is now possible to specify clj-http options for REST API connections, e.g. to specify a timeout:

1
2
(esr/connect "http://127.0.0.1:9200/" {:conn-timeout 1000
                                       :basic-auth ["username" "pa$$w0rd"]})

Source Filtering Support in Native Client

Native client now supports source filtering just like the REST API client:

1
2
3
4
(doc/search conn index-name mapping-type
            :query   (q/match-all)
            :sort    {"first-name" "asc"}
            :_source ["first-name" "age"])
1
2
3
4
5
6
(doc/search conn index-name mapping-type
            :query   (q/match-all)
            :sort    {"first-name" "asc"}
            :_source {"exclude" ["title" "country"
                                 "planet" "biography"
                                 "last-name" "username"]})

GH issue: #73.

Search Can Return Fields and Source

Previously a search would return either the source document, or specific fields and not both. There are certain circumstances where having both are beneficial, for example when searching for a child document and you want to include the parent ID:

1
2
3
(require '[clojurewerkz.elastisch.native.document :as esd])

(esd/search conn "index" "child-type" :query (q/match-all) :fields ["_parent"])

The above would return the parent document ID in the :_parent field of each hit, but would not return the document itself. You can now have both by:

1
(esd/search conn "index" "child-type" :query (q/match-all) :fields ["_parent" "_source"])

Now the parent ID is in the :_parent field of each hit, and the matching document will be in :_source as per a normal search.

Contributed by Ben Ashford.

Full Change Log

Elastisch change log is available on GitHub.

Thank You, Contributors

Kudos to

  • Ben Ashford
  • Oliver Hine
  • Martin Klepsch

for contributing to this release.

Elastisch is a ClojureWerkz Project

Elastisch is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Clojure Cassandra client
  • Titanium, a Clojure graph library
  • Neocons, a client for the Neo4J REST API
  • Welle, a Riak client with batteries included
  • Quartzite, a powerful scheduling library

and several others. If you like Elastisch, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

About the Author

Michael on behalf of the ClojureWerkz Team

Pantomime 2.3.0 Is Released

TL;DR

Pantomime is a tiny Clojure library for working with MIME types and file metadata.

Changes between Pantomime 2.2.0 and 2.3.0

Extension Detection From MIME Type

pantomime.mime/extension-for-name is a new function that suggests common extensions for MIME type names:

1
2
3
4
5
6
7
8
(require '[pantomime.mime :as pm])

(pm/extension-for-name "application/vnd.ms-excel")
;= ".xls"
(pm/extension-for-name "image/jpeg")
;= ".jpg"
(pm/extension-for-name "application/octet-stream")
;= ".bin"

Changes between Pantomime 2.1.0 and 2.2.0

Clojure 1.6

The library now depends on Clojure 1.6.

Change Log

Pantomime change log is available on GitHub.

Pantomime is a ClojureWerkz Project

Pantomime is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Cassaforte, a Clojure Cassandra client built around CQL
  • Monger, a Clojure MongoDB client for a more civilized age
  • Welle, a Riak client with batteries included
  • Neocons, a client for the Neo4J REST API
  • Quartzite, a powerful scheduling library

and several others. If you like Pantomime, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Machine Head 1.0.0-beta9 Is Released

TL;DR

Machine Head is a small Clojure MQTT client.

1.0.0-beta9 is a development milestone that introduces one breaking API change.

Changes Between 1.0.0-beta8 and 1.0.0-beta9

Subscription API Change

Previously clojurewerkz.machine-head.client/subscribe accepted a list of topics and optionally a list of QoS levels:

1
2
3
(mh/subscribe c ["mh/topics/#" "mh/alt.topics/+"]
                  (fn [^String topic meta ^bytes payload])
                  {:qos [0 1]})

This turns out to be a fairly confusing API.

We’ve changed it to accept a map of topics to QoS levels. While a bit more verbose, this API make it very clear what topic will use what QoS:

1
2
(mh/subscribe c {"mh/topics/#" 0 "mh/alt.topics/+" 1}
                  (fn [^String topic meta ^bytes payload] ))

Changes Between 1.0.0-beta7 and 1.0.0-beta8

Clojure 1.6 By Default

The project now depends on org.clojure/clojure version 1.6.0. It is still compatible with Clojure 1.4 and if your project.clj depends on a different version, it will be used, but 1.6 is the default now.

We encourage all users to upgrade to 1.6, it is a drop-in replacement for the majority of projects out there.

Changes Between 1.0.0-beta6 and 1.0.0-beta7

Retain Default Change

When publishing, retain now defaults to false, which is a much more sensible default.

Contributed by Martin Trojer.

Change Log

Machine Head change log is available on GitHub.

Machine Head is a ClojureWerkz Project

Machine Head is part of the group of Clojure libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Cassandra client built around CQL 3
  • Neocons, a feature rich Clojure client for Neo4J REST API

and several others. If you like Machine Head, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

About the Author

Michael on behalf of the ClojureWerkz Team