The ClojureWerkz Blog

News and updates about ClojureWerkz projects

Elastisch 1.3.0 Is Released

TL;DR

Elastisch is a battle tested, small but feature rich and well documented Clojure client for ElasticSearch. It supports virtually every Elastic Search feature and has solid documentation.

1.3.0 is a minor feature and usability release. We recommend all users to upgrade to it.

We recommend all users to upgrade to 1.3.0 as soon as possible.

Changes between Elastisch 1.2.0 and 1.3.0

:sort Option Can Be a SortBuilder

:sort option can now be a com.elasticsearch.search.sort.SortBuilder instance and not just a string.

Contributed by Mark Wong-VanHaren.

ElasticSearch Native Client Upgrade

Elastisch now depends on ElasticSearch native client version 0.90.7.

Bulk Indexing Fix

Elastisch no longer erroneously inserts _index and _type fields into documents inserted via bulk API.

Contributed by Max Barnash.

Result Scrolling as Lazy Sequences

clojurewerkz.elastisch.native.document/scroll-seq and clojurewerkz.elastisch.rest.document/scroll-seq are new functions that accept a search query response and return a lazy sequence of paginated search results.

This makes working with result sets that require pagination much more natural:

1
2
3
4
5
6
7
8
9
10
11
(require '[clojurewerkz.elastisch.native.document :as doc])

(let [index-name   "articles"
      mapping-type "article"
      res-seq      (doc/scroll-seq
                       (doc/search index-name mapping-type
                                   :query (q/term :title "Emptiness")
                                   :search_type "query_then_fetch"
                                   :scroll "1m"
                                   :size 2))]
    res-seq))

Contributed by Max Barnash.

Upserts in Native Client

Native client now supports upserts of documents:

1
2
3
(require '[clojurewerkz.elastisch.native.document :as doc])

(doc/upsert "people" "person" "elastisch" {:name "Elastisch" :language "Clojure"})

Date Histogram Fix

Date histogram in the native client now includes :total field.

Contributed by Jim Dunn.

Fields in Search Hit Results in Native Client

Native client now returns the same value in :fields and :_fields keys in search hits. This makes it both backwards compatible with earlier versions and the format ElasticSearch HTTP API uses.

Bulk Index and Delete Operations Support More Options

Bulk index and delete operations support _parent and _routing keys.

Contributed by Baptiste Fontaine.

Clojure 1.3 Support Dropped

Elastisch now requires Clojure 1.4.

Cheshire Update

Cheshire dependency has been upgraded to version 5.2.0.

Elastisch change log is available on GitHub.

Thank You, Contributors

Kudos to

  • Max Barnash
  • Jim Dunn
  • Baptiste Fontaine
  • Mark Wong-VanHaren

for contributing to this release.

Elastisch is a ClojureWerkz Project

Elastisch is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Clojure Cassandra client
  • Titanium, a Clojure graph library
  • Neocons, a client for the Neo4J REST API
  • Welle, a Riak client with batteries included
  • Quartzite, a powerful scheduling library

and several others. If you like Elastisch, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Machine Head 1.0.0-beta5 Is Released

TL;DR

Machine Head is a small Clojure MQTT client.

1.0.0-beta5 is a development milestone that support for MQTT Last Will and Testament.

Changes Between 1.0.0-beta4 and 1.0.0-beta5

Last Will and Testament

Machine Head now supports providing client last will and testament:

1
2
3
4
(require '[clojurewerkz.machine-head.client :as mh])

(let [will {:topic "lw-topic" :payload (.getBytes "last will") :qos 0 :retain false}]
  (mh/connect "" (mh/generate-id) {:will will}))

Contributed by Paul Bellamy.

Change Log

Machine Head change log is available on GitHub.

Machine Head is a ClojureWerkz Project

Machine Head is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Cassandra client built around CQL 3
  • Neocons, a feature rich Clojure client for Neo4J REST API

and several others. If you like Machine Head, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Neocons 2.0.1 Is Released

TL;DR

Neocons is a feature rich idiomatic Clojure client for the Neo4J REST API.

2.0.1 is a bug fix release that focuses on Neo4J 2.0 GA compatibility.

Changes between Neocons 2.0.0 and 2.0.1

clj-http upgraded to 0.7.8

Neocons now uses clj-http 0.7.8.

Neo4J 2.0 Index Creation Fix

Neocons will now use a key name accepted by Neo4J 2.0.0-rc1 when creating indexes.

Contributed by Rohit Aggarwal.

Change Log

We recommend all users to upgrade to 2.0.1 as soon as possible. It will provide full Neo4J 2.0 compatibility.

Neocons change log is available on GitHub.

Neocons is a ClojureWerkz Project

Neocons is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB client for a more civilized age
  • Welle, a Riak client with batteries included
  • Quartzite, a powerful scheduling library

and several others. If you like Monger, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Introducing Buffy

Buffy The Byte Buffer Slayer

Buffy is a Clojure library to working with binary data, writing complete binary protocol implementations in Clojure, storing complex data structures in an off-heap cache, reading binary files and doing everything you would usually do with byte buffers.

Buffy is built on top of Netty byte buffers, which addresses many of the issues with java.util.ByteBuffer.

Main features

Buffy enables you to access buffers in the similar way you work with the regular Clojure data structures. You can get and set fields separately from one another, give names to different parts of binary representation and so on:

  • partial deserialization (read and deserialise parts of a byte buffer)
  • named access (access parts of your buffer by names)
  • composing/decomposing from key/value pairs
  • pretty hexdump
  • many useful default types that you can combine and extend easily

Data Types

Data types used in Buffy buffers include:

  • primitives, such as int32, boolean, byte, short, medium, float, and long
  • arbitrary-length string
  • byte arrays
  • composite types (combine multiple primitive values together)
  • repeated type (repeat any primitive arbitrary amount of times in payload)
  • enum type (for mapping between human-readable and binary representation of constants)

The Future of Buffy

Buffy has been serving us well for some time, and no major issues were revealed. However, until it reaches GA, we can’t rule breaking API changes out.

We are also very much behind on documentation.

Buffy is a ClojureWerkz Project

Buffy is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Cassaforte, a Clojure Cassandra client built around CQL
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • EEP, a Clojure event processing library
  • Meltdown, Clojure interface to Reactor
  • Monger, a Clojure MongoDB client for a more civilized age
  • Neocons, a client for the Neo4J REST API
  • Quartzite, a powerful scheduling library

and several others. If you like Buffy, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

@ifesdjeen on behalf of the ClojureWerkz Team

Langohr 1.7.0 Is Released

TL;DR

Langohr is a Clojure RabbitMQ client that embraces AMQP 0.9.1 Model.

1.7.0 is primarily a bug fix release.

Changes between Langohr 1.6.0 and 1.7.0

RabbitMQ Java Client Upgrade

RabbitMQ Java client dependency has been updated to 3.2.1.

Automatic Recovery Improvements

Langohr will now make sure to handle network I/O-related exceptions during recovery and reconnect every N seconds.

The following code example can be used to test connection recovery. Either restart RabbitMQ or close a connection using Management UI to test drive it.

Change Log

Langohr change log is available on GitHub.

Langohr is a ClojureWerkz Project

Langohr is part of the group of libraries known as ClojureWerkz, together with

  • Elastisch, a minimalistic well documented Clojure client for ElasticSearch
  • Cassaforte, a Clojure Cassandra client built around CQL 3.0
  • Monger, a Clojure MongoDB client for a more civilized age
  • Neocons, a client for the Neo4J REST API
  • Quartzite, a powerful scheduling library

and several others. If you like Langohr, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Welle 2.0.0-beta1 Is Released

TL;DR

Welle is an expressive Clojure client for Riak with batteries included.

2.0.0-beta1 is a development release of 2.0 that has breaking API changes and introduces an important bug fix, dependency updates and support for Riak 1.4 counters.

Changes between Welle 1.5.0 and 2.0

Welle 2.0 has breaking API changes in clojurewerkz.welle.kv functions.

Changes in K/V Function Return Values

This is a breaking API change.

Welle 2.0 changes how Riak responses are represented as Clojure maps. Welle now will correctly preserve all vector clocks associated with multiple siblings in the response and the response itself.

This means that welle.kv/modify will work correctly and won’t make sibling explosions worse.

The most important part of the change is how responses are represented: every response is an immutable map that has :result key as well as other metadata keys (:has-value?, :has-siblings?, :modified?, :content-type and so on).

This contrasts with earlier versions, where results were returned directly by functions such as welle.kv/fetch, it is now possible to destructure the response in order to obtain the returned value:

1
2
3
4
(require '[clojurewerkz.welle.kv :as kv])

(let [{:keys [result] :as m} (kv/fetch bucket-name k :r 1)]
  (comment "Do something with the result"))

Here are the keys that clojurewerkz.welle.kv/fetch returns now for every response:

  • :result: one or more objects returned by Riak
  • :vclock: vector clock of the response
  • :has-siblings?: true if response has siblings
  • :has-value?: true if response is non-empty
  • :modified?: false when conditional GET returned a non-modified response
  • :deleted?: true if this object has been deleted but there is a vclock for it

Clojure 1.3 Support Dropped

Welle no longer supports Clojure 1.3.

Counters Support (Riak 1.4+)

clojurewerkz.welle.counters is a new namespace that provides operations on Riak counters:

1
2
3
4
5
6
7
8
9
10
11
12
(require '[clojurewerkz.welle.counters :as wcnt])

(let [bucket-name "counters"
      counter     "hit-points"]
  (wcnt/increment-counter bucket-name counter)
  ;= 1
  (wcnt/fetch-counter bucket-name counter)
  ;= 1
  (wcnt/increment-counter bucket-name counter :value 2))
  ;= 3
  (wcnt/increment-counter bucket-name counter :value -3))
  ;= 0

Riak Java Client Update

Welle now uses Riak Java client 1.4.2.

Cheshire Update

Cheshire has been updated to 5.2.0.

clj-http Update

clj-http has been updated to 0.7.7.

Support Update

ClojureWerkz Support has been updated to 0.20.0.

Ports Support in PB Cluster Client

While creating a protocol buffer cluster client you can now provide hosts and ports separated by a colon, e.g. “127.0.0.1:10017”. If a port is not provided, the default port will be used.

So now you can use following format:

1
2
3
4
5
6
7
(wc/connect-to-cluster-via-pb! ["10.0.1.2",
                  "10.0.1.3",
              "10.0.1.4",
              "10.0.1.5",
              "10.0.1.6"])

as well as following format:

clojure (wc/connect-to-cluster-via-pb! [“127.0.0.1:10017”,

                "127.0.0.1:10027",
            "127.0.0.1:10037",
            "127.0.0.1:10047"])

“`

Validateur Dependency Dropped

Validateur is no longer a dependency of Welle.

Don’t worry, they still work well together.

Optional Keywordization of Keys With JSON Serialization

Automatic JSON serialization previously unconditionally converted keys to keywords. This may be a problem for some projects, because keywords are not garbage collected.

clojurewerkz.welle.conversion/*convert-json-keys-to-keywords* is a new dynamic var that controls this behavior. When bound to false, automatic JSON serialization won’t convert keys to keywords.

Change Log

We recommend all users to give 2.0.0-beta1 a try.

Welle change log is available on GitHub.

Welle is a ClojureWerkz Project

Welle is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB client for a more civilized age
  • Neocons, a feature rich idiomatic Clojure client for the Neo4J REST API
  • Quartzite, a powerful scheduling library

and several others. If you like Welle, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Donations

ClojureWerkz accepts donations. If you feel like our projects save you time, consider donating. Thanks.

Michael on behalf of the ClojureWerkz Team

Elastisch 1.3.0-rc1 Is Released

TL;DR

Elastisch is a battle tested, small but feature rich and well documented Clojure client for ElasticSearch. It supports virtually every Elastic Search feature and has solid documentation.

1.3.0-rc1 is a release candidate that is compatible with 1.2.0.

Changes between Elastisch 1.3.0-beta5 and 1.3.0-rc1

Bulk Indexing Fix

Elastisch no longer erroneously inserts _index and _type fields into documents inserted via bulk API.

Contributed by Max Barnash.

Result Scrolling as Lazy Sequences

clojurewerkz.elastisch.native.document/scroll-seq and clojurewerkz.elastisch.rest.document/scroll-seq are new functions that accept a search query response and return a lazy sequence of paginated search results.

This makes working with result sets that require pagination much more natural:

1
2
3
4
5
6
7
8
9
10
11
(require '[clojurewerkz.elastisch.native.document :as doc])

(let [index-name   "articles"
      mapping-type "article"
      res-seq      (doc/scroll-seq
                       (doc/search index-name mapping-type
                                   :query (q/term :title "Emptiness")
                                   :search_type "query_then_fetch"
                                   :scroll "1m"
                                   :size 2))]
    res-seq))

Contributed by Max Barnash.

Change log

Elastisch change log is available on GitHub.

Thank You, Contributors

Kudos to Max Barnash for contributing to this release.

Elastisch is a ClojureWerkz Project

Elastisch is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Clojure Cassandra client
  • Titanium, a Clojure graph library
  • Neocons, a client for the Neo4J REST API
  • Meltdown, a Clojure interface to Reactor
  • Quartzite, a powerful scheduling library

and several others. If you like Elastisch, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Machine Head 1.0.0-beta4 Is Released

TL;DR

Machine Head is a small Clojure MQTT client.

1.0.0-beta4 is a development milestone that works around a bug in Eclipse Paho Java.

Changes Between 1.0.0-beta3 and 1.0.0-beta4

Client ID Limit

MQTT spec dictates that client ID’s should be limited to 23 bytes. Eclipse Paho client generator can produce ID’s longer than that.

Machine Head now will limit ID length to the last 23 bytes.

Contributed by Yodit Stanton.

Change Log

Machine Head change log is available on GitHub.

Machine Head is a ClojureWerkz Project

Machine Head is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Cassandra client built around CQL 3
  • Neocons, a feature rich Clojure client for Neo4J REST API

and several others. If you like Machine Head, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team

Neocons 2.0.0 Is Released

TL;DR

Neocons is a feature rich idiomatic Clojure client for the Neo4J REST API.

2.0.0 is a major release that targets Neo4J 2.0. It is backwards compatible with 1.1. Note that Neo4J 2.0 contains some breaking Cypher syntax changes.

Neocons documentation has been updated to cover the new features in Neocons 2 and Neo4J.

Changes between Neocons 1.1.0 and 2.0.0

Transaction Support (Neo4J Server 2.0)

Neocons 2.0 gains support for transactions.

Higher Level API

A group of Cypher statements can be executed in a transaction that will be committed automatically upon success. Any error during the execution will trigger a rollback.

1
2
3
4
5
(require '[clojurewerkz.neocons.rest.transaction :as tx])

(tx/in-transaction
  (tx/statement "CREATE (n {props}) RETURN n" {:props {:name "Node 1"}})
  (tx/statement "CREATE (n {props}) RETURN n" {:props {:name "Node 2"}}))

Lower Level API

Transactions are instantiated from a group of Cypher statements that are passed as maps to clojurewerkz.neocons.rest.transaction/begin:

1
2
3
4
5
(let [t (tx/begin-tx [{:statement "CREATE (n {props}) RETURN n" {:props {:name "My node"}}}])]
  (tx/commit t))

(let [t (tx/begin-tx)]
  (tx/rollback t))

clojurewerkz.neocons.rest.transaction/commit and clojurewerkz.neocons.rest.transaction/rollback commit and roll a transaction back, respectively.

Macro for working with a transaction

If you want a more fine grained control of working in a transaction without manually committing or checking for exceptions, you can use the clojurewerkz.neocons.rest.transaction/with-transaction macro.

1
2
3
4
5
6
7
8
(require '[clojurewerkz.neocons.rest.transaction :as tx])

(let [transaction (tx/begin-tx)]
  (tx/with-transaction
    transaction
    true
    (let [[_ result] (tx/execute transaction [(tx/statement "CREATE (n) RETURN ID(n)")])]
    (println result))))

If there any errors while processing, the transaction is rolled back.

The first argument is the variable which holds the transaction information. The second argument to the macro is commit-on-success, which commits the transaction there are no errors.

Constraints Support (Neo4J 2.0 Only)

clojurewerkz.neocons.rest.constraints is a new namespace that implements Neo4J 2.0 constraints.

1
2
3
4
5
6
7
8
9
10
(require '[clojurewerkz.neocons.rest.constraints :as cts])

;; create a uniqueness constraint
(cts/create-unique "Person" :name)

;; get constraint info
(cts/get-unique "Person" :name)

;; drop a constraint
(cts/drop "Person" :name)

Labels Support (Neo4J 2.0 Only)

clojurewerkz.neocons.rest.labels is a new namespace that provides support for labels in Neo4J 2.0.

It is possible to add, replace, remove and retrieve labels to/from a node.

To add labels to a node, use clojurewerkz.neocons.rest.labels/add:

1
2
3
(require '[clojurewerkz.neocons.rest.labels :as nl])

(nl/add node ["neo4j" "clojure"])

To add replaces all labels on a node, use clojurewerkz.neocons.rest.labels/replace:

1
2
3
(require '[clojurewerkz.neocons.rest.labels :as nl])

(nl/replace node ["graph" "database"])

Deleting a label from a node is possible with clojurewerkz.neocons.rest.labels/remove:

1
2
3
(require '[clojurewerkz.neocons.rest.labels :as nl])

(nl/remove node "database")

clojurewerkz.neocons.rest.labels/get-all-labels is the function that lists either all labels in the database (w/o arguments) or on a specific node (1-arity):

1
2
3
4
5
6
(require '[clojurewerkz.neocons.rest.labels :as nl])

(nl/get-all-labels node)
;= [all labels]
(nl/get-all-labels node)
;= [labels on node]

Clojure 1.6 Compatibility Fixes

Neocons 2.0 is compatible with recent releases of Clojure 1.6 (master).

Clojure 1.3 Support Dropped

Neocons no longer supports Clojure 1.3.

Documentation Updates

Neocons documentation has been updated to cover the new features in Neocons 2 and Neo4J:

Change Log

We encourage all users to give this version (and Neo4J 2.0!) a try.

Neocons change log is available on GitHub.

Neocons is a ClojureWerkz Project

Neocons is part of the group of libraries known as ClojureWerkz, together with

  • Langohr, a Clojure client for RabbitMQ that embraces the AMQP 0.9.1 model
  • Elastisch, a minimalistic Clojure client for ElasticSearch
  • Monger, a Clojure MongoDB client for a more civilized age
  • Cassaforte, a Cassandra client built around CQL 3
  • Quartzite, a powerful scheduling library

and several others. If you like Neocons, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

About the Author

Michael on behalf of the ClojureWerkz Team

Langohr 1.6.0 Is Released

TL;DR

Langohr is a Clojure RabbitMQ client that embraces AMQP 0.9.1 Model.

1.6.0 is a minor feature release.

Changes between Langohr 1.5.0 and 1.6.0

RabbitMQ Java Client Upgrade

RabbitMQ Java client dependency has been updated to 3.2.0.

Automatic Recovery Improvements

Connections will only be recovered if shutdown was not application-initiated.

Contributed by Stephen Dienst.

Support Update

Langohr now depends on ClojureWerkz Support 0.20.0.

langohr.conversion/BytePayload and langohr.conversion/to-bytes are replaced by clojurewerkz.support.bytes/ByteSource and clojurewerkz.support.bytes/to-byte-array, respectively.

Change Log

Langohr change log is available on GitHub.

Langohr is a ClojureWerkz Project

Langohr is part of the group of libraries known as ClojureWerkz, together with

  • Elastisch, a minimalistic well documented Clojure client for ElasticSearch
  • Cassaforte, a Clojure Cassandra client built around CQL 3.0
  • Monger, a Clojure MongoDB client for a more civilized age
  • Neocons, a client for the Neo4J REST API
  • Quartzite, a powerful scheduling library

and several others. If you like Langohr, you may also like our other projects.

Let us know what you think on Twitter or on the Clojure mailing list.

Michael on behalf of the ClojureWerkz Team