---
title: Commit History Metrics
url: https://www.grahambrooks.com/post/2017-06-15-commit-history-metrics/
date: 2017-06-15T00:00:00+00:00
author: Graham Brooks
tags:
- observability
- Version Control
- diagrams
---

# Commit History Metrics

For the last few years I have been very interested in what we can learn about our software development processes by looking at the data generated by the tools that we use to manage and deploy our work. This work started the Kuona project (Kuona is the [shona](https://en.wikipedia.org/wiki/Shona_people) word "To see.." <http://kuona.io>. All the code is open source on github <https://github.com/kuona>

Initial work focused on creating a dashboard from gathered metrics. Building full stack meant great flexibility but a lot of work and finding time for that work challenging with busy schedules. Over the past couple of weeks I have been working on feeding data from the collectors into [elasticsearch](https://www.elastic.co/products/elasticsearch) and then using [kibana](https://www.elastic.co/products/kibana) to visualise the collected data.

Cutting to the chase heres and example chart:

<div wrapper="1" align="center" width="100%">

<figure>
<img src="https://www.grahambrooks.com/images/spring-boot-commits-by-day.png" />
</figure>

</div>

I am not close enough to the project to draw significant insight from this chart but as an outside observer I can see that the project has been continually active since 2013 in this repository.

Here’s another metric drawn from Spring Boot history:

<div wrapper="1" align="center" width="100%">

<figure>
<img src="https://www.grahambrooks.com/images/spring-boot-kuona-metrics.png" />
</figure>

</div>

Putting these metrics together in Kibana is really straightforward. But how do we get the data there?

# Setting up the Git Kuona Collectors

Kuona and the collectors are still in development so you will need to build/run from source.

Pre-requisites

- Make sure you have an up to date Java Development Kit JDK.

- Install Leiningen <https://leiningen.org/#install>

- Clone the git collector locally `git clone git@github.com:kuona/git-collector.git`

Before running you will need to write a configuration file for the collector to use.

<div>

<div class="title">

properties.json

</div>

``` json
{"workspace": "<workspace directory>",
  "git": [{ "url": "git@github.com:spring-projects/spring-framework.git" },
                { "url": "git@github.com:spring-projects/spring-boot.git" }]
}
```

</div>

Change \<workspace directory\> to some temporary space. On first run the git collector clones the repositories into the workspace. Subsequent runs update the repository before scanning history.

The collector only supports history on a single branch (master). Still need to work out how to handle multiple branches well.

- Install ElasticSearch. If you are using a mac `brew` `install` `elasticseach` works nicely

- Install Kibana. Again homebrew has a recipe `brew` `install` `kibana`

<!-- -->

- Make sure Elastic search is running <http://localhost:9200>

- Make sure that Kibana is running <http://localhost:5601>

Elasticsearch does not have any of or data so we will need to postpone Kibana configuration until we have at least one record.

Running the collector:

In the git-collector directory

- Create your properties file (see above)

`lein` `run`

Sit back and relax - collecting revision history takes a while.

Once the initial clone is complete records will appear in Elasticsearch so we can configure Kibana to use the generated data:

<div wrapper="1" align="center" width="100%">

<figure>
<img src="https://www.grahambrooks.com/images/kuona-kibana-index-setup.png" />
</figure>

</div>

Once set up the index should look something like this:

<div wrapper="1" align="center" width="100%">

<figure>
<img src="https://www.grahambrooks.com/images/kuona-kibana-index.png" />
</figure>

</div>

It would be great to get feedback on this and the other tools being developed for Kuona. Currently working on a collector for Jenkins <https://github.com/kuona/jenkins-collector> that works like the git collector but for Jenkins build jobs.

Check out the project <https://github.com/kuona>. Join the team <http://kuona.io> <https://groups.google.com/forum/#!forum/kuona-dev> or check out the slack channel <https://kuonadev.slack.com/>