Author: Ben Hall

I just would like to store some data… is that too much to ask for? It doesn’t need to scale, it just needs to be saved for later…

— Ben Hall (@Ben_Hall) January 2, 2015

Looking back at 2014 I worked with CouchDb, MongoDb, LevelDb, Cassandra, ElasticSearch, Redis, Neo4j, Postgresql and MySQL to manage data. Faced with a new prototype I reached the point where I needed to save data. I don’t need it to scale yet, I don’t need it to have map/reduce and storage for billion of records, I don’t even need it to be quick. I just want to store data and in future be able to easily have the data returned.

Turns out my choices are limited to be point of flat files looking like the best option. Before I went down that path I tried one more approach, Sqlite3. This post will investigate how sane Sqlite3 would be given it’s stable and embeddable.

Firstly we need to create the database schema, the solution is already becoming time consuming and boring. The script I created when the application loads is as follows:

var path = require("path");
var fs = require("fs");
var file = path.join(__dirname, "data.db");
var sqlite3 = require("sqlite3").verbose();

function create(cb) {
  var db = new sqlite3.Database(file);

  console.log("Creating db...");
  db.serialize(function() {
    db.run("CREATE TABLE user (id integer primary key, fb_id TEXT, name TEXT, email TEXT, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)");
    console.log("Created db");

    cb();
  });
};

function init(cb) {
  fs.exists(file, function(exist) {
    if(exist) {
      return cb();
    } else {
      create(cb);
    }
  });
};

module.exports = init;

If the schema changes then we’ll need to write an additional script, a problem we can worry about for another day.

Once we’ve created the DB then inserting data becomes straight forward apart from the fact that we might not know the data in advance meaning migration scripts are likely to happen sooner rather than later.

db.run("INSERT INTO user (fb_id, name, email) VALUES (?,?,?)", [fb_id, name, email], function(err) {
  res.status = 201;
 res.end();
});

One nice added bonus is the sqlite3 command line tool.

$ sqlite3 db/data.db
SQLite version 3.7.13 2012-06-11 02:05:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from user;

White Sqlite3 works nicely to store data, having to manage a schema is an overhead and additional problems I don’t want to deal with. It’s 2015, why can’t I just store some data?

It’s fair to say that 2014 was “The Year Of The Container” with Docker and the ecosystem growing at exponential rates. With fast movements and innovations happening it’s easy to overlook some early considerations and consider them best practice. One concern is focused on the use of the :latest tag in a Dockerfile.

The FROM instruction in your Dockerfile accepts either the image or an image and a tag. In the documentations it states that “If no tag is given to the FROM instruction, latest is assumed.”

Let’s take a closer look at how the :latest works with Node.js based on the official images. The list of tags for Node can be found on the Docker Hub Registry.

Firstly, node:latest will always point to the latest version. This has two side-effects. The first is that you’ll automatically be running future major releases which could include breaking changes for your application. If everyone uses node:latest then once 0.12 is released there will be a number of companies running 0.12 without being prepared. While we hope test coverage would capture potential issues it could have adverse effects.

The second is based on Docker’s ability to reuse base images. If a new minor release occurs between image builds then you’ll need to download and store these minor revisions. This increases space required on the build server along with the time required due to downloading the latest version of Node.js.

Given this we have three choices when picking our FROM instruction.

FROM node:latest – Always download the latest stable, ignoring major/minor revisions.
FROM node:0.10 – We’re happy with any 0.10 releases, we’ll upgrade to 0.12 when we are ready.
FROM node:0.10.34 – We’ll manage the upgrade between minor versions.

The last one defines that we’ll always run against 0.10.34 of node, this gives us confidence that our base-line won’t change without us knowing.

While you may think this isn’t an issue because it’s only node, what about the latest version of ubuntu? As Dockerfiles become a long term foundation of a project, using “FROM ubuntu” could point to a different version than what the original developers intended. In future I will be using a fixed tag and upgrading when required.