DynamoDB OneTable

DynamoDB OneTable (OneTable) is an access library for DynamoDB applications that use single-table design patterns with NodeJS. OneTable makes dealing with DynamoDB and single-table design patterns dramatically easier while still providing easy access to the full DynamoDB API.

OneTable is provided open source (MIT license) from GitHub OneTable or NPM OneTable.

History and Credits

After watching the famous Rick Houlihan DynamoDB ReInvent Video, we changed how we used DynamoDB for our EmbedThis Ioto serverless developer studio to use single-table design patterns. However, we found the going tough and thus this library was created to make our single-table patterns less tedious, more natural and a joy with DynamoDB.

A big thank you to Alex DeBrie and his excellent DynamoDB Book. Highly recommended.

OneTable Overview

OneTable provides a convenience API over the DynamoDB APIs. It offers a flexible high-level API that supports single-table design patterns and eases the tedium of working with the standard, unadorned DynamoDB API. OneTable can invoke DynamoDB APIs or it can be used as a generator to create DynamoDB API parameters that you can save or execute yourself.

OneTable is not opinionated (as much as possible) and provides hooks for you to customize requests and responses to suit your exact needs.

Here are some of the key features of OneTable

• Schema supported single-table access to DynamoDB APIs.
• Efficient storage and access of multiple entities in a single DynamoDB table.
• High level API with type marshaling, validations, and extended query capability for get/delete/update operations.
• Bidirectional conversion of DynamoDB types to Javascript types.
• Option to invoke DynamoDB or simply generate API parameters.
• Generation of Conditional, Filter, Key and Update expressions.
• Schema item definitions for attribute types, default values, enums and validations.
• Powerful field level validations with required and transactional unique attributes.
• Easy parameterization of filter and conditional queries.
• Multi-page response aggregation.
• Compound and templated key management.
• Encrypted fields.
• Support for Batch, Transactions, GSI, LSI indexes.
• Hooks to modify DynamoDB requests and responses and for item/attribute migrations.
• Controllable logging to see exact parameter, data and responses.
• Simple, easy to read source to modify (< 1000 lines).
• Safety options to prevent “rm -fr *”.
• No module dependencies.
• Support for the AWS SDK v3 and v2
• TypeScript type inference from schema for full type validation on APIs, parameters, returns, and entities and attributes.

Installation

npm i dynamodb-onetable

Quick Tour

Import the OneTable library. If you are not using ES modules or Typescript, use require to import the libraries.

import {Table} from 'dynamodb-onetable'

If you are using the AWS SDK V2, import the AWS DynamoDB class and create a DocumentClient instance.

import DynamoDB from 'aws-sdk/clients/dynamodb'
const client = new DynamoDB.DocumentClient(params)

This version includes prototype support for the AWS SDK v3.

If you are using the AWS SDK v3, import the AWS v3 DynamoDBClient class and the OneTable Dynamo helper. Then create a DynamoDBClient instance and Dynamo wrapper instance.

import {DynamoDBClient} from '@aws-sdk/client-dynamodb'
import Dynamo from 'dynamodb-onetable/Dynamo'
const client = new Dynamo({client: new DynamoDBClient(params)})

Initialize your your OneTable Table instance and define your models via a schema. The schema defines your single-table entities, attributes and indexes.

const table = new Table({
    client: client,
    name: 'MyTable',
    schema: MySchema,
})

This will initialize your your OneTable Table instance and define your models via a schema.

Schemas

Schemas define your models (entities), keys, indexes and attributes. Schemas look like this:

const MySchema = {
    version: '0.1.0',
    format: 'onetable:1.0.0',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' }
        gs1:     { hash: 'gs1pk', sort: 'gs1sk' }
    },
    models: {
        Account: {
            pk:          { value: 'account:${name}' },
            sk:          { value: 'account:' },
            id:          { type: String, uuid: true, validate: /^[0-9A-F]{32}$/i, },
            name:        { type: String, required: true, }
            status:      { type: String, default: 'active' },
            zip:         { type: String },
        },
        User: {
            pk:          { value: 'account:${accountName}' },
            sk:          { value: 'user:${email}', validate: EmailRegExp },
            id:          { type: String },
            accountName: { type: String },
            email:       { type: String, required: true },
            firstName:   { type: String, required: true },
            lastName:    { type: String, required: true },
            username:    { type: String, required: true },
            role:        { type: String, enum: ['user', 'admin'],
                           required: true, default: 'user' }
            balance:     { type: Number, default: 0 },

            gs1pk:       { value: 'user-email:${email}' },
            gs1sk:       { value: 'user:' },
        }
    },
    params: {
        isoDates: true,
        timestamps: true,
    }
}

Schemas define your models and their attributes. Keys (pk, gs1pk) can derive their values from other attributes via templating.

Alternatively, you can define models one by one:

const Card = new Model(table, {
    name: 'Card',
    fields: {
        pk: { value: 'card:${number}'}
        number: { type: String },
        ...
    }
})

To create an item:

let account = await Account.create({
    id: '8e7bbe6a-4afc-4117-9218-67081afc935b',
    name: 'Acme Airplanes'
})

This will write the following to DynamoDB:

{
    pk:         'account:8e7bbe6a-4afc-4117-9218-67081afc935b',
    sk:         'account:98034',
    id:         '8e7bbe6a-4afc-4117-9218-67081afc935b',
    name:       'Acme Airplanes',
    status:     'active',
    zip:        98034,
    created:    1610347305510,
    updated:    1610347305510,
}

Get an item:

let account = await Account.get({
    id: '8e7bbe6a-4afc-4117-9218-67081afc935b',
    zip: 98034,
})

which will return:

{
    id:       '8e7bbe6a-4afc-4117-9218-67081afc935b',
    name:     'Acme Airplanes',
    status:   'active',
    zip:      98034,
}

To use a secondary index:

let user = await User.get({email: 'user@example.com'}, {index: 'gs1'})

To find a set of items:

let users = await User.find({accountId: account.id})

let adminUsers = await User.find({accountId: account.id, role: 'admin'})

let adminUsers = await User.find({accountId: account.id}, {
    where: '${balance} > {100.00}'
})

To update an item:

await User.update({id: userId, balance: 50})
await User.update({id: userId}, {add: {balance: 10.00}})

To do a transactional update:

let transaction = {}
await Account.update({id: account.id, status: 'active'}, {transaction})
await User.update({id: user.id, role: 'user'}, {transaction})
await table.transact('write', transaction)

TypeScript

OneTable fully supports TypeScript apps and OneTable APIs are fully type checked.

However, OneTable goes further creates type declarations for your table entities and attributes. TypeScript will catch any invalid schema, entity or entity attribute references.

Using TypeScript dynamic typing, OneTable automatically converts your OneTable schema into fully typed generic Model APIs.

For example:

const schema = {
    version: '0.1.0',
    format: 'onetable:1.0.0',
    models: {
        Account: {
            pk:     { type: String, value: 'account:${name}' },
            name:   { type: String },
        }
    }
}

//  Fully typed Account object based on the schema
type AccountType = Entity<typeof schema.models.Account>

let account: AccountType = {
    name: 'Coyote',        //  OK
    unknown: 42,           //  Error
}

//  Create a model to get/find/update...

let Account = new Model<AccountType>(table, 'Account')

let account = await Account.update({
    name: 'Acme',               //  OK
    unknown: 42,                //  Error
})

account.name = 'Coyote'         //  OK
account.unknown = 42            //  Error

Why OneTable?

DynamoDB is a great NoSQL database that comes with a steep learning curve. Folks migrating from SQL often have a hard time adjusting to the NoSQL paradigm and especially to DynamoDB which offers exceptional scalability but with a fairly low-level API.

The standard DynamoDB API requires a lot of boiler-plate syntax and expressions. This is tedious to use and can unfortunately can be error prone at times. I doubt that creating complex attribute type expressions, key, filter, condition and update expressions are anyone’s idea of a good time.

Net/Net: it is not easy to write terse, clear, robust Dynamo code for single-table patterns.

Our goal with OneTable for DynamoDB was to keep all the good parts of DynamoDB and to remove the tedium and provide a more natural, “JavaScripty / TypeScripty” way to interact with DynamoDB without obscuring any of the power of DynamoDB itself.

More?

You can read more in the detailed documentation at:

• GitHub OneTable or
• NPM OneTable.

We also have several pre-built working samples that demonstrate OneTable.

• OneTable Overview Sample — A quick tour through OneTable.
• OneTable CRUD Sample — Basic CRUD.
• OneTable TypeScript Sample — Basic TypeScript use.
• OneTable Migrate Sample — how to use OneTable Migrate and the Migrate CLI.
• OneTable Packed Attributes Sample — How to use packed attributes.
• All OneTable Samples

EmbedThis with OneTable

At EmbedThis, we’ve used the OneTable module extensively with our EmbedThis Ioto IoT service. All data is stored in a single DynamoDB table and we extensively use single-table design patterns. We could not be more satisfied with DynamoDB implementation. Our storage and database access costs are insanely low and access/response times are excellent.

Links

• OneTable Discussions
• GitHub OneTable
• NPM OneTable