Streamlined DynamoDB with OneTable and TypeScript

TypeScript is a scripting language built on JavaScript that provides strong typing support to describe the types and object shapes used in your applications.

DynamoDB is increasingly being used with TypeScript where it provides strong API checks and guarantees. However, it is natural to want TypeScript type support for both the API and the data entities that are passed to and from the database.

This post discusses how the DynamoDB OneTable library uses dynamic TypeScript support fully type check DynamoDB data items and create reliable type-checked data management for your apps.

The Problem

DynamoDB is a NoSQL, key-value document database. As such, it imposes no schema on the data stored. This lack of a schema becomes increasingly apparent when using Single Table design patterns where all (or most) application data entities are stored in a single DynamoDB table and attributes are overloaded with different data values for the various entities.

Developers using DynamoDB are increasingly turning to TypeScript as it provides them with strong guarantees regarding API correctness via type signatures for APIs. These TypeScript users also want the same type guarantees when reading and writing database data. However, how do you create type signatures for untyped data that is being stored in a NoSQL database with no schema?

DynamoDB OneTable solves this issue and provides data typing for both APIs and database entities and attributes.

OneTable is an access library for DynamoDB applications that makes dealing with DynamoDB and single-table design patterns dramatically easier. OneTable provides TypeScript type declarations for the public API. However, this is just the start, because via TypeScript dynamic typing, OneTable creates new types automatically to validate database entities and attributes.

Why a Schema?

TypeScript users may ask “Why use a Schema?” Why not just derive everything from a foundation of TypeScript types?

The answer is that while TypeScript type declarations do provide type checking and API guarantees, they do not further validate the data being written to database. Database data typically needs more constraints that just basic typing.

A OneTable schema provides definition and control over the following data attribute properties:

• Data type
• Data value range checking via regular expressions
• Whether the attribute is required when created
• Whether the attribute value is unique across other items
• A default value if not supplied by the caller
• A derived value based upon other attribute values

For example, here is a basic OneTable schema.

const MySchema = {
    version: '0.1.0',
    format: 'onetable:1.0.0',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
    },
    models: {
        Account: {
            pk:          { type: String, value: 'account:${name}' },
            sk:          { type: String, value: 'account:' },
            id:          { type: String, uuid: true, validate: /^[0-9A-F]{32}$/i },
            name:        { type: String, required: true },
            status:      { type: String, default: 'active' },
        },
        User: {
            pk:          { type: String, value: 'account:${accountName}' },
            sk:          { type: String, value: 'user:${email}', validate: EmailRegExp },
            id:          { type: String, required: true },
            accountName: { type: String, required: true, unique: true },
            email:       { type: String, required: true },
            firstName:   { type: String, required: true },
            lastName:    { type: String, required: true },
            role:        { type: String, enum: ['user', 'admin'],
                           required: true, default: 'user' },
            balance:     { type: Number, default: 0 },
        }
    }
}

TypeScript OneTable

Before explaining how OneTable generates dynamic types, let’s see what it looks like.

To create a TypeScript type for an entity, we use the TypeScript typeof operator:

type Account = Entity<typeof MySchema.models.Account>

The Account type will now fully validate property access and catch invalid references and data assumptions. For example:

let account: Account = {
    name: 'Coyote',        //  OK
    unknown: 42,           //  Error
}

To access DynamoDB, we use a previously constructed Table object to get an access model object.

//  Get an Account access model
let AccountModel: Model<Account> = table.getModel('Account')

The AccountModel provides access to the find, get, remove, update and other OneTable APIs to interact with DynamoDB.

let account = await AccountModel.update({
    name: 'Acme',               //  OK
    unknown: 42,                //  Error
})

account.name = 'Coyote'         //  OK
account.unknown = 42            //  Error

TypeScript Magic

The challenge in creating dynamic types involves tackling three key issues:

• Creating type declarations for each entity (MySchema.models.*)
• Creating type declarations for the attributes under each entity (MySchema.models..)
• Interpret the schema attribute data type and create a corresponding TypeScript type

OneTable uses several TypeScript operators to dynamically create types from a OneTable schema.

These include:

• Indexed access types

• Mapped types

• Keyof types

Indexed Access Types

To create type declarations for each model, we used TypeScript indexed access types. This dynamically creates type signatures for each model defined in the schema.

type OneSchema = {
    models?: {
        [key: string]: OneModelSchema
    },
};

Similarly, the OneModelSchema creates type declarations for each attribute in the model.

export type OneModelSchema = {
    [key: string]: OneFieldSchema
};

For each entity attribute, we sleuth the attribute type: property and define a type union that selects the appropriate data type for the attribute.

type EntityField<T extends OneTypedField> =
      T['type'] extends StringConstructor ? string
    : T['type'] extends NumberConstructor ? number
    : T['type'] extends BooleanConstructor ? boolean
    : T['type'] extends ObjectConstructor ? object
    : T['type'] extends DateConstructor ? Date
    : T['type'] extends ArrayConstructor ? any[]
    : never;

Finally, we create a generic Entity type that is used to create entity types based on the schema.

export type Entity<T extends OneTypedModel> = {
    [P in keyof T]?: EntityField<T[P]>
};

The end result of all this TypeScript magic is fully dynamic type declarations for your data entities and attributes based on a single source of truth: the OneTable schema.

Here is the complete OneTable Model.d.ts dynamic type definition file in GitHub:

• https://github.com/sensedeep/dynamodb-onetable/blob/main/src/Model.d.ts

Working Sample

If you’d like to try it out, check out our working sample at:

• OneTable TypeScript Sample
• All OneTable Samples

TypeScript and OneTable

By using dynamic TypeScript operators, OneTable is able to provide strong type guarantees for both its API and for your database entities and attributes. The net result is fewer errors, earlier detection of errors and faster serverless development.

References

• Dynamic Static Typing
• TypeScript Advanced Typing
• TypeScript Handbook

Links

• OneTable
• DynamoDB Checklist
• DynamoDB Single Table Design
• DynamoDB Sparse Indexes
• OneTable Migrate CLI