Creating Plugins

Plugins are a special type of package that provides extended processing for the Expansive pipeline. Plugins provide services that are used to transform a file from one format to another. For example: the exp-md plugin provides the compile-markdown-html service that transforms Markdown files with a .md extension into HTML files with a .html extension.

Plugin Structure

Plugins are delivered as Pak packages. They always contain a minimum of two files:

package.json — that describes the package
expansive.es — that implements the plugin

The package.jsonfile contains the name of the plugin, the current version and the list of dependent packages required for the plugin.

The expansive.es file is a Javascript file that typically contains just one call to Expansive.load to define the services provided by the plugin. For example:

Expansive.load({ 
    services: [ {
        name: 'my-markdown',
        options: 'my configuration options',
        init: function(transform) {
        },

        transforms: [ {
            name: 'compile',
            mappings: {
                'md': 'html'
            },
            init: function(transform) {
            },
            resolve: function(path) {
                return path
            },
            render: function(contents, meta, transform) {
                return contents
            },
            pre: function(transform) {
            },
            post: function(transform) {
            },
        } ]
    } ]
})

The plugin calls Expansive.load and provides a services array of one or more service definitions. Each service is a hash of properties that contains a name and a set of transforms. The transforms applied to specific content mappings that defines the file types for which the transform applies.

Transforms can provide a set of callback functions that will be invoked by the Expansive transformation pipeline as required. A transform may provide a name property that is appended to the service name to create a unique transform name. If omitted, the name of the service is used for the transformation name.

Expansive Pipeline

The Expansive pipeline calls plugin services to transform files from one format to another. It does this by calling a transform render function and providing it with the current file contents, meta data and service configuration. The order of transformations is determined by the content filename extensions. For example, consider the file:

index.html.md.exp

This file uses embedded Javascript (exp) that is run to generate a Markdown file (md), that is to be converted to HTML (html). Note extensions are read right to left. To implement this, the pipeline will use the transformations:

exp → md
md → html

The pipeline performs these transformations by invoking the plugin service transforms have the requisite mappings:

exp
markdown

Null Transforms

Plugins can also define a service for that maps a file type to itself. For example: html → html. This "null" transformation can be used for the last extension in a filename to perform final stage processing when the contents are mutated, but the file type does not change. For example: to minify a script file.

Mappings

A service defines the file types for which it should apply via the mappings property. The mappings takes the following format:

mappings: {
    'from': 'to',
    'from': [ 'to' ],
}

The from property is the original (outer-most) file extension. The to property is the destination (inner) file extension. It may also be an array of destination extensions. Expansive supports two abbreviated mappings syntax. Mapping entries may be set to a single string if the from and to values are the same. The Mappings property may be also set to a file extension string if only one extension is supported and the from and to file extensions are the same. For example:

mappings: 'html'

mappings: {
    'html'
}

Init Function

A service may define an init function to be invoked before processing commences. A transform may also define an init function for a similar purpose.

Resolve Function

Some transformations may need to alter the destination filename. For example, a Javascript minifier may change the output filename by using a .min.js extension. The resolve function is invoked to determine the final destination filename for the contents. The filename may be prefixed with a "|" (pipe) character to signifiy that this is the end of the pipeline processing and not more analysis of the filename extensions should take place. Returning null signifies that this content is not required.

Render Function

The render function is invoked to transform the file contents. It is passed three arguments:

contents — The current file contents as an in-memory string.
meta — Meta data properties for the file.
transform — The transform data object containing a reference to the service configuration.

The render function should process and return the new contents. The transform can instruct the pipeline to delete the file by returning null.

Pre Function

If a transform defines a pre function, it will be invoked before all pipeline processing. This "pre-processor" is useful to configure the environment or dynamically create pages. It is passed two arguments:

meta — Meta data properties for the file.
transform — The transform data object containing the service configuration.

Post Function

If a transform defines a post function, it will be invoked after all pipeline processing to perform "post" processing. It is passed two arguments:

meta — Meta data properties for the file.
transform — The transform data object containing the service properties.

Using Meta Data

Expansive defines various meta data properties for the original filename, destination filename and various other useful path properties.

Property	Description
source	Current source file being processed. Relative path including contents, layouts or partials directory prefix. For example: contents/sub/index.html.
sourcePath	Current source file being processed. Relative path excluding contents, layouts or partials directory prefix. For example: sub/index.html.
dest	Destination filename being created. Relative path including dist. For example: dist/sub/index.html.
destPath	Destination filename being created. Relative path excluding dist. For example: sub/index.html.
base	Source file without contents or lib. For example: sub/index.html.
document	Source of the document being processed. For partials or layouts, it is set to the invoking document. For example: sub/index.html.
url	Url reference made from path. For example: sub/index.html.
top	Relative URL to application home page. For example: ../.

See Meta Data for the full list of meta data properties.

Installing Plugins

Some plugins need to rely on external programs. If these external programs are delivered via Pak packages, then the plugin can simply specify the other package in its dependency list. If the plugin needs to configure the environment or run other external commands before it can run, then some installation setup may be needed.

The plugin package.json file may specify a script to run once the plugin package is installed. This script can then perform any required plugin configuration. Packages can "hook" two key package events:

postcache — run after the package is installed in the ~/.paks package cache
install — run after the package is locally installed under ./paks

For example, the exp-js package.json file contains the following event script:

"scripts": {
    "postcache": {
        "script": "Cmd.locate('uglifyjs') || run('npm install -g uglifyjs')" 
    }
}

This will try to locate the uglifyjs command and if not found, will use npm to install it globally.

Plugin Configuration

The Plugin services may define default configuration in their service definition that may then be overridden by user configuration in the top level expansive.json. A service may define any custom property in the service transform definition. For example:

Expansive.load({ 
    services: {
        name: 'minify-html',
        options:  '--remove-comments'
        ...
    }
})

The options property defines the default HTML minification options. The user can then override in the expansive.json via:

{
    services: {
        'minify-html': {
            options: '--remove-comments --remove-attribute-quotes'
        }
    }
}

Expansive creates an enable property for all services so the service does not need to explicitly create it.

Example

Here is a sample plugin that transforms shell scripts and captures their output. This plugin implements the shell service and will handle files with a .bash and .sh extension.

Expansive.load({
    services: {
        name: 'shell',
        transforms: {
            mappings: {
                'bash',
                'sh'
            },
            render: function(contents, meta, transform) {
                return run(file.extension, contents)
            }
        }
    }
})

Here is a sample plugin that implements the compress service to post-process files using gzip.

Expansive.load({
    services: {
        name:   'compress',
        files:  [ '**.html', '**.css', '**.js' ],
        transforms: {
            post: function(transform) {
                let gzip = Cmd.locate('gzip')
                if (!gzip) {
                    trace('Warn', 'Cannot find gzip')
                    return
                }
                for each (file in directories.dist.files(transform.service.files, {directories: false})) {
                    file.joinExt('gz', true).remove()
                    Cmd.run('gzip ' + file, {filter: true})
                }
            }
        `
    }
})

In this example, the plugin access the service files property for the set of file patterns to match.

Plugin Scripting

Expansive provides full access to the entire Ejscript Javascript language. See the Ejscript API Documentation and Ejscript Script Library for full details.

Expansive Convenience Functions

Expansive provides several convenience routines to make writing plugins easier.

Syntax	Description
addItems(collection: String, items: String\|Array)	Add items to the named collection.
getFiles(patterns: String\|Array, query: Object): Array	Query the meta data for matching files and return list of matching filenames.
getFileMeta(path: String): String	Get the file meta data for the given path.
getItems(collection: String): String	Return the items for a named collection.
removeItems(collection: String, items: String \| Array)	Remove items from the named collection
trace(tag: String, ...args)	Emit trace to the console.
vtrace(tag: String, ...args)	Emit trace to the console if expansive is run in verbose mode.
run(cmd: String, contents: String): String	Run the command with the contents as standard input and return the output of the command.

Popular Plugins

Here are some sample plugin implementations

exp-babel — Babel for ES6 support.
exp-css — CSS management.
exp-gzip — File compression.
exp-js — JS file management.
exp-less — Less stylesheet processing.
exp-markdown — Markdown file processing.
exp-sass — Sass stylesheet processing.
exp-shell — Shell script processing.