Configuring ESP

When serving stand-alone esp pages, you do not need any external ESP configuration. ESP provides sensible defaults and you can just run esp serve to serve pages.

$ esp serve

However, larger ESP applications are configured via the esp.json, pak.json and expansive.json files that permits fine-grained control over how the application will run

The pak.json file manages:

The esp.json file manages:

The expansive.json will be installed by most skeletons. It manages:

This document will cover the various esp.json properties and the steps to configure ESP applications. See the Expansive Documentation for details about the expansive.json.

If you are hosting your application in Appweb you will need to specify additional configuration in the appweb.conf configuration file. For details, see the Hosting in Appweb.

pak.json

The pak.json file contains your application name, description, version and specifies any required packages that your application depends upon. The format is a JSON literal. For example:

{ "name": "blog", "title": "My Blog", "description": "My blog with thoughts and tales from my travels in Africa", "version": "1.0.0", "dependencies": { "jquery": "^2.1" } }

Name and Description

All ESP applications have name, title, description and version properties that uniquely identify the application. The application name is used in C function entry points, so it is important that it only contain valid C identifer characters. The title provides a multi-word title suitable for display to the user.

Dependencies

The dependencies hash lists all the installed packages and the required versions. Entries will be added as packages are installed via pak install. For example:

$ pak install jquery

ESP relies heavily on the Pak package manager to install and manage extension packages.

esp.json

The esp.json file provides configuration for ESP and how HTTP requests will be served. The format is an enhanced JSON literal that supports property names without quotes, multi-line literals and commas on trailing properties. Here is the initial generated contents of the esp.json

{ esp: { app: true }, http: { server: { listen: [ "http://127.0.0.1:4000" ] }, ssl: { certificate: "", key: "" } } }

Creating Configuration Files

To create an esp.json and pak.json, run the following in an empty directory.

esp init

Alternatively, or if you install an ESP skeleton, then the esp.json and pak.json files will be created for you. For example:

pak install embedthis/esp-html-skeleton

Configuration Mode

ESP provides a convenient way to select a set of properties that correspond to a desired execution profile such as: debug, test or release profiles. In each profile, different configuration may be defined.

The selected profile is specified by the profile property in the pak.json file. This then selects a collection of properties of that same name that will be copied to the top-level before the application runs. By modifying the profile setting, an entirely different execution configuration may be selected. ESP makes this easy via the command:

$ esp profile debug

This will set the pak profile property to debug in the pak.json file and select the debug collection of properties in the esp.json file. The profile property is centralized in the pak.json file because the Pak and Expansive utilities follow a similar scheme and having one single profile property is desirable over each utility replicating the profile definition.

pak.json Properties

Name Description
description Short sentence description of the application.
dependencies Extension packages used by the ESP application. The key is the pak name and the value is a SemVer version expression. See Pak Documentation for details.
directories Directory locations
include List of external files to include.
name One word application name. This value is used to name the application and will be used in controller and application function entry point names.
profile Configuration profile for the application that selects a property collection of the same name from the profiles collection in the pak.json and esp.json files. Those properties are copied up to the top level.
title Multi-word application name to display.
version Application version conforming to SemVer.

Dependency Expressions

Dependent extension packages may be specified in the dependencies property. This is an object collection of paks with version expressions. When Pak installs a package, an entry for the new pak will be added to the dependency collection. For example:

"dependencies": { "jquery": "1.11.0", "bootstrap": "3.0.0", }

Dependency Versions

ESP versions comply with the SemVer version specification. Briefly, this means versions are expressed :

MAJOR.MINOR.PATCH-PRERELEASE

Dependency versions may also include expressions that describe a range of acceptable pak versions.

For example:

"dependencies": { "bootstrap": "~3", /* Versions 3.0.0 or later including pre-releases */ "jquery": "^1.9.1", /* Version 1.9.1 and not pre-releases */ "less": "1.3.x", /* Version 1.3.x and pre-releases / }

Directory Configuration — directories.*

Name Description
cache Directory for caching compiled items. Defaults to cache.
controllers Directory for server-side MVC controllers. Defaults to "controllers".
db Directory for server-side database files. Defaults to "db".
documents Directory containing client visible document content. All files under this directory are visible to the client. Defaults to documents.
home Home directory for the application. Defaults to ".".
layouts Directory for Expansive layout pages. Defaults to "layouts".
lib Directory for public extension pack scripts and resources. Defaults to "client/lib".
paks Directory to contain installed packages. Defaults to "./paks".
partials Directory for Expansive partial pages. Defaults to "partials".
pakcache Location of the per-user cache of packages. Defaults to "~/.paks".
source Source of input documents for Expansive to render. Defaults to "source".
src Directory for application C/C++ code. Defaults to "src".
upload Directory for uploaded files. Defaults to "/tmp".

Esp.json Properties

Name Description
esp Collection of ESP configuration.
http Collection of HTTP configuration.
profiles Collection of profile specific configuration. Typically there will be collections for debug or release but can be any identifier word.

Client Configuration — http.client.*

Name Description
mappings

Set of property keys that map server-side properties to client-side properties to send to the client. Mappings select a subset of properties so the entire JSON configuration is not sent to the client. For example:

mappings: { auth: 'http.auth', timeouts: { session: 'http.timeouts.session', } }

This computes a JSON string with properties "auth" and "timeouts" to send to the client.

render

Ordered set of scripts to send to the client when parsing the ESP scripts() API. This may include the "*" wildcard to match any file or "**" to match any file in any directory. For example:

render: { js: [ 'lib/angular/angular.js', 'lib/angular/angular-animate.js', 'lib/angular/angular-route.js', 'main.js', '*/**.js*' ] }

ESP Collection — esp.*

Name Description
app

Define whether the directory contains an ESP application or stand-alone ESP pages. For example:app: true

An ESP application follows an MVC design pattern and provides controllers to respond to client requests and views to render responses. An ESP application differs from stand-alone ESP pages in that it pre-configures ESP for the MVC pattern. This includes defining the standard directories: cache, controllers, contents, db, layouts, migrations, paks and partials.

apps

Define external ESP applications to load. ESP can host multiple applications by defining the URI prefix and ESP configuration file for each application. This directive takes three forms. The first is just a simple string with the path to the configuration file for a single application. The URI prefix is determined by the last path segment before the esp.json filename.

The second form takes an array of esp configuration file names. Both these forms may use wild-cards in the filename. The last form is an array of URI prefix and config file properties for each application.

Examples: apps: 'blog/esp.json', apps: [ 'apps/*/esp.json', 'test/*/esp.json' ], apps: [ { prefix: '/blog', config: 'apps/blog/esp.json' } ]

build Collection of compilation and link command lines for various platforms.
combine Define whether to combine the entire ESP application into a single output source file and library. This is typically enabled for release profile. Defaults to false.
compile Enable compiling controllers, modules and views at runtime. Defaults to true.
optimize Define whether to compile optimized or with debug symbols. Set to "true" for an optimized build and to "false" for building with debug symbols. Defaults to false.
generate Collection of properties guiding the esp generate commands.
keep Preserve intermediate source code in the cache directory for debugging. Defaults to false unless the pak.json profile is set to "debug".
preload List of source files to preload when starting the application. For example: preload: [ 'auth.c' ]
rules Path to the JSON file containing the compilation rules. Defaults to esp-compile.json in the ESP installation directory. If you want to modify this file, take a copy and then set the compile property to point to it.
update Determine if updating of responding applications and pages is enabled. Some web frameworks will automatically rebuild or reload applications if the source is modified.

Http Configuration — http.*

All properties in the http collection with the exception of the http.server collection apply per-route. This means these properties they may be nested in routes defined under http.routes. The http.server collection is ignored if ESP is hosted in a web server such as Appweb.

Name Description
aliases Collection of URL to directory mapping aliases. The property key is the leading URI prefix and the value is the directory
attach List of listen endpoints on which to attach a virtual host. For example: attach: [ '*:80', ],

If there is no attach property, the virtual host will attach to all listening endpoints.

auth Collection of authentication configuration.
cache List of cache directives.
compress

Enable serving of compressed content. ESP can serve files compressed with gzip and with an additional .gz extension. ESP adds the appropriate HTTP header so the browser can automatically decompress. Ideal for big files or slow networks.

compress: true or compress: [ 'html', 'js', 'css' ]

If the Expansive exp-gzip extension is installed, suitable static content will be automatically gzipped.

database Database to create and open for the application. Of the form: provider://name.ext where provider is set to either "mdb" for the embedded memory database or "sdb" for SQLite. The extensions are "mdb" or "sdb" by convention. Set to "default" for a database named after the application with the default database provider.
deleteUploads Define if uploaded files should be automatically deleted after the request or if they should accumulate in the upload directory.
domain Matching domain for virtual hosting. For example: domain: '*embedthis',
errors Collection of error pages to serve for various HTTP status codes. For example: errors: { '404': '/notFound.html, },
headers Collection response header directives. This includes sub-collections for headers to "add", "remove"" and "set". The "remove"" collection is a list, whereas "add" and "set" are collections. For example: headers: { add: { 'Content-Security-Policy': 'allow "self"; unsafe-inline; img-src "self"', }, remove: [ 'Etag' ], set: { 'Custom-Header': '43', } },
hosts

A single server may support multiple virtual hosts on a shared IP address endpoint. The client HTTP request includes a Host header that specifies the desired hostname. The supported virtual hosts are configured via the hosts directive which specifies one more more virtual host configurations.

Each virtual host may include other directives to customize the configuration and routes for that specific host. Each host may nominate the specific listen endpoints for the host via the attach property. Without an attach property, the virtual host will attach to all listening endpoints.

Example:

hosts: [ {
    name: '*embedthis.com',
    attach: '*:80',
}, {
    name: '*appwebserver.org',
    attach: [
        '*:80',
        '*:4443'
    ],
}]
indexes Ordered list of index documents to serve for directory level requests. For example: indexes: [ 'index.esp', 'index.html' ],
languages Collection of language definitions to customize served content based on preferred language. For example: languages: { en: { path: '/path/to/english' prefix: 'en', suffix: 'en', default: true, } } Typically, you would choose one of prefix, suffix or path
limits Resource limits to "sandbox" the server. Note that values may either be numbers or strings using natural language suffixes like MB or GB. The rxBody, rxForm, txBody and upload properties may be set to unlimited to disable the limit checking. For example: limits: { buffer: '32KB', cache: '10MB', cacheItem: '200KB', chunk: '64KB', clients: 100, connections: 50, files: 0, keepAlive: 200, processes: 0, rxBody: '100K', rxForm: '32K', rxHeader: '32K', memory: '200MB', requests: 20, sessions: 100, txBody: '2GB', upload: '2GB', uri: '8K', webSockets: 20, webSocketsMessage: '50K', webSocketsPacket: '50K', webSocketsFrame: '4K', workers: 4, },
methods List of acceptable HTTP methods. For example: methods: [ 'get', 'post', ], Can also set "*" for all possible methods. The default set of methods is: "DELETE, GET, OPTIONS, POST, PUT".
params List of required parameters before the request will match the current route. For example: params: [ { name: 'name', value: 'john', equals: true, } ],
pattern

Matching URI regular expression for requests to be served by the current route. The route pattern is an enhanced JavaScript-compatibile regular expression. It is enhanced by optionally embedding braced tokens "{name}" in the pattern. During request URI matching, these tokens are extracted and defined in the request params and are available to the request and to route targets. To use the standard regular expression repeat syntax {}, quote the braces with back quote \.

Sub-expressions and token expressions are also available in route targets as numbered tokens $N. For example:

pattern: '^/app/(.*)(\.html)$',

This will enable the target $1.${request.Language=fr}.$2.

Use ^ to anchor to the start of the URI and $ for the end of the URI. If the pattern does not start with ^, the route pattern will have its parent route prefix prepended to the pattern.

pipeline Collection defining the request/response pipeline including filters and handlers. The Filters property is an ordered list of filters which modify the incoming and outgoing request data. Each entry specifies the filter name and the matching URI extensions. Use "*" to match all extensions. The handlers hash specifies supported handlers and the matching URI extensions. For example: pipeline: { filters: [{ name: 'uploadFilter', extensions: [ 'txt', '*' ], }], handlers: { fileHandler: [ 'html', 'gif', 'jpg', 'png', 'pdf', 'ico', 'css', 'js', 'txt' ], espHandler: [ '*' ], }, },
prefix URI prefix to remove from request URIs before passing to the application.
redirect

Collection of redirection rules. These include the failing URI, destination URI and redirection status.

redirect: [ { from: '/pressRelease', status: 301, to: 'https://${request:serverName}/fixedPressRelease.html' } ]

Alternatively, a rule may be just the string "https://" in which case all HTTP content will be redirected over HTTPS.

renameUploads Automatically rename uploaded files to use the client supplied filename after sanitizing. ESP typically does not use this property and rather, an ESP controller or ESP page should validate the client details and copy / rename the uploaded temporary file as required. Recommended to not use this with ESP and only ever use on a private network. Defaults to false. accumulate in the upload directory.
resources

Create routes sets for the requested resources. Route sets select pre-defined route sets. Some of the available sets are: 'esp-server' and 'esp-restful'. Resource groups create RESTful routes for a resource that has multiple instances. Singletons create RESTful routes for singleton resources.

resources: { sets: [ 'esp-restful' ], groups: [ 'users', 'administrators' ], singletons: [ 'profile' ], }
routes Ordered list of routes. This may be a pre-defined route set name or it may be an array of routes. Examples: routes: 'esp-restful', routes [ 'esp-restful'], routes [ { pipeline: { handlers: 'espHandler' } /* Any collection of properties */ }
scheme Define the required HTTP protocol scheme for all requests. Set to "https" to permit only secure connections. Alternatively use redirect: "https://" to redirect HTTPS traffic over to HTTPS.
server Collection of properties that apply to the whole server and not per-route. The server collection is special in that it is only parsed when ESP is not hosted in a web server such as Appweb. If hosted, it is expected that the web server will define these attributes externally.
showErrors Controls if application errors should be displayed in the browser. Set to true to display errors in the browser. Defaults to true if the source is compiled in for debug and false if compiled in release profile.
source Source file for the responding application. Used by ESP controllers.
ssl Collection of SSL configuration.
stealth Respond in stealth mode emitting as little context information as possible.
stream Define whether the request body data should be buffered or stream. This property should be an array of objects with "mime", "uri" and "stream" properties. For example: stream: [ { mime: 'application/custom-mime', stream: false, uri: '/some-uri' }, ],
target Routing destination for processing the request.
timeouts Collection of timeouts. The timeout units are in seconds but may also be defined as strings with natural suffixes such as "20secs", "1day" or "unlimited".
timeouts: {
    exit: '5secs',
    parse: '5secs',
    inactivity: '30secs',
    request: 'unlimited',
    session: '30mins',
}
trace Trace filtering collection. For example: trace: { location: 'stdout', level: '2', backup: 5, anew: true, size: '10MB', timestamp: '1hr', maxContent: 50000, },

The trace properties are:

  • location — Filename to trace to. May also be 'stdout', or 'stderr'.
  • level — Trace level (0-5) with 5 being the most verbose.
  • backup — Number of trace backups to keep.
  • anew — Start trace log anew each time the application restarts.
  • size — Maximum size of the trace log before it will be restarted.
  • timestamp — Write a timestamp at this frequency.
  • maxContent — Maximum size of request body content to trace. Defaults to 40K.
xsrf Control if an XSRF anti-forgery token should be checked for POST forms. If enabled, server side ESP form pages should use the following to generate a hidden input field that contains the XSRF token. <% inputSecurityToken(); %> Ajax requests should use the following to add the security token to the Ajax request payload. Both mechanisms will add an XSRF HTTP header named X-XSRF-TOKEN to the response. For example, this code generates the token on the server and utilizes its value in client-side Javascript. <script> var token = <%= securityToken(); %>; </script>

Note: restarting the server or creating a new ESP session will invalidate XSRF tokens. In this case, the form will need to be reloaded to generate a new token.

Http Server Configuration — http.server.*

The server collection is special in that it is only parsed when ESP is not hosted in a web server such as Appweb. If hosted, it is expected that the web server will define these attributes externally. All server properties apply to the whole server and are not modifiable per-route.

Name Description
account User and group account names. If defined, ESP must be run as super-user to be able to change accounts. For example: http: { server: { account: { user: 'www-data', group: 'www-data', },
chroot Path to set as the root directory when running using the chroot API.
defenses Set of defense rules that can be utilized by monitors. defenses: { block: { args: { status: 406, message: 'Client temporarily banned due to monitored limit exceeded', remedy: 'ban', period: '30mins', }, }, },
listen List of IP:PORT endpoints on which to listen for requests. To listen on all network interfaces, use "*". To listen using SSL, add https:// as a prefix. For example: listen: [ '*:4000', 'https://127.0.0.1:4443', ],
log Log file configuration. For example: log: { location: 'stdout', level: '2', backup: 5, anew: true, size: '10MB', timestamp: '1hr', },
  • location — Filename to log to. May also be 'stdout', or 'stderr'.
  • level — Trace level (0-5) with 5 being the most verbose.
  • backup — Number of log backups to keep.
  • anew — Start log anew each time the application restarts.
  • size — Maximum size of the log before it will be restarted.
  • timestamp — Write a timestamp at this frequency.
monitors Set of counters to monitor for unusual activity. For example: missing: { expression: 'NotFoundErrors > 5', period: '10 secs', defenses: [ 'block' ], enable: true, },

Authentication Configuration — http.auth.*

Name Description
auto Collection of automatic login related properties. The auto.name property defines an account name that the application will automatically login. This is useful during development to bypass the login step. The auto.roles property defines the roles for which the user will be authorized. The url property defines the URI for the login dialog.
login Login page when using the form authentication scheme. If the user is not logged in, they will be redirected to this page.
realm Login realm string if using Basic or Digest authentication abilities.
require The permitted user account names or roles that are required before authorization for this route. For example: require: { roles: [ 'administrator'], users: [ 'julie' ] }, If users is defined, then a user must login before using the application. The users property may be set to a list of users, a string with a single user name or true to imply any authenticated user. Note: it is application dependent how the required login feature is implemented.
single Set to true to permit only one simultaneous login. Ideal for some management user interfaces. Note: it is application dependent how the required login feature is implemented.
roles Definition of roles that users may perform. Each role consists of a set of abilities or roles. For example: roles: { executive: ['manage', 'direct'], user: ['view'], administrator: ['view', 'edit'], },
session

Configure the user session details. The cookie property can define a unique prefix to use for the session cookie. The default session cookie prefix is: "-http-session-". If set to 'none', then no cookie is emitted. The visibleproperty specifies whether the cookie is readable by scripts in the browser. By default, the session cookie is not visible to Javascript. It is desirable to keep the session cookie invisible to minimize XSS security vulnerabilities.

ESP implements this by adding the httpOnly option on the Set-Cookie response header.

The persist property defines if the cookie is persisted to disk in the browser. If false, the browser discards the cookie when the browser exits. Otherwise the cookie is retained until the session timeout limit expires.

session: { cookie: "-esp-cookie-", persist: true, visible: true, },
store Authentication store. Set to "app", "config", or "system".
type Authentication type. Set to "basic", "digest" or "form". No need to define if using the "app" authentication store.
users Definition of users, their passwords and roles if using the "config" authentication store. For example: users: { ralph: { password: 'BF1:00128:bWFC5x45BYfuf2GW:w+zzIU0wTZQYaNae0sPecWyry8Z9GrKH', roles: ['user', 'administrator', 'purchase'], }, },

Cache Collection — http.cache.*

The cache property contains a list of cache records. For example:

cache: [{ client: '1day', extensions: [ 'css', 'html', 'js' ], }],

The cache property may also be set to true which is equivalent to defining caching on the client side for one day for common static file types.

Name Description
extensions List of extensions to cache
client Define the lifespan of content cached in the client's browser. For example: client="1hour
server Define the lifespan of content cached in the server. For example: server="1hour
manual If true, then do not transparently cache qualifying response output. Rather, the application must seed the cache manually via httpWriteCached
methods List of HTTP request methods to cache.
mime List of request content mime types to cache. For example: mime: [ 'image/jpeg', 'image/png' ]
unique Uniquely cache requests using the full URI including the request query and parameters. By default, only the URI without parameters is considered when caching content.
uris List of qualifying URIs to cache

SSL Collection — http.ssl.*

The content property describes how web content should be built and processed. The SSL directives may be used globally or per-route.

ssl { authority: { file: 'ca.crt', directory: './ca-dir', }, certificate: 'mycert.crt', ciphers: [ 'TLS_RSA_WITH_AES_256_CBC_SHA', 'TLS_RSA_WITH_RC4_128_SHA' ], key: 'mykey.key', verify: { client: true, issuer: true, }, },
Name Description
authority Certificate authority file or directory. Choose one of these two properties to define the CA certificates for verifying clients. Only used if ssl.verify.client or ssl.verify.issuer is true.
certificate Path to the SSL certificate to use for connections on this route.
ciphers List of acceptable ciphers to use. These names are dependant on the SSL provider selected The format of this directive varies depending on the SSL protocol stack. The OpenSSL stack uses its own proprietary cipher naming. If using OpenSSL, please consult the OpenSSL documentation for how to format the ciphers. Other stack use the standard IANA Cipher Suite Registrya names.
key Path to the private key file for connections on this route.
verify Control whether client certificates are required and verified. If verify.client is true, the client must provide a valid certificate. If verify.issuer is true, the client's certificate must be verifiable by the specified authority certificate.

Timeouts Collection — http.timeouts.*

The timeouts property specifies the various request and connection timeouts. Timeouts are expressed as strings with natural language suffixes like: "30mins".

Name Description
exit Maximum exit delay while waiting for requests to complete.
parse Maximum time to receive a the request headers. Default 5 seconds.
inactivity Maximum time a connection will be preserved in the absence of I/O activity. Default 30 seconds.
refresh Default time to refresh pages with dynamic content.
request Maximum time for a request to complete. Default infinite.
session Maximum time to preserve session state storage for a client. Default 5 minutes.

© Embedthis Software. All rights reserved.