Using CGI
The Common Gateway Interface (CGI) was originally developed as part of the NCSA HTTP server and is an old standard for interfacing external applications with HTTP servers. It still enjoys considerable use.
CGI was created to allow dynamic data to be generated in response to HTTP requests and return the results to the clients's browser. Plain HTML documents are typically static, while a CGI program allows the response data to be dynamically created.
However, since CGI was first developed, several better means of creating dynamic web pages have been created that are faster and more efficient. Read more about such replacements in Web Frameworks.
Embedthis Appweb supports CGI so that existing CGI applications can be fully supported. Appweb has a high-performance and fully featured CGI Handler that alleviates many of the pains with configuring CGI setup.
Configuring CGI Programs
Requests for CGI programs may be configured in two primary ways:
- By URI prefix
- By extension
When invoked by URI prefix, the CGI programs and scripts are stored in special CGI directories (for example "cgi-bin"). When invoked by URI extension, the CGI programs may be stored anywhere in the web directory.
For security, it is usually best to store all CGI programs and scripts outside the directory containing the web content. Consequently, invoking CGI programs by extension should only be used in combination with a URI prefix that allows the CGI directory to be specified.
Invoking by URI Prefix
Appweb nominates a directory as a CGI directory via the ScriptAlias configuration file directive. For example:
ScriptAlias /cgi-bin/ $ROUTE_HOME/cgi-bin/
The ScriptAlias directive above is short hand for:
<Route ^/cgi-bin/(.*)$>
Documents $ROUTE_HOME/cgi-bin/
SetHandler cgiHandler
Target run $1
</Route>
When a URI is requested by a browser that includes the "/cgi-bin/" prefix, the script name immediately after "/cgi-bin/" will be run. For example:
https://www.embedthis.com/cgi-bin/testCgi
Invoking by URI Extension
To configure Appweb to run CGI programs based on the URI extension use the AddHandler configuration file directive. For example:
AddHandler cgiHandler .myExt
This configures Appweb to pass URIs that contain the ".myExt extension to the CGI handler.
Mime Types and Action Programs
To determine which program the CGI handler should run, the CGI handler looks up the mime type associated with the ".myExt" extension. Mime types are read by Appweb from the "mime.types" file at startup. This file maps extensions to mime types. This is an example line from the "mime.types" file:
application/x-perl myExt
This maps the ".myExt" extension to the perl mime type. This mime type must then be mapped to a program via the the Action directive. For example:
Action application/x-perl /usr/bin/perl
This will cause /usr/bin/perl to be run to process the request. Output from perl is captured by the CGI handler and then returned to the user's browser.
Invoking CGI Programs
When a CGI program is run, the Appweb CGI handler communicates request information to the CGI program via Environment Variables and in some cases, via the command line.
CGI Command Line
The command line will be set differently depending on how the CGI program is being invoked. There are four possible scenarios:
- Program invoked directly via the request URI.
- Program invoked indirectly if the CGI script contains a Bang path directive.
- The program is specified via an ActionProgram directive in the Appweb configuration file.
- On Windows if the program is a Windows Batch file
The command line arguments for the CGI program will be set differently in each case. See the tables below for the specifications as to how the command line arguments are defined:
Programs Invoked Directly via the Request URI
| Arg | Contents |
|---|---|
| argv[0] | Program name immediately after the CGI URI prefix (E.g. after /cgi-bin/) |
| argv[1..N] | Each arg is set to portions of the QUERY_STRING. The query is split at "+" characters after unescaping the query. |
Programs Invoked Indirectly with Bang Directive
If the CGI program/script specified in the URI contains a "#!/pathToProgram" directive on the first line, it is interpreted to be the path to the real CGI program to run. The script name is then passed in the command line.
| Arg | Contents |
|---|---|
| argv[0] | Program name defined in the first line of the CGI script after the "#!" characters. |
| argv[1] | The name of the CGI script originally specified in the URI. |
| argv[2..N] | Each arg is set to portions of the QUERY_STRING. The query is split at "+" characters after unescaping the query. |
Programs Specified via an ActionProgram Directive
| Arg | Contents |
|---|---|
| argv[0] | Program name specified in the ActionProgram directive in the Appweb configuration file. |
| argv[1] | The name of the CGI script originally specified in the URI. |
| argv[2..N] | Each arg is set to portions of the QUERY_STRING. The query is split at "+" characters after unescaping the query. |
Windows Batch Commands
| Arg | Contents |
|---|---|
| argv[0] | Set to "cmd.exe" |
| argv[1] | /Q |
| argv[2] | /C |
| argv[3] | Command |
The "Command" is a quoted string set to the name of the CGI script originally specified in the URI followed by the Query String, split at "+" characters. The entire Command string is escaped so that dangerous characters are preceded by "^" to prevent security attacks.
CGI Environment Variables
CGI uses environment variables to send your program additional parameters. The following environment variables are defined:
| Variable | Description |
|---|---|
| AUTH_TYPE | Set to the value of the HTTP AUTHORIZATION header. Usually "basic" or "digest". |
| CONTENT_LENGTH | Set to the length of any associated posted content. |
| DOCUMENT_ROOT | Set to the path location of web documents. Defined by the Documents directive in the Appweb configuration file. |
| GATEWAY_INTERFACE | Set to "CGI/1.1" |
| HTTP_ACCEPT | Set to the value of the HTTP ACCEPT header. This specifies what formats are acceptable and/or preferable for the client. |
| HTTP_CONNECTION | Set to the value of the HTTP CONNECTION header. This specifies how the connection should be reused when the request completes. (Keep-alive) |
| HTTP_HOST | Set to the value of the HTTP HOST header. This specifies the name of the server to process the request. When using Named virtual hosting, requests to different servers (hosts) may be processed by a single HTTP server on a single IP address. The HTTP_HOST field permits the server to determine which virtual host should process the request. |
| HTTP_USER_AGENT | Set to the value of the HTTP USER_AGENT header. |
| PATH_INFO | The PATH_INFO variable is set to the URI portion (if any) after the SCRIPT_NAME. |
| PATH_TRANSLATED | The physical on-disk path name corresponding to PATH_INFO. |
| QUERY_STRING | The QUERY_STRING variable is set to the URI string portion that follows the first "?" in
the URI. The QUERY_STRING is URI encoded in the standard URI format by changing spaces to "+",
and encoding all URI special characters with %xx hexadecimal encoding. Most major
scripting languages provide routines to assist in decoding the QUERY_STRING. |
| REMOTE_ADDR | Set to the IP address of the requesting client. |
| REMOTE_HOST | Set to the IP address of the requesting client (same as REMOTE_ADDR). |
| REMOTE_USER | Set to the name of the authenticated user. |
| REMOTE_METHOD | Set to the HTTP method used by the request. Valid values are: "GET", "HEAD", "OPTIONS", "POST", or "TRACE". "PUT" and "DELETE" are not supported. |
| REQUEST_URI | The complete request URI after the host name portion. It always begins with a leading "/". |
| SCRIPT_NAME | The name of the CGI script or program to run. If an ActionProgram is specifying the name of a CGI interpreter, then SCRIPT_NAME is set to the name of the script to interpret. |
| SERVER_ADDR | The IP address of the server or virtual host responding to the request. |
| SERVER_NAME | The name of the default server or virtual host serving the request. |
| SERVER_PORT | The HTTP port of the server or virtual host serving the request. |
| SERVER_PROTOCOL | Set to "HTTP/1.0" or "HTTP/1.1" depending on the protocol used by the client. |
| SERVER_SOFTWARE | Set to "Embedthis Appweb/VERSION" |
Example
Consider the following URI which will run the Perl interpreter to execute the "pricelists.pl" script.
http://hostname/cgi-bin/myScript/products/pricelists.pl?id=23&payment=creditCard
This URI will cause the following environment settings:
| Variable | Value |
|---|---|
| PATH_INFO | /products/pricelists |
| PATH_TRANSLATED | /var/appweb/web/products/pricelists # where /var/appweb/web is the Documents directory |
| QUERY_STRING | id=23&payment=credit+Card |
| REQUEST_URI | /cgi-bin/myScript/products/pricelists?id=23&payment=credit+Card |
| SCRIPT_NAME | myScript |
| ARGV[0] | /usr/bin/perl |
| ARGV[1] | pricelists.pl |
| ARGV[2] | id=23&payment=creditCard |
This URI below demonstrates some rather cryptic encoding of URIs. The important thing to remember is that command line arguments are delimited by "+". The hex encoding %20, is the encoding for the space character. Once passed to the CGI program, the convention is for CGI variables to be delimited by "&".
http://hostname/cgi-bin/cgiProgram/extra/Path?var1=a+a&var2=b%20b&var3=c
This URI will cause the following environment settings:
| Variable | Value |
|---|---|
| PATH_INFO | /extra/Path |
| PATH_TRANSLATED | /var/appweb/web/extra/Path |
| QUERY_STRING | var1=a+a&var2=b%20b&var3=c |
| REQUEST_URI | /cgi-bin/cgiProgram/extra/Path?var1=a+a&var2=b%20b&var3=c |
| SCRIPT_NAME | cgiProgram |
| ARGV[0] | cgiProgram |
| ARGV[1] | var1=a |
| ARGV[2] | a&var2=b b&var3=c |
URI Encoding
When a URI is sent via HTTP, certain special characters must be escaped so the URI can be processed unambiguously by the server. To escape the special characters, the HTTP client should convert them to their %hex equivalent. Form and query variables are separated by "&". For example: a=1&b=2 defines two form variables "a" and "b" with their values equal to "1" and "2" respectively.
CGI Programming
CGI program can return almost any possible content type back to the client's browser: plain HTML, audio, video or any other format. CGI programs can also control the user's browser and redirect it to another URI. To do this, CGI programs return pseudo-HTTP headers that are interpreted by Appweb before passing the data on to the client.
Appweb understands the following CGI headers that can be output by the CGI program. They are case-insensitive.
| Header | Description |
|---|---|
| Content-type | Nominate the content Mime Type. Typically "text/html". See the mime.types for a list of possible mime types. |
| Status | Set to a HTTP response code. Success is 200. Server error is 500. |
| Location | Set to the URI of a new document to which to redirect the client's browser. |
| ANY | Pass any other header back to the client. |
For example:
Content-type: text/html <HTML><HEAD><TITLE>Sample CGI Output</TITLE></HEAD> <BODY> <H1>Hello World</H1> </BODY></HTML>
To redirect the browser to a new location:
Location: /newUrl.htmlTo signify an error in the server:
Status: 500
Hints and Tips
If you have special data or environment variables that must be passed to your CGI program, you can wrap it with a script that defines that environment before invoking your script.
Other Resources
The following URIs may be helpful in further reading about CGI:
- For an introduction to CGI: http://en.wikipedia.org/wiki/Common_Gateway_Interface
- For the actual CGI specification: http://tools.ietf.org/html/draft-robinson-www-interface-00
- Other CGI resources: http://www.cgi-resources.com