B. Using Q

The following discussion assumes that you have already installed the Q programming system on your machine. In particular, you should make sure that the programs q and qc have been copied to a directory which is searched for executables, and that the standard library scripts have been installed properly.

You may also refer to the UNIX manual page q(1) for a brief description of the Q programs.

The Q programming system consists of two main parts: the compiler and the interpreter. The compiler is used to translate Q scripts into a binary format, a so-called "bytecode" file, which is then executed by the interpreter. Since the interpreter invokes the compiler automatically when you invoke it with a source script, you usually do not have to care about the compiler yourself. But if you wish you can also run compiler and interpreter separately.

B.1 Running Compiler and Interpreter

Here is how you normally run a script with the interpreter:

q [options] [ file | - ] [argument …]

The interpreter will then start up, display its prompt, and wait for you to type some expressions or other commands (see Command Language). You can exit the interpreter by typing quit (which invokes the built-in quit function), or by entering the end-of-file character at the beginning of the input line.

The first non-option argument denotes either the main script or a bytecode file. You can also specify `-' here to indicate that the main script is to be read from standard input. At most one source or code file may be specified; the remaining command line arguments can be accessed in the interpreter by means of the built-in ARGS variable. You can also use the `--' option at any point the argument list to indicate that the interpreter should stop its option processing and pass on the remaining arguments unchanged, which is useful if the Q script takes option parameters itself.

If the given file is in bytecode format, it is loaded immediately. Otherwise, the interpreter invokes the compiler to translate the source script to a temporary code file. Then, if the source script was translated successfully, the interpreter loads the generated code file and you can start typing in expressions and commands. (After the code file has been loaded by the interpreter, it is no longer needed and is discarded immediately. If you want to keep the code file, you must invoke the compiler separately, see below.)

Scripts and code files which have no absolute path specified are searched for on Q's library path (as given by the QPATH shell environment variable or with the --path option, see below). As with the PATH environment variable, the individual directories are separated with a colon `:' (semicolon `;' on DOS/Windows systems). The directories are searched in the indicated order; if the file cannot be found on the search path, an attempt is made to open it by its name. You should include the `.' directory in the search path if you wish to search the current directory prior to other locations. The default search path is usually something like .:/usr/share/q/lib:/usr/lib/q which causes compiler and interpreter to first search the current directory, and then other (system-dependent) locations where "library scripts" are kept.

Note that in difference to shell path searches, the Q interpreter will also search for relative pathnames involving a directory prefix (other than `~', `.' and `..') in subdirectories. Thus, e.g., given the search path .:/usr/share/q/lib:/usr/lib/q, the module foo/bar.q will be searched for under ./foo/bar.q, /usr/share/q/lib/foo/bar.q and finally /usr/lib/q/foo/bar.q.

By default, the interpreter automatically includes the prelude.q script (which will be searched for on the library path as usual) as the first module in your program. The prelude provides the definitions which are available without explicit import or include; normally it is used to include most of the standard library, see Scripts and Modules. You can also suppress the automatic inclusion of the prelude with the --no-prelude compiler option, see below, or provide your own, see Setting up your Environment.

The interpreter can also be invoked with no arguments at all, in which case an empty script is assumed. This means that only the built-in operations of the Q language will be available, as well as the definitions in the prelude.q script (unless the --no-prelude option has been specified). (If you want to specify an empty script while still supplying arguments, specify "" as the script name.)

If you want to avoid recompilations of large scripts, you can also run compiler and interpreter separately. For this purpose, you first invoke the compiler as described below to translate your script(s) to a bytecode file. Then you can invoke the interpreter, specifying the name of the bytecode file (normally q.out, unless another code file name has been specified with the -o compiler option). The interpreter can then load the bytecode file and start execution immediately.

The compiler is executed as follows:

qc [options] [ file | - ] …

It compiles the specified files and writes the resulting bytecode to the code file, normally q.out. Note that multiple source files may be specified. The first file denotes the main script, whose namespace defines the global scope; the name of the main script can also be `-' to indicate that the script is to be read from standard input, or it can be missing or be specified as the empty string "", to indicate an empty main script. The remaining source files are additional scripts to be imported into the global scope.

The compiler recognizes the following options:

--debug, -d

Print a listing of the output code on standard output. The listing includes both the bytecode for the right-hand sides of equations and the pattern-matching automaton generated for the left-hand sides. This option is mainly useful for debugging the compiler itself. You may wish to combine this with the -n option below if you only want to see the listing without actually creating a code file.

--encoding=encoding

Specify the default encoding of script files (Q 7.0 and later). If this option is not used, the system encoding (as specified by the user's locale) is assumed. You only need this option if your script files contain non-ASCII text and the encoding of the files differs from the system encoding. (By including this option in the script files themselves, you can also specify the encoding on a per-file basis, see Running Scripts from the Shell.)

--help, -h

Print a short help message.

--list=list-file, -l list-file

When this option is used, error messages are written to the specified file instead of the standard error device.

--no-code, -n

This option suppresses code file generation, which is useful if you only want to produce a listing or check the given scripts for syntax errors. (Please note that in the current implementation the code file will be created anyway, so that appropriate information can be given with the -d and -v options, but will finally be removed if the -n option is used.)

--no-prelude

Suppresses automatic inclusion of the prelude.

--output=output-file, -o output-file

This option allows you to change the default name of the generated code file.

--path=path, -p path

This option sets the library path used to search for source and code files (default: system-dependent hardcoded default or value of the QPATH shell variable). If the path argument starts or ends with the path delimiter (`:' on UNIX) then the given path is appended or prepended to the current path, respectively.

--prelude=file

Specify the name of the prelude script (default: prelude.q).

--tablesize=size, -t size

This option allows you to determine the size of the runtime hash table used for the hashing of identifiers. For best performance this should be a prime number; the default size is 5711. With this option, you can change the size of the hash table if you are compiling a script with a very large number of function symbols, or a script which dynamically creates a lot of symbols. Inreasing the size of the hash table will reduce the "load" of the table and thereby improve the performance of symbol hashing at runtime.

--verbose, -v

Display processed source files and statistics.

--version, -V

This option causes version number and copyright information to be displayed, after which the program terminates.

--warnings[=level], -w[level], --pedantic, --paranoid

Print warnings about undeclared function (and, optionally, variable) symbols. If just -w or -w1 is specified then function symbols will be reported as undeclared if they are neither imported nor declared anywhere in the file, where all appearances of a function symbol on the left-hand side of equations and variable definitions count as implicit declarations.

To get more warnings, use the -w2 option (or, equivalently, --pedantic), which will warn you about all function symbols which are used without a prior explicit declaration. I.e., when using that option, you really have to declare each and every function symbol explicitly, to make the compiler happy. Moreover, as of Q 7.8 -w2 also warns about non-builtin and non-prelude symbols from unqualified imports which are used without proper qualification. This is useful to report a common source of errors, namely the accidental "reuse" of an imported symbol due to a missing local symbol declaration. (But note that in the current implementation you will never be warned about unqualified operator symbols, as these are treated as special tokens which are handled already during lexical analysis.)

If this still isn't enough, then you can use the -w3 option (or, equivalently, --paranoid), which will also warn you about undeclared free variables and unqualified prelude symbols. Thus you'll also have to declare each and every free variable, and use qualified symbols or qualified import even for prelude symbols, if you want to avoid all warnings from the compiler. Please note that this option will produce an excessive amount of warnings even for most perfectly legal scripts. But it may occasionally be useful to check your script for missing declarations or mistyped identifiers. If you find yourself using this option regularly, then Q probably is not for you. ;-)

NOTE: As of Q 7.8, the default warning level can also be set with the QWARN environment variable. A note for library writers: If you publish libraries of Q modules for public consumption, they should pass at least -w2 without any warnings, since many Q programmers will want to use that option at least occasionally.

Options are generally parsed using getopt which means that multiple single-letter options may be combined (e.g., -vn) and a parameterized option can be followed immediately by the corresponding argument (e.g., -ofoo.out); see getopt in Section 3 of the UNIX manual for details. All programs also accept long option names following the GNU conventions, e.g., you can use --output instead of -o. Note that in the case of parameterized long options the parameter must either follow in the next command line argument, or it must be separated from the option name by a = character. Long options may be abbreviated by an unambigious prefix of the option name. For instance, you may use --out instead of --output. Option parsing ends as soon as the -- option is found; the remaining command line arguments are taken as normal parameters, even if they start with `-'.

After a script has been compiled, you can invoke the interpreter on the generated code file as already discussed above (in this case, the code file is not discarded). Below we describe the options which are accepted by the interpreter. Note that the interpreter also recognizes all of the compilation options discussed above, and will pass them on when it invokes the compiler.

--break

This option causes invocation of the debugger in case of an exception like Ctl-C, invalid rule qualifier, user-raised exception (throw), or a call to the built-in break function. See Debugging, for details.

--command=cmd, -c cmd

This option instructs the interpreter to execute the given command in "batch mode" (see below).

--cstacksize=size

Specify the maximum amount of C stack in KB (default: 256) that can be used for the recursive evaluation of special form arguments; see the comments on memory management below.

--debug, -d

This option causes activation of the debugger. See Debugging, for details.

--debug-options

Set various options which control how much detail the debugger prints when showing a reduction or rule. See Debugging, for details.

--dec, --hex, --oct

Sets the integer output format to decimal (default), hexadecimal or octal, respectively.

--echo, -e

Enable the echoing of commands when running in batch mode.

--exitrc=file

This option allows you to specify the name of the file which is executed when the interpreter is exited; the default is usually ~/.qexitrc.

--fix[=prec], --sci[=prec], --std[=prec]

Sets the floating point output format to fixed point, scientific (always using exponents) or standard (only using exponents if it results in a shorter notation; this is the default). For the scientific and standard formats, the given precision denotes the number of significant digits to be printed, and defaults to 15. For the fixed point format, the precision specifies the number of digits following the decimal point (2 by default).

--gnuclient

Lets the interpreter run as a client of gnuserv(1). This allows the interpreter to interact with an emacs process driving the interpreter, e.g., in "comint" mode. In particular, the edit and help commands of the interpreter (see Command Language) will then be handled by sending corresponding Emacs Lisp commands via the gnuclient command instead of processing them directly. The actual name of the gnuclient program can be set using the GNUCLIENT_PROGRAM environment variable. Note that this option also disables the interpreter's own history processing as it is assumed that emacs takes care of it.

--help, -h

Print a short help message.

--histfile=file

Specify the name of the file in which the command history is stored (default: ~/.q_history).

--histsize=size

Specify the size of the command history (default: 500).

--initrc=file

Specify the name of the file which is executed at startup (default: ~/.qinitrc).

--interactive, -i

Indicates that the interpreter is running interactively.

--memsize=size

Specify the maximum size of the expression heap (default: 4096000; see comments on memory management below).

--no-editing

Disable command line editing using readline.

--no-exitrc

Suppress execution of the exitrc file.

--no-initrc

Suppress execution of the initrc file.

--path=path, -p path

As with qc, sets the library path used to search for source and code files.

--prompt=str

Set the prompt string (default: `\n==> ').

--quiet, -q

This option causes a quiet startup; it suppresses the sign-on message.

--source=file, -s file

Source file with interpreter commands (batch mode).

--stacksize=size

Specify the maximum size of the internal stacks (default: 1024000; see comments on memory management below).

--version, -V

As with qc, display version number and copyright information.

Options for Interactive and Batch Usage

Unless one of the -c and -s options is specified, the interpreter starts up in interactive mode, in which the user is repeatedly prompted to enter an expression, and the interpreter answers with the corresponding normal form. The interpreter also understands some other special commands, which are discussed in Command Language. With the -c and -s options, the interpreter runs in batch mode, in which it executes the commands and command files specified on the command line, prints the results on the standard output device, and then exits immediately. The -d option invokes the symbolic debugger built into the interpreter which allows you to trace evaluations, cf. Debugging.

When the interpreter starts up in interactive mode, and is connected to a terminal, it also provides a command history through the GNU readline library. (See Readline: (readline)Command Line Editing section `Command Line Editing' in The GNU Readline Library.) That is, the lines you type are memorized, and you can search for and edit previous commands. The command history is also saved to a file when the interpreter is exited, and completion of filenames, keywords, command names and defined function and variable symbols is supported as well. You can disable this feature with the --no-editing option.

You can redirect the interpreter's input and output streams to a file or a pipe as usual. The Q interpreter checks whether it is connected to a terminal in order to prevent garbled output when input or output is redirected. If you're feeling adventurous, try something like the following:

q -d <<EOF
def Xs = [1..50]
msort (>) Xs
EOF

The -i option can be used to force the interpreter to interactive mode even when input or output is redirected. This causes sign-on and prompt to be printed, and the standard output buffers to be flushed after each evaluation, which is useful when another program is driving the interpreter. (If you still want to suppress the sign-on message, use -i in combination with -q.) Furthermore, the -i option also suppresses the batch mode execution of -c and -s options.

The --initrc and --exitrc options allow you to specify command files which are executed when the interpreter starts and exits, which is commonly used to customize the environment when running the interpreter interactively, see Setting up your Environment. The execution of these files will be suppressed when the interpreter is run in batch mode, or when using the --no-initrc and --no-exitrc options.

Memory Management

The Q interpreter has a fully dynamic memory management system. In particular, it automatically resizes both the expression heap and the evaluation stack as required during the course of a computation. Thus you normally do not have to worry about memory management issues.

However, the --stacksize option allows you to specify a maximum size for the evaluation stack (more precisely, it sets the limit for both the expression and the rule stack), which keeps the stack from growing to infinity. This is useful, e.g., to catch infinite recursion. Note that as the stack is also used internally for the purpose of parsing expressions, a reasonable minimum size is required for the interpreter to function properly. Therefore the interpreter enforces that the given size is not less than a certain minimum value (usually 100); otherwise the option is ignored and the stack size is reset to its default value.

Similarly, the --memsize option limits the number of expression nodes on the heap and thereby prevents the interpreter from using up all available main memory on your system. You can disable both limits by specifying 0 as the argument to the corresponding option.

In the current implementation, the interpreter also needs a variable amount of C stack space for evaluating "deferred" values like special form arguments. This might cause the interpreter to exit unexpectedly due to C stack overflows. To prevent this, as of Q 7.7 the interpreter poses a limit on the amount of stack space used during evaluation of deferred values and will gracefully terminate the computation with a stack overflow error when the allocated C stack space is exhausted. The default stack size is 256 kilobytes; you can change this with the --cstacksize option. Setting this option to zero disables the C stack checks. (Please note that this option does not change the interpreter program's C stack size, it only sets the limit used in the stack test; your operating system should supply you with a command, such as ulimit on UNIX-like systems, to actually set the program stack size.)

The interpreter manages expression memory using a reference counting scheme. This has the considerable advantage that no disruptive "garbage collection" process has to be invoked during an evaluation, and that expression objects are reclaimed and other automatic cleanup (closing files, etc.) is performed on them as soon as they become inaccessible. The downside of this scheme is that cyclic expression structures (which can be created, e.g., using expression references, cf. Expression References) cannot be collected and thus should be avoided.

B.2 Command Language

The commands understood by the interpreter make up a simple command language which is described in the following. Commands can also be passed to the interpreter by means of the -c option, and you can put a sequence of commands into a file and execute them using the -s option (see Running Compiler and Interpreter). Command files can also be executed in the interpreter using the source command (see below). Moreover, when the interpreter starts up and is exited in interactive mode, it reads and executes commands from its initialization and termination files, which allows you to customize your environment according to your taste (see Setting up your Environment).

The command language is line-oriented, i.e., commands are terminated by the newline character. As usual, line ends can be escaped by putting the \ character immediately before the end of the line; this also works inside of string literals. A line may contain multiple expressions and commands, which are separated by the `;' delimiter:

==> def X = foo Y; X; X/2

(You may also terminate a command line with an extra ; delimiter, but this is not required.)

Here are the most common commands which allow you to evaluate expressions and bind global variables:

expression

Evaluate the given expression and print the corresponding result.

def expression = expression, …

Define free variable values.

undef variable, …

Undefine the given variables.

var variable [ = expression ], …

Declare the given variables and optionally initialize their values.

The syntax of expressions and variable declarations and definitions is the same as that of the corresponding constructs in scripts. After evaluating an expression, the interpreter usually prints the corresponding normal form as the result. (Note, however, that some scripts may also provide special "views" to implement custom pretty-printing of certain types of objects, see Views.) No response is printed for variable declarations and definitions, unless an error occurs.

As of Q 7.7, the interpreter allows you to declare variables interactively with the var command, which works pretty much like var declarations in scripts (see Declarations). The declared variables are created in the global scope (i.e., the scope of the main script). In particular, this command allows you to declare uncapitalized identifiers (which would otherwise be interpreted as function symbols) as variables.

After a variable has been assigned a value, it will be replaced by its associated value whenever it is evaluated. This allows you to store intermediate results in a computation, set the values of free variables occuring in a script, and to define new functions interactively in the interpreter. For instance:

==> var double = (*) 2, X = double 4; X
8

A subsequent undef removes a variable definition:

==> undef X; X
X

As explained in Free Variables, the left-hand side of a variable definition may actually be an arbitrary pattern to be matched against the right-hand side value:

==> def [X|Xs] = [a,b,c]

When starting up, the interpreter always defines the variables INPUT, OUTPUT, ERROR and ARGS. INPUT and OUTPUT are set to the standard input/output devices used by the terminal I/O functions (see Terminal I/O), while ERROR denotes the standard error device normally used to display error messages. You can use these symbols when passing a file to one of the built-in file I/O functions (see File I/O). As already mentioned, the ARGS variable is assigned a string list which contains the additional command line arguments the Q interpreter was invoked with (starting with the script name). These variable symbols are "read-only"; they cannot be re- or undefined.

There is one other special built-in variable, the "anonymous" variable `_', which, when used in an expression or the right-hand side of a variable definition, refers to the result of the most recent expression evaluation:

==> 1/3
0.333333333333333

==> 1/_
3.0

The interpreter also provides some other special commands which extend the expression and definition syntax of the Q language. They usually consist of a command name and possibly some parameters. Unlike the def, undef and var keywords, which are also reserved words in the Q language itself, the other command names are only treated specially when occuring at the beginning of a command; anywhere else they are parsed as ordinary Q identifiers.

The parameters of special commands are simply string literals which may be single-quoted, double-quoted or unquoted, and which are delimited with whitespace, very much like in the UNIX command shell. The interpreter will not try to evaluate these parameters. However, the usual escapes for special characters are supported. The following table lists the available commands, where […] is used to denote optional arguments.

! command

Shell escape. The given command is executed using the shell. Unlike other types of commands, the shell escape must be on a line by itself.

? expression

Escape an expression. This just evaluates expression and prints the result as usual, but inhibits the interpretation of special command names at the beginning of the expression. E.g.,

==> ? echo "some string"

will evaluate an expression involving some echo function instead of executing the echo command.

copying

Show the GNU license conditions.

help [S]

Start the GNU info reader (as specified by the INFO_PROGRAM environment variable, info by default) with the Q manual; if a keyword S is given, perform an index search. When the interpreter has been started with the --gnuclient option (see Running Compiler and Interpreter), then the command submits a request to invoke the info reader of a driving emacs process using the gnuclient program.

path [S]

Set the library search path (QPATH) to S (print the current setting if S is omitted). If the S argument starts or ends with the path delimiter (`:' on UNIX) then the given path is appended or prepended to the current path, respectively.

prompt [S [S2 [S3]]]

Set the prompt string (print the current setting if S is omitted). The \\v, \\s, \\w, \\W, \\m and \\M escape sequences may be used in the prompt string to denote the Q version number, host system information, current working directory, basename of the current directory, full pathname of the current script, and the script's basename, respectively. (Note the double escapes; these are required to make one \ get through to the command.) The optional second and third argument allow you to change the default continuation (`> ') and debugger prompt (`: '); no special escape sequences are substituted in these strings.

dec, hex, oct

Sets the integer output format to decimal, hexadecimal and octal, respectively (see also the corresponding interpreter command line options in Running Compiler and Interpreter). Note that these commands only affect the display of expressions which are printed in response to interpreter and debugger commands, otherwise integer values will always be unparsed in decimal notation.

fix [prec], sci [prec], std [prec]

Sets the floating point output format to fixed point, scientific and standard, respectively (see also the corresponding interpreter command line options in Running Compiler and Interpreter). If the precision is omitted, it defaults to 2 for the fixed point format, and to 15 otherwise. Note that these commands only affect the display of expressions which are printed in response to interpreter and debugger commands, otherwise floating point values will always be unparsed with the full internal precision.

histfile [S], histsize [N]

Change the history filename and set the size of the history (print the current setting if no argument).

cstacksize [N]

Set the maximum C stack size in kilobytes (print the current setting if no argument); see the discussion of the --cstacksize option in Running Compiler and Interpreter.

stacksize [N]

Set the maximum stack size (print the current setting if no argument); see the discussion of the --stacksize option in Running Compiler and Interpreter.

memsize [N]

Set the maximum number of expression nodes on the heap (print the current setting if no argument); see the discussion of the --memsize option in Running Compiler and Interpreter.

stats [all]

Print statistics about the most recent expression evaluation in the main thread. If the optional all parameter is given then the command also lists finished background threads which have not been recycled yet. The given figures are the cpu time needed to complete the evaluation, the total number of reductions performed by the interpreter, and the number of expression nodes (heap cells) created during the course of the computation. The application of each built-in rule counts as a single reduction. Note that since different built-in rules may have different execution times (in fact, the time needed by many of the built-in operations varies with the size of the input), the number of reductions usually only provides a rough estimate of the actual running time. But this value is still quite useful to perform asymptotic analysis (up to linear factors, if one wants to be picky), and can also be used to compare profiling results obtained on different machines with different processors. The cell count is the maximum number of cells required at any time during the evaluation, which does not include the size of the input expression. This value is used to measure space requirements, and is proportional to the "real" memory requirements of the interpreter (each expression node requires some 24 bytes in the current implementation). But note that subexpressions may be "shared" between different expressions, and thus the cell count will usually be less than the tree size of the constructed expressions.

debug [on|off|options]

When invoked with arguments on or off, activate or deactivate debugging; print the current setting if no argument is specified. Alternatively, you can also use this command to set various options which control how much detail the debugger prints when showing a reduction or rule. See Debugging, for details.

break [on|off|function …]

When invoked with arguments on or off, this command sets the interpreter's break flag which controls how the interpreter responds to an exception like Ctl-C, invalid rule qualifier, user-raised exception (throw), or a call to the built-in break function. When invoked with a list of function symbols, it sets breakpoints on the given symbols, and also enables the break flag if necessary. If no argument is given, the current setting of the break flag is printed, along with a list of the current breakpoints. See Debugging, for details.

tbreak function …

This commands works like break when invoked with a list of function symbols, but sets temporary breakpoints which are removed when hit.

profile [function …]

Sets a flag on the given function symbols, so that reductions are counted whenever an application of the function is reduced. The rules for counting reductions are the same as for the stats command, i.e., each invocation of a built-in or external function counts as a single reduction. When invoked without arguments, the command prints a list of all flagged function symbols along with the corresponding reduction numbers (sorted by decreasing counts) and resets the reduction counters. This command is useful for profiling purposes, e.g., if you want to find "bottlenecks" among the functions of your script, whose optimization will most likely lead to noticeable improvements of running times.

clear [variable …]

Undefine the given variables. If no argument is specified, clear all user-defined variables (including the value of the `_' variable).

clear break [function …]

Remove breakpoints on the given function symbols (all breakpoints if no argument is specified).

clear profile [function …]

Disable profiling on the given function symbols (all functions if no argument is specified).

unparse [on|off]

Enable or disable custom pretty-printing of expressions (print current setting if no option is given), as defined by rules for the built-in view function, cf. Views. This is enabled by default.

echo [on|off], echo S

Enable or disable command echoing (print current setting if no option is given), or echo the given string to standard output (with a newline character appended). When echoing is enabled, all command lines executed in batch mode (using the -s or -c option, or the source command) are echoed to standard output before they are executed. (You can inhibit command echoing by prefixing a command line with the @ character.)

chdir [S], cd [S]

Change to the given directory (your home directory if none given).

pwd

Print the current working directory.

ls [S ...]

List the contents of the current directory, or the files matching the given patterns. This command simply invokes the corresponding UNIX command with the given arguments.

which [S]

Print the full name of the script or command file (as given by a search on the library path) that would be run when the relative pathname S is given to the run or source command; if invoked without arguments, print the full pathname of the running script.

edit [S]

Edit the given file (current main script if none specified), using the editor specified by the EDITOR environment variable, or vi by default. When the interpreter has been started with the --gnuclient option (see Running Compiler and Interpreter), then the command causes an edit request to be submitted to a driving emacs process using the gnuclient program.

run [S [args ...]]

Run the given script with the given arguments. If no script is specified, rerun the current script with the current arguments. (Note that to force the interpreter back to an empty script, you can specify "" for the script name. You can also use `-' as the script name to indicate that the script is to be read from standard input.) When in interactive mode, the exitrc and initrc files are sourced as usual. Note that in the current implementation this operation also clears the variable memory, so you might wish to save your variables before invoking this command, unless your exitrc file already takes care of this.

import [[+|-]S …]

Rerun the current script with the current arguments, as with run, but also add or remove the given scripts to/from the current "import list". A `+' prefix (or no prefix at all) causes the module to be added to the import list and `-' removes the module, if possible. If no arguments are given then the import list is cleared. Modules on the import list are imported into the global scope (i.e., the namespace of the main script), in addition to the prelude and the imports of the main script. The current import list can be shown with the imports command, see below.

Note that you can also add modules which are already included in the running script, but which are not currently visible in the global scope. On the other hand, you can only remove those modules which have actually been added to the import list, not any modules which are only included in the running script. The main script can be removed as well, which has the same effect as doing run with an empty script name.

IMPORTANT: This command will only work as advertised when running a source script. If a bytecode file is run, the import list will be cleared, and subsequent import commands will have no effect.

source S, . S

Source a file with interpreter commands (may be nested). As indicated, this command may be abbreviated as `.'. The given command file is searched on the Q library path.

save [S], load [S]

Save and restore variable values to/from the given file (if no file specified, use the file last specified with save or load, or .q_vars by default).

The save command stores all user-defined variables in the form of variable definitions, i.e., var commands. Floating point values are always stored with maximum internal precision so that they can be reconstructed exactly, at least on machines with IEEE compatible 64 bit doubles. The built-in variables are not saved with this command; thus, if you want to also save the value of the `_' variable, you must explicitly assign it to a user variable.

The load command is very much like source, so the file may actually contain arbitrary interpreter commands, but no library path search is performed, so a path must be given unless the file is contained in the current directory.

Putting a pair of load and save commands in your initrc/exitrc files (see Setting up your Environment) is a convenient method to have the interpreter automatically remember the variable environment from a previous session. Note, however, that the load command reevaluates all values in the context of the current script, and hence the results may be quite different from the original values if the script has been changed during invocations of the interpreter. Another potential obstacle is that the load command cannot reconstruct "external" objects (<<typeid>>, see File I/O, and C Language Interface), since these objects do not have a parseable representation at all.

modules

List all loaded modules, i.e., all scripts imported or included in the running script (including the prelude). External modules are indicated by a trailing `*'.

imports

Like the modules command, but only list the modules visible in the global scope. This includes the main script itself, the prelude and all the modules it includes (normally the standard library), the imports and includes of the main script, and the extra imports specified with the import command (see above). The latter modules and the main script are indicated by an initial `+' character. (These are the modules which can be removed with the import command.)

who [all]

List all user-defined variable symbols. This only lists variables which have been defined in the interpreter. If the optional all parameter is specified then all variables are listed.

whos symbol …

Describe the given (function, variable or operator) symbols. Prints useful information about each symbol, such as whether it is a function symbol or a variable, name of the script file and line number where the symbol is first declared, defined or used, const and special attributes, etc.

whois symbol …

As of Q 7.8, this is a new and improved version of the whos command which also lists, for each given symbol, the fully qualified names under which the symbol is available in the global namespace.

completion_matches S

List the possible completions of a token prefix. This command allows programs like emacs (see Running Scripts in Emacs) to implement completion when driving the interpreter as an inferior process.

B.3 Setting up your Environment

There are basically two ways in which you can tailor the Q interpreter to your needs: you can provide your own prelude providing "preloaded" function and variable definitions; and you can use the qinitrc and qexitrc command files to execute a sequence of interpreter commands when an interactive instance of the interpreter starts up and exits. We describe each of these in turn.

The prelude, which has already been discussed in Scripts and Modules, is just an ordinary Q script which is by default imported in all your scripts. Normally the prelude includes the standard library modules, but you can also provide your own prelude into which you can put any "standard" definitions you want to have available in all your scripts. To customize the prelude, you can either modify the version provided with the standard library (to provide your definitions system-wide), or put your private version somewhere on your library path where the compiler will find it prior to the standard one. In the latter case you can still have your version of the prelude include the standard prelude by renaming the latter using an `as' clause (see Scripts and Modules). E.g., your prelude.q script may contain something like the following (assuming the standard prelude is in /usr/share/q/lib):

// include the standard prelude as `stdprelude'
include "/usr/share/q/lib/prelude.q" as stdprelude;
// my own definitions here ...

Let us now turn to the interpreter's initialization and termination files. As already mentioned, when the interpreter runs in interactive mode, it automatically sources a startup file (usually named ~/.qinitrc, but you can change this with the --initrc option) before entering the command loop. Thus you can put some commands in this file which set up your initial environment, e.g., initialize some variables and load variable definitions from a previous session. A typical startup file may look as follows (lines with initial `//' are comments):

// sample init file

// common constants
//var e = exp 1.0
//var pi = 4.0*atan 1.0

// set your preferred defaults here
//path .:~/q:/usr/share/q/lib:/usr/lib/q
//fix 2
//cstacksize 1024
//stacksize 1024000
//memsize 4096000
//break on
//debug pathnames=y detail=all maxchars=80
prompt "\n\\M>> "

// read variable definitions from .q_vars file
load

Similarly, when the interpreter is exited, it normally sources the ~/.qexitrc file (or the file named with the --exitrc option), which might take care of saving the values of currently defined variables:

// sample exit file

// autosave variables
save

// want your daily epigram?
//! fortune

B.4 Running Scripts from the Shell

The Q programming system has been designed primarily to facilitate interactive usage. However, with a little additional effort it can also be used to create standalone application programs which are invoked directly from the shell. We discuss how to do this in some detail below.

Using "Shebangs"

For the purposes of this section, your script should be equipped with some "main" function which acts as the entry point to your application. On UNIX systems, you can then use the `#!' ("shebang") notation to specify the Q interpreter as a language processor to be invoked on the script instead of the shell:

#!/usr/bin/q -cmain

If you include this line (which will be treated as a comment by the Q interpreter) at the top of your main script and give the script execute permissions, most UNIX shells will be able to execute the script just like any other program.

You can pass the script's command line parameters as an argument to the main function using a shebang line like the following:

#!/usr/bin/q -cmain ARGS

In any case, your main function should normally use the exit function (cf. Process Control) to explicitly terminate the script with an exit code (unless you specifically want the interpreter to print the result of the main function on standard output). Alternatively, you may include the call to exit in the shebang line:

#!/usr/bin/q -cmain||exit 0

Or your main function can return an integer exit code, in which case you would call it as follows:

#!/usr/bin/q -cexit main

Note that the "shebang" feature is generally only available on UNIX-like systems. On Windows, you'll have to work with "batch file" wrappers for your script, or you can compile your script to a standalone executable with the qcwrap program as described in Translating Scripts to C.

A Simple Example

For the sake of a concrete example, let us write a little program which takes a number as a command line parameter and prints out all the Fibonacci numbers up to the given index on standard output:

#!/usr/bin/q -c main (val (ARGS!1))

fib N           = A where (A,B) = fibs N;
fibs N          = (B,A+B) where (A,B) = fibs (N-1) if N>0;
                = (0,1) otherwise;

main N:Int      = do (printf "%s\n".str) $ map fib [0..N] || exit 0;
main _          = printf "Usage: %s <number>\n" (ARGS!0) || exit 1
                    otherwise;

Note that in this case we simply passed the Q value of the first command line parameter into the main function and have that function do all the remaining processing. We also handle the case that the parameter is missing or does not denote an integer value, and print a little usage message in this case. (At least on UNIX-like systems it is customary to print the real program name in such diagnostics, which is readily available as ARGS!0, see the second line of main's definition.) Also note that we print the numbers in the list map fib [0..N] using printf "%s\n".str rather than simply printf "%d\n", since the numbers get pretty large pretty soon, so the `%d' format (which only supports 32 bit integers on most systems) would not be able to handle them.

Save the above script as a file, say, fibs.q, and make it executable. That is all that is needed, the script should now be ready to go. Run it as, e.g., ./fibs.q 100 to see it in action.

Command Line Parameters

Most shells only accept a single argument with `#!'. This means that you must pass all compiler and interpreter options in one argument. For instance:

#!/usr/bin/q -dcmain ARGS

Obviously, this can be cumbersome and it also prevents you from having more than one parameterized option. As a remedy, the Q compiler and interpreter let you pass options by including any number of additional "shebang" lines in the format

#! option

at the beginning of the main script. This even works if there is no #!/usr/bin/q line at the top of the script, but in this case it is a good idea to leave the first script line empty, so that the option line is not mistaken for an ordinary shell shebang line. Note that this is not a shell feature, but an extension provided by the Q programming tools, and hence also works on Windows. Also note that here the `#!' symbol and the option must be separated with whitespace, like so:

#!/usr/bin/q
#! -ofoo
#! -cmain ARGS

Using these facilities, fairly elaborate application setups can be handled which require setting the environment for a certain installation directory with additional modules and data files used by the main script. For instance, you can modify the Q path and set a PROGDIR variable to the installation directory as follows:

#!/usr/bin/q
#! -p/usr/share/myprog:
#! -cdef PROGDIR="/usr/share/myprog"
#! -cmain ARGS

By these means, all the necessary setup information is collected at the beginning of your main script where it can be changed easily at installation time.

Locating Data Files

Another common trick is that you can also have the built-in which function (see Miscellaneous Functions) search the application's data files for you. This only requires that you have set up a search path pointing to the application directory containing your data files with `-p' as described above (which you'll probably have to do anyway, so that the script can find its auxiliary imports). For instance:

myconfig = which "myconfig.rc";

Because of the way the interpreter does its path searches (see Running Compiler and Interpreter), this even works if the file is located in some subdirectory of the application directory:

myconfig = which "etc/myconfig.rc";

Here is a reasonable "standard" setup that should work for most applications: Install the entire code for your application, including any needed data files and auxiliary modules, in the directory /usr/share/q/apps/myprog (assuming that the interpreter lives under /usr), where myprog denotes the name of your application. Locate data files in your program using which as described above (you might want to check the return value of which and abort the program with a suitable error message if a data file cannot be found). Using this setup, a shebang line like the following should do the trick (add other command line options as needed):

#!/usr/bin/q
#! -p/usr/share/q/apps/myprog:
#! -cmain ARGS

During installation, make sure to edit the /usr prefix in the second line with a program like sed to reflect the actual installation prefix. Finally, make your main script in /usr/share/q/apps/myprog executable and create a symbolic link to it in /usr/bin (or any other directory on the system PATH). Your script should now be executable from the command line no matter where you actually installed it.

Using Env

The shell shebangs we have been using so far all required that the absolute path of the interpreter executable be given. However, many UNIX systems also offer the env command which executes a given command and also performs a path search. This enables you to write something like:

#!/usr/bin/env q

Using this method, you do not have to know where the interpreter executable is located; env will search for it on the system PATH as usual. Passing additional command line parameters can then be done via secondary shebangs as described above.

Embedding Q Programs in Shell Scripts

As of Q 7.8, the interpreter allows you to specify `-' as the name of the main script, to indicate that the script is to be read from standard input. This allows you to employ shell "here" documents in order to embed a Q program in an ordinary shell script. This has the disadvantage that first another shell has to be invoked to execute the embedded program, but in return you get the possibility to do much more elaborate preprocessing of program parameters than a simple shebang can offer. Moreover, it is also possible to include several different Q programs in a single shell script and execute them in sequence or even in a loop, which is great for testing purposes or any other kind of "automation".

One word of caution: When you specify a Q script as a "here" document, shebangs in the included script are not processed; they are simply treated as comments. But this is not a problem at all since of course our shell script contains an explicit command invoking the interpreter, so we can easily specify any interpreter options that we might need. For example, here is another version of the Fibonacci program from above, this time written as a shell script.

#!/bin/sh

# Note that we pass the real script name as ARGS!1 here (and the number
# parameter as ARGS!2), so that we can give proper diagnostics.

q - "$0" "$1" -c 'main (val (ARGS!2))' <<EOF

fib N           = A where (A,B) = fibs N;
fibs N          = (B,A+B) where (A,B) = fibs (N-1) if N>0;
                = (0,1) otherwise;

main N:Int      = do (printf "%s\n".str) $ map fib [0..N] || exit 0;
main _          = printf "Usage: %s <number>\n" (ARGS!1) || exit 1
                    otherwise;

EOF

Encoding Options

As of Q 7.0, there is one additional compiler option, --encoding, which, in difference to all other options, is also interpreted while the compiler parses each script file belonging to a program (not just the main script). This option can be used to tell the compiler about the encoding of scripts on a per-file basis, employing a "shebang" line like the following:

#! --encoding=UTF-8

This is useful if a script file contains non-ASCII text and may be deployed in environments where you cannot be sure that the encoding of the file matches the system encoding.

B.5 Translating Scripts to C

As of version 5.1, the Q programming system also includes a little utility, qcwrap, which employs Q's interpreter interface library libqint (see Embedding Q in C/C++ Applications) to turn a Q script into a C program which can be compiled to a binary executable. This will be a real executable for the host operating system, i.e., no `#!' magic is required, and you don't have to distribute the Q source with the executable program, as the binary also contains the entire bytecode of the script.

Similar to a self-hosting compiler, the qcwrap program is implemented by a Q script which is run on itself to turn it into an executable. Note, however, that qcwrap does not really "translate" the source script to C code, i.e., it is not a "real" Q-to-C compiler. It just creates a C wrapper for running the Q interpreter on the script's bytecode. The compiled script is still executed by the Q interpreter (in its libqint incarnation), so it will not run any faster or use less memory than the source script. Nevertheless, qcwrap is quite useful whenever a Q application has to be shipped in a self-contained binary form.

The qcwrap program is simply invoked with the filename of the script file to be converted:

$ qcwrap myscript.q

Be patient, generating the C source for the bytecode of the script takes a while … If all went well, you should now have a myscript.c file. Compile the C program as follows (taking gcc as an example):

$ gcc -o myscript myscript.c -lqint

Just as with scripts executed via shebangs, the converted script must "do" something when invoked from the command line. The usual way to accomplish this is to include a shebang in your script, as discussed under Running Scripts from the Shell, above. The qcwrap utility interprets the -c options in the shebang lines at the beginning of the script and generates corresponding C code for each of them. (Note that only the -c options are interpreted. Moreover, only valid Q expressions can be executed this way, other special interpreter commands will not work.)

The compiled script still depends on the Q runtime and any other non-Q source files it uses, but it is not a great deal to create a minimal runtime including libqint and the required module binaries (i.e., shared libraries), which can be distributed with the compiled script and other support files needed by the script itself. (You could even do without the module binaries when linking everything statically; otherwise you'll just have to make sure that libqint and the module binaries are installed in a place where the system finds them.) E.g., if your script uses the standard library and nothing else, a package including your script binary and the libqint and clib binaries should work, even without any Q installation on the target system.

This document was generated by Albert Gräf on February, 23 2008 using texi2html 1.76.