NameDateSize

..27-Jun-202090

autogen.sh23-Mar-201424

ChangeLog.md19-Feb-20171.9 KiB

CMakeLists.txt01-Oct-20168.8 KiB

configure.ac19-Feb-20175.6 KiB

COPYING29-Nov-20141.3 KiB

doc/28-Oct-20157

examples/28-Oct-20153

haskell/01-Oct-20163

include/19-Feb-20175

klib/28-Oct-20154

libucl.pc.in29-Nov-2014250

lua/19-Feb-20176

m4/01-Oct-20164

Makefile.am19-Apr-20162.8 KiB

Makefile.unix01-Oct-20164.1 KiB

Makefile.w3201-Oct-20164.1 KiB

python/01-Oct-20165

README.md19-Feb-201712.4 KiB

src/19-Feb-201717

stamp-h.in23-Mar-201410

tests/19-Apr-201620

uthash/05-Jun-20165

utils/19-Apr-20166

README.md

1# LIBUCL
2
3[![Build Status](https://travis-ci.org/vstakhov/libucl.svg?branch=master)](https://travis-ci.org/vstakhov/libucl)
4[![Coverity](https://scan.coverity.com/projects/4138/badge.svg)](https://scan.coverity.com/projects/4138)
5[![Coverage Status](https://coveralls.io/repos/github/vstakhov/libucl/badge.svg?branch=master)](https://coveralls.io/github/vstakhov/libucl?branch=master)
6
7**Table of Contents**  *generated with [DocToc](http://doctoc.herokuapp.com/)*
8
9- [Introduction](#introduction)
10- [Basic structure](#basic-structure)
11- [Improvements to the json notation](#improvements-to-the-json-notation)
12	- [General syntax sugar](#general-syntax-sugar)
13	- [Automatic arrays creation](#automatic-arrays-creation)
14	- [Named keys hierarchy](#named-keys-hierarchy)
15	- [Convenient numbers and booleans](#convenient-numbers-and-booleans)
16- [General improvements](#general-improvements)
17	- [Comments](#comments)
18	- [Macros support](#macros-support)
19	- [Variables support](#variables-support)
20	- [Multiline strings](#multiline-strings)
21- [Emitter](#emitter)
22- [Validation](#validation)
23- [Performance](#performance)
24- [Conclusion](#conclusion)
25
26## Introduction
27
28This document describes the main features and principles of the configuration
29language called `UCL` - universal configuration language.
30
31If you are looking for the libucl API documentation you can find it at [this page](doc/api.md).
32
33## Basic structure
34
35UCL is heavily infused by `nginx` configuration as the example of a convenient configuration
36system. However, UCL is fully compatible with `JSON` format and is able to parse json files.
37For example, you can write the same configuration in the following ways:
38
39* in nginx like:
40
41```nginx
42param = value;
43section {
44    param = value;
45    param1 = value1;
46    flag = true;
47    number = 10k;
48    time = 0.2s;
49    string = "something";
50    subsection {
51        host = {
52            host = "hostname";
53            port = 900;
54        }
55        host = {
56            host = "hostname";
57            port = 901;
58        }
59    }
60}
61```
62
63* or in JSON:
64
65```json
66{
67    "param": "value",
68    "param1": "value1",
69    "flag": true,
70    "subsection": {
71        "host": [
72        {
73            "host": "hostname",
74            "port": 900
75        },
76        {
77            "host": "hostname",
78            "port": 901
79        }
80        ]
81    }
82}
83```
84
85## Improvements to the json notation.
86
87There are various things that make ucl configuration more convenient for editing than strict json:
88
89### General syntax sugar
90
91* Braces are not necessary to enclose a top object: it is automatically treated as an object:
92
93```json
94"key": "value"
95```
96is equal to:
97```json
98{"key": "value"}
99```
100
101* There is no requirement of quotes for strings and keys, moreover, `:` may be replaced `=` or even be skipped for objects:
102
103```nginx
104key = value;
105section {
106    key = value;
107}
108```
109is equal to:
110```json
111{
112    "key": "value",
113    "section": {
114        "key": "value"
115    }
116}
117```
118
119* No commas mess: you can safely place a comma or semicolon for the last element in an array or an object:
120
121```json
122{
123    "key1": "value",
124    "key2": "value",
125}
126```
127### Automatic arrays creation
128
129* Non-unique keys in an object are allowed and are automatically converted to the arrays internally:
130
131```json
132{
133    "key": "value1",
134    "key": "value2"
135}
136```
137is converted to:
138```json
139{
140    "key": ["value1", "value2"]
141}
142```
143
144### Named keys hierarchy
145
146UCL accepts named keys and organize them into objects hierarchy internally. Here is an example of this process:
147```nginx
148section "blah" {
149	key = value;
150}
151section foo {
152	key = value;
153}
154```
155
156is converted to the following object:
157
158```nginx
159section {
160	blah {
161		key = value;
162	}
163	foo {
164		key = value;
165	}
166}
167```
168
169Plain definitions may be more complex and contain more than a single level of nested objects:
170
171```nginx
172section "blah" "foo" {
173	key = value;
174}
175```
176
177is presented as:
178
179```nginx
180section {
181	blah {
182		foo {
183			key = value;
184		}
185	}
186}
187```
188
189### Convenient numbers and booleans
190
191* Numbers can have suffixes to specify standard multipliers:
192    + `[kKmMgG]` - standard 10 base multipliers (so `1k` is translated to 1000)
193    + `[kKmMgG]b` - 2 power multipliers (so `1kb` is translated to 1024)
194    + `[s|min|d|w|y]` - time multipliers, all time values are translated to float number of seconds, for example `10min` is translated to 600.0 and `10ms` is translated to 0.01
195* Hexadecimal integers can be used by `0x` prefix, for example `key = 0xff`. However, floating point values can use decimal base only.
196* Booleans can be specified as `true` or `yes` or `on` and `false` or `no` or `off`.
197* It is still possible to treat numbers and booleans as strings by enclosing them in double quotes.
198
199## General improvements
200
201### Comments
202
203UCL supports different style of comments:
204
205* single line: `#`
206* multiline: `/* ... */`
207
208Multiline comments may be nested:
209```c
210# Sample single line comment
211/*
212 some comment
213 /* nested comment */
214 end of comment
215*/
216```
217
218### Macros support
219
220UCL supports external macros both multiline and single line ones:
221```nginx
222.macro_name "sometext";
223.macro_name {
224    Some long text
225    ....
226};
227```
228
229Moreover, each macro can accept an optional list of arguments in braces. These
230arguments themselves are the UCL object that is parsed and passed to a macro as
231options:
232
233```nginx
234.macro_name(param=value) "something";
235.macro_name(param={key=value}) "something";
236.macro_name(.include "params.conf") "something";
237.macro_name(#this is multiline macro
238param = [value1, value2]) "something";
239.macro_name(key="()") "something";
240```
241
242UCL also provide a convenient `include` macro to load content from another files
243to the current UCL object. This macro accepts either path to file:
244
245```nginx
246.include "/full/path.conf"
247.include "./relative/path.conf"
248.include "${CURDIR}/path.conf"
249```
250
251or URL (if ucl is built with url support provided by either `libcurl` or `libfetch`):
252
253	.include "http://example.com/file.conf"
254
255`.include` macro supports a set of options:
256
257* `try` (default: **false**) - if this option is `true` than UCL treats errors on loading of
258this file as non-fatal. For example, such a file can be absent but it won't stop the parsing
259of the top-level document.
260* `sign` (default: **false**) - if this option is `true` UCL loads and checks the signature for
261a file from path named `<FILEPATH>.sig`. Trusted public keys should be provided for UCL API after
262parser is created but before any configurations are parsed.
263* `glob` (default: **false**) - if this option is `true` UCL treats the filename as GLOB pattern and load
264all files that matches the specified pattern (normally the format of patterns is defined in `glob` manual page
265for your operating system). This option is meaningless for URL includes.
266* `url` (default: **true**) - allow URL includes.
267* `path` (default: empty) - A UCL_ARRAY of directories to search for the include file.
268Search ends after the first match, unless `glob` is true, then all matches are included.
269* `prefix` (default false) - Put included contents inside an object, instead
270of loading them into the root. If no `key` is provided, one is automatically generated based on each files basename()
271* `key` (default: <empty string>) - Key to load contents of include into. If
272the key already exists, it must be the correct type
273* `target` (default: object) - Specify if the `prefix` `key` should be an
274object or an array.
275* `priority` (default: 0) - specify priority for the include (see below).
276* `duplicate` (default: 'append') - specify policy of duplicates resolving:
277	- `append` - default strategy, if we have new object of higher priority then it replaces old one, if we have new object with less priority it is ignored completely, and if we have two duplicate objects with the same priority then we have a multi-value key (implicit array)
278	- `merge` - if we have object or array, then new keys are merged inside, if we have a plain object then an implicit array is formed (regardless of priorities)
279	- `error` - create error on duplicate keys and stop parsing
280	- `rewrite` - always rewrite an old value with new one (ignoring priorities)
281
282Priorities are used by UCL parser to manage the policy of objects rewriting during including other files
283as following:
284
285* If we have two objects with the same priority then we form an implicit array
286* If a new object has bigger priority then we overwrite an old one
287* If a new object has lower priority then we ignore it
288
289By default, the priority of top-level object is set to zero (lowest priority). Currently,
290you can define up to 16 priorities (from 0 to 15). Includes with bigger priorities will
291rewrite keys from the objects with lower priorities as specified by the policy.
292
293### Variables support
294
295UCL supports variables in input. Variables are registered by a user of the UCL parser and can be presented in the following forms:
296
297* `${VARIABLE}`
298* `$VARIABLE`
299
300UCL currently does not support nested variables. To escape variables one could use double dollar signs:
301
302* `$${VARIABLE}` is converted to `${VARIABLE}`
303* `$$VARIABLE` is converted to `$VARIABLE`
304
305However, if no valid variables are found in a string, no expansion will be performed (and `$$` thus remains unchanged). This may be a subject
306to change in future libucl releases.
307
308### Multiline strings
309
310UCL can handle multiline strings as well as single line ones. It uses shell/perl like notation for such objects:
311```
312key = <<EOD
313some text
314splitted to
315lines
316EOD
317```
318
319In this example `key` will be interpreted as the following string: `some text\nsplitted to\nlines`.
320Here are some rules for this syntax:
321
322* Multiline terminator must start just after `<<` symbols and it must consist of capital letters only (e.g. `<<eof` or `<< EOF` won't work);
323* Terminator must end with a single newline character (and no spaces are allowed between terminator and newline character);
324* To finish multiline string you need to include a terminator string just after newline and followed by a newline (no spaces or other characters are allowed as well);
325* The initial and the final newlines are not inserted to the resulting string, but you can still specify newlines at the beginning and at the end of a value, for example:
326
327```
328key <<EOD
329
330some
331text
332
333EOD
334```
335
336## Emitter
337
338Each UCL object can be serialized to one of the three supported formats:
339
340* `JSON` - canonic json notation (with spaces indented structure);
341* `Compacted JSON` - compact json notation (without spaces or newlines);
342* `Configuration` - nginx like notation;
343* `YAML` - yaml inlined notation.
344
345## Validation
346
347UCL allows validation of objects. It uses the same schema that is used for json: [json schema v4](http://json-schema.org). UCL supports the full set of json schema with the exception of remote references. This feature is unlikely useful for configuration objects. Of course, a schema definition can be in UCL format instead of JSON that simplifies schemas writing. Moreover, since UCL supports multiple values for keys in an object it is possible to specify generic integer constraints `maxValues` and `minValues` to define the limits of values count in a single key. UCL currently is not absolutely strict about validation schemas themselves, therefore UCL users should supply valid schemas (as it is defined in json-schema draft v4) to ensure that the input objects are validated properly.
348
349## Performance
350
351Are UCL parser and emitter fast enough? Well, there are some numbers.
352I got a 19Mb file that consist of ~700 thousand lines of json (obtained via
353http://www.json-generator.com/). Then I checked jansson library that performs json
354parsing and emitting and compared it with UCL. Here are results:
355
356```
357jansson: parsed json in 1.3899 seconds
358jansson: emitted object in 0.2609 seconds
359
360ucl: parsed input in 0.6649 seconds
361ucl: emitted config in 0.2423 seconds
362ucl: emitted json in 0.2329 seconds
363ucl: emitted compact json in 0.1811 seconds
364ucl: emitted yaml in 0.2489 seconds
365```
366
367So far, UCL seems to be significantly faster than jansson on parsing and slightly faster on emitting. Moreover,
368UCL compiled with optimizations (-O3) performs significantly faster:
369```
370ucl: parsed input in 0.3002 seconds
371ucl: emitted config in 0.1174 seconds
372ucl: emitted json in 0.1174 seconds
373ucl: emitted compact json in 0.0991 seconds
374ucl: emitted yaml in 0.1354 seconds
375```
376
377You can do your own benchmarks by running `make check` in libucl top directory.
378
379## Conclusion
380
381UCL has clear design that should be very convenient for reading and writing. At the same time it is compatible with
382JSON language and therefore can be used as a simple JSON parser. Macro logic provides an ability to extend configuration
383language (for example by including some lua code) and comments allow to disable or enable the parts of a configuration
384quickly.
385