--- /dev/null
+## 0.9.7\r
+\r
+### Lua 5.2 compatibility\r
+\r
+(These are all now defined in pl.utils)\r
+\r
+- setfenv, getfenv defined for Lua 5.2 (by Sergey Rozhenko)\r
+\r
+### Changes\r
+\r
+- array2d.flatten is new\r
+- OrderedMap:insert is new\r
+\r
+### Fixes\r
+\r
+- seq.reduce re-implemented to give correct order (Carl Ådahl)\r
+- seq.unique was broken: new test\r
+- tablex.icopy broken for last argument; new test\r
+- utils.function_arg last parm 'msg' was missing\r
+- array2d.product was broken; more sensible implementation\r
+- array2d.range, .slice, .write were broken\r
+- text optional operator % overload broken for 'fmt % fun'; new tests\r
+- a few occurances of non-existent function utils.error removed\r
+\r
+\r
+## 0.9.6\r
+\r
+### Lua 5.2 compatibility\r
+\r
+- Bad string escape in tests fixed\r
+\r
+### Changes\r
+\r
+- LuaJIT FFI used on Windows for Copy/MoveFile functionality\r
+\r
+### Fixes\r
+\r
+- Issue 13 seq.sort now calls seq.copy\r
+- issue 14 bad pattern to escape trailing separators in path.abspath\r
+- lexer: string tokens broken with some combinations\r
+- lexer: long comments broken for Lua and C\r
+- stringx.split behaves according to Python spec; extra parm meaning 'max splits'\r
+- stringx.title behaves according to Python spec\r
+- stringx.endswith broken for 2nd arg being table of postfixes\r
+- OrderedMap.set broken when value was nil and key did not exist in map; ctor throws\r
+ error if unhappy\r
+\r
+## 0.9.5\r
+\r
+### Lua 5.2 compatibility\r
+\r
+ - defines Lua 5.2 beta compatible load()\r
+ - defines table.pack()\r
+\r
+### New functions\r
+\r
+ - stringx.title(): translates "a dog's day" to "A Dog's Day"\r
+ - path.normpath(): translates 'A//B','A/./B' and 'A/C/../B' to 'A/B'\r
+ - utils.execute(): returns ok,return-code: compatible with 5.1 and 5.2\r
+\r
+### Fixes\r
+\r
+ - pretty.write() _always_ returns a string, but will return also an error string\r
+if the argument is not a table. Non-integer indices between 1 and #t are no longer falsely considered part of the array\r
+ - stringx.expandtabs() now works like the Python string method; it will expand each field up to the next tab stop\r
+ - path.normcase() was broken, because of a misguided attempt to normalize the path.\r
+ - UNC specific fix to path.abspath()\r
+ - UNC paths recognized as absolute; dir.makedir() works here\r
+ - utils.quit() varargs broken, e.g. utils.quit("answer was %d",42)\r
+ - some stray globals caused trouble with 'strict'\r
+
\ No newline at end of file
--- /dev/null
+Copyright (C) 2009 Steve Donovan, David Manura.\r
+\r
+Permission is hereby granted, free of charge, to any person obtaining a copy\r
+of this software and associated documentation files (the "Software"), to deal\r
+in the Software without restriction, including without limitation the rights\r
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\r
+copies of the Software, and to permit persons to whom the Software is\r
+furnished to do so, subject to the following conditions:\r
+\r
+The above copyright notice and this permission notice shall be included in\r
+all copies or substantial portions of the Software.\r
+\r
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF\r
+ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED\r
+TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A\r
+PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT\r
+SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR\r
+ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\r
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE\r
+OR OTHER DEALINGS IN THE SOFTWARE.\r
--- /dev/null
+Penlight Lua Libraries\r
+\r
+1. Why a new set of libraries?\r
+\r
+Penlight brings together a set of generally useful pure Lua modules,\r
+focussing on input data handling (such as reading configuration files),\r
+functional programming (such as map, reduce, placeholder expressions,etc),\r
+and OS path management. Much of the functionality is inspired by the\r
+Python standard libraries.\r
+\r
+2. Requirements\r
+\r
+The file and directory functions depend on LuaFileSystem (lfs). If you want\r
+dir.copyfile to work elegantly on Windows, then you need Alien. (Both are\r
+present in Lua for Windows.)\r
+\r
+3. Known Issues\r
+\r
+Error handling is still hit and miss.\r
+\r
+There are 7581 lines of source and 1764 lines of formal tests, \r
+which is not an ideal ratio.\r
+\r
+Formal documentation for comprehension and luabalanced is missing.\r
+\r
+4. Installation\r
+\r
+The directory structure is\r
+\r
+ lua\r
+ pl \r
+ (module files)\r
+ examples\r
+ (examples)\r
+ tests\r
+ (tests) \r
+ docs\r
+ (index.html)\r
+ api\r
+ (index.html)\r
+ modules\r
+\r
+All you need to do is copy the pl directory into your Lua module path, which\r
+is typically /usr/local/share/lua/5.1 on a Linux system (of course, you\r
+can set LUA_PATH appropriately.)\r
+\r
+With Lua for Windows, if LUA stands for 'c:\Program Files\Lua\5.1',\r
+then pl goes into LUA\lua, docs goes into LUA\examples\penlight and\r
+both examples and tests goes into LUA\examples\r
+\r
+5. Building the Documentation\r
+\r
+The Users Guide is processed by markdown.lua. If you like the section headers,\r
+you'll need to download my modified version:\r
+\r
+http://mysite.mweb.co.za/residents/sdonovan/lua/markdown.zip\r
+\r
+docgen.lua will preprocess the documentation (handles @see references)\r
+and use markdown.\r
+\r
+gen_modules.bat does the LuaDoc stuff.\r
+\r
+6. What's new with 0.8b ?\r
+\r
+Features:\r
+\r
+pl.app provides useful stuff like simple command-line argument parsing and require_here(), which \r
+makes subsequent require() calls look in the local directory by preference.\r
+\r
+p.file provides useful functions like copy(),move(), read() and write(). (These are aliases to\r
+dir.copyfile(),movefile(),utils.readfile(),writefile())\r
+\r
+Custom error trace will only show the functions in user code.\r
+\r
+More robust argument checking.\r
+\r
+In function arguments, now supports 'string lambdas', e.g. '|x| 2*x'\r
+\r
+utils.readfile,writefile now insist on being given filenames. This will cause less confusion.\r
+\r
+tablex.search() is new: will look recursively in an arbitrary table; can specify tables not to follow.\r
+tablex.move() will work with source and destination tables the same, with overlapping ranges.\r
+\r
+Bug Fixes:\r
+\r
+dir.copyfile() now works fine without Alien on Windows\r
+\r
+dir.makepath() and rmtree() had problems.\r
+\r
+tablex.compare_no_order() is now O(NlogN), as expected.\r
+tablex.move() had a problem with source size\r
+\r
+7. What's New with 0.7.0b?\r
+\r
+Features:\r
+\r
+utils.is_type(v,tp) can say is_type(s,'string') and is_type(l,List).\r
+utils.is_callable(v) either a function, or has a __call metamethod.\r
+\r
+Sequence wrappers: can write things like this:\r
+\r
+seq(s):last():filter('<'):copy()\r
+\r
+seq:mapmethod(s,name) - map using a named method over a sequence.\r
+\r
+seq:enum(s) If s is a simple sequence, then \r
+ for i,v in seq.enum(s) do print(i,v) end\r
+\r
+seq:take(s,n) Grab the next n values from a (possibly infinite)\r
+sequence.\r
+\r
+In a related change suggested by Flemming Madsden, the in-place List\r
+methods like reverse() and sort() return the list, allowing for\r
+method chaining.\r
+\r
+list.join() explicitly converts using tostring first.\r
+\r
+tablex.count_map() like seq.count_map(), but takes an equality function.\r
+\r
+tablex.difference() set difference\r
+tablex.set() explicit set generator given a list of values\r
+\r
+Template.indent_substitute() is a new Template method which adjusts\r
+for indentation and can also substitute templates themselves.\r
+\r
+pretty.read(). This reads a Lua table (as dumped by pretty.write)\r
+and attempts to be paranoid about its contents.\r
+\r
+sip.match_at_start(). Convenience function for anchored SIP matches.\r
+\r
+Bug Fixes:\r
+\r
+tablex.deepcompare() was confused by false boolean values, which\r
+it thought were synonymous with being nil.\r
+\r
+pretty.write() did not handle cycles, and could not display tables\r
+with 'holes' properly (Flemming Madsden)\r
+\r
+The SIP pattern '$(' was not escaped properly.\r
+sip.match() did not pass on options table.\r
+\r
+seq.map() was broken for double-valued sequences.\r
+seq.copy_tuples() did not use default_iter(), so did not e.g. like\r
+table arguments.\r
+\r
+dir.copyfile() returns the wrong result for *nix operations.\r
+dir.makepath() was broken for non-Windows paths.\r
+\r
+8. What's New with 0.6.3?\r
+\r
+The map and reduce functions now take the function first, as Nature intended.\r
+\r
+The Python-like overloading of '*' for strings has been dropped, since it\r
+is silly. Also, strings are no longer callable; use 's:at(1)' instead of\r
+'s(1)' - this tended to cause Obscure Error messages.\r
+\r
+Wherever a function argument is expected, you can use the operator strings\r
+like '+','==',etc as well as pl.operator.add, pl.operator.eq, etc.\r
+(see end of pl/operator.lua for the full list.)\r
+\r
+tablex now has compare() and compare_no_order(). An explicit set()\r
+function has been added which constructs a table with the specified\r
+keys, all set to a value of true.\r
+\r
+List has reduce() and partition() (This is a cool function which \r
+separates out elements of a list depending on a classifier function.)\r
+\r
+There is a new array module which generalizes tablex operations like\r
+map and reduce for two-dimensional arrays.\r
+\r
+The famous iterator over permutations from PiL 9.3 has been included.\r
+\r
+David Manura's list comprehension library has been included.\r
+\r
+Also, utils now contains his memoize function, plus a useful function\r
+args which captures the case where varargs contains nils.\r
+\r
+There was a bug with dir.copyfile() where the flag was the wrong way round.\r
+\r
+config.lines() had a problem with continued lines.\r
+\r
+Some operators were missing in pl.operator; have renamed them to be\r
+consistent with the Lua metamethod names.\r
+\r
+\r
--- /dev/null
+project = 'Penlight'\r
+description = 'Penlight Lua Libraries 1.0.0'\r
+full_description = 'The documentation is available @{01-introduction.md|here}.'\r
+title = 'Penlight Documentation'\r
+dir = 'api'\r
+topics = 'manual'\r
+examples = {'../examples','../tests/test-data.lua'}\r
+package = 'pl'\r
+format = 'discount'\r
+file = '../lua/pl'\r
--- /dev/null
+## Introduction
+
+### Purpose
+
+It is often said of Lua that it does not include batteries. That is because the goal of Lua is to produce a lean expressive language that will be used on all sorts of machines, (some of which don't even have hierarchical filesystems). The Lua language is the equivalent of an operating system kernel; the creators of Lua do not see it as their responsibility to create a full software ecosystem around the language. That is the role of the community.
+
+A principle of software design is to recognize common patterns and reuse them. If you find yourself writing things like `io.write(string.format('the answer is %d ',42))` more than a number of times then it becomes useful just to define a function `printf`. This is good, not just because repeated code is harder to maintain, but because such code is easier to read, once people understand your libraries.
+
+Penlight captures many such code patterns, so that the intent of your code becomes clearer. For instance, a Lua idiom to copy a table is `{unpack(t)}`, but this will only work for 'small' tables (for a given value of 'small') so it is not very robust. Also, the intent is not clear. So `tablex.deepcopy` is provided, which will also copy nested tables and and associated metatables, so it can be used to clone complex objects.
+
+The default error handling policy follows that of the Lua standard libraries: if a argument is the wrong type, then an error will be thrown, but otherwise we return `nil,message` if there is a problem. There are some exceptions; functions like `input.fields` default to shutting down the program immediately with a useful message. This is more appropriate behaviour for a _script_ than providing a stack trace. (However, this default can be changed.) The lexer functions always throw errors, to simplify coding, and so should be wrapped in `pcall`.
+
+By default, the error stacktrace starts with your code, since you are not usually interested in the internal details of the library. ??
+
+If you are used to Python conventions, please note that all indices consistently start at 1.
+
+The Lua function `table.foreach` has been deprecated in favour of the `for in` statement, but such an operation becomes particularly useful with the higher-order function support in Penlight. Note that `tablex.foreach` reverses the order, so that the function is passed the value and then the key. Although perverse, this matches the intended use better.
+
+The only important external dependence of Penlight is [LuaFileSystem](http://keplerproject.github.com/luafilesystem/manual.html) (`lfs`), and if you want `dir.copyfile` to work cleanly on Windows, you will need either [alien](http://alien.luaforge.net/) or be using [LuaJIT](http://luajit.org) as well. (The fallback is to call the equivalent shell commands.)
+
+Some of the examples in this guide were created using [ilua](http://lua-users.org/wiki/InteractiveLua), which doesn't require '=' to print out expressions, and will attempt to print out table results as nicely as possible. This is also available under Lua for Windows, as a library, so the command `lua -lilua -s` will work (the s option switches off 'strict' variable checking, which is annoying and conflicts with the use of `_DEBUG` in some of these libraries.
+
+### To Inject or not to Inject?
+
+It was realized a long time ago that large programs needed a way to keep names distinct by putting them into tables (Lua), namespaces (C++) or modules (Python). It is obviously impossible to run a company where everyone is called 'Bruce', except in Monty Python skits. These 'namespace clashes' are more of a problem in a simple language like Lua than in C++, because C++ does more complicated lookup over 'injected namespaces'. However, in a small group of friends, 'Bruce' is usually unique, so in particular situations it's useful to drop the formality and not use last names. It depends entirely on what kind of program you are writing, whether it is a ten line script or a ten thousand line program.
+
+So the Penlight library provides the formal way and the informal way, without imposing any preference. You can do it formally like:
+
+ local utils = require 'pl.utils'
+ utils.printf("%s\n","hello, world!")
+
+or informally like:
+
+ require 'pl'
+ utils.printf("%s\n","That feels better")
+
+`require 'pl'` makes all the separate Penlight modules available, without needing to require them each individually.. Generally, the formal way is better when writing modules, since then there are no global side-effects and the dependencies of your module are made explicit.
+
+With Penlight after 0.9, please note that `require 'pl.utils'` no longer implies that a global table `pl.utils` exists, since these new modules are no longer created with `module()`.
+
+Penlight will not bring in functions into the global table, or clobber standard tables like 'io'. require('pl') will bring tables like 'utils','tablex',etc into the global table _if they are used_. This 'load-on-demand' strategy ensures that the whole kitchen sink is not loaded up front, so this method is as efficient as explicitly loading required modules.
+
+You have an option to bring the `pl.stringx` methods into the standard string table. All strings have a metatable that allows for automatic lookup in `string`, so we can say `s:upper()`. Importing `stringx` allows for its functions to also be called as methods: `s:strip()`,etc:
+
+ require 'pl'
+ stringx.import()
+
+or, more explicitly:
+
+ require('pl.stringx').import()
+
+A more delicate operation is importing tables into the local environment. This is convenient when the context makes the meaning of a name very clear:
+
+ > require 'pl'
+ > utils.import(math)
+ > = sin(1.2)
+ 0.93203908596723
+
+`utils.import` can also be passed a module name as a string, which is first required and then imported. If used in a module, `import` will bring the symbols into the module context.
+
+Keeping the global scope simple is very necessary with dynamic languages. Using global variables in a big program is always asking for trouble, especially since you do not have the spell-checking provided by a compiler. The `pl.strict` module enforces a simple rule: globals must be 'declared'. This means that they must be assigned before use; assigning to `nil` is sufficient.
+
+ > require 'pl.strict'
+ > print(x)
+ stdin:1: variable 'x' is not declared
+ > x = nil
+ > print(x)
+ nil
+
+The `strict` module provided by Penlight is compatible with the 'load-on-demand' scheme used by `require 'pl`.
+
+`strict` also disallows assignment to global variables, except in the main program. Generally, modules have no business messing with global scope; if you must do it, then use a call to `rawset`. Similarly, if you have to check for the existance of a global, use `rawget`.
+
+If you wish to enforce strictness globally, then just add `require 'pl.strict'` at the end of `pl/init.lua`.
+
+### What are function arguments in Penlight?
+
+Many functions in Penlight themselves take function arguments, like `map` which applies a function to a list, element by element. You can use existing functions, like `math.max`, anonymous functions (like `function(x,y) return x > y end`), or operations by name (e.g '*' or '..'). The module `pl.operator` exports all the standard Lua operations, like the Python module of the same name. Penlight allows these to be referred to by name, so `operator.gt` can be more concisely expressed as '>'.
+
+Note that the `map` functions pass any extra arguments to the function, so we can have `ls:filter('>',0)`, which is a shortcut for `ls:filter(function(x) return x > 0 end)`.
+
+Finally, `pl.func` supports _placeholder expressions_ in the Boost lambda style, so that an anonymous function to multiply the two arguments can be expressed as `_1*_2`.
+
+To use them directly, note that _all_ function arguments in Penlight go through `utils.function_arg`. `pl.func` registers itself with this function, so that you can directly use placeholder expressions with standard methods:
+
+ > _1 = func._1
+ > = List{10,20,30}:map(_1+1)
+ {11,21,31}
+
+Another option for short anonymous functions is provided by `utils.string_lambda`; since 0.9 you have to explicitly ask for this feature:
+
+ > L = require 'pl.utils'.string_lambda
+ > = List{10,20,30}:map (L'|x| x + 1')
+ {11,21,31}
+
+### Pros and Cons of Loopless Programming
+
+The standard loops-and-ifs 'imperative' style of programming is dominant, and often seems to be the 'natural' way of telling a machine what to do. It is in fact very much how the machine does things, but we need to take a step back and find ways of expressing solutions in a higher-level way. For instance, applying a function to all elements of a list is a common operation:
+
+ local res = {}
+ for i = 1,#ls do
+ res[i] = fun(ls[i])
+ end
+
+This can be efficiently and succintly expressed as `ls:map(fun)`. Not only is there less typing but the intention of the code is clearer. If readers of your code spend too much time trying to guess your intention by analyzing your loops, then you have failed to express yourself clearly. Similarly, `ls:filter('>',0)` will give you all the values in a list greater than zero. (Of course, if you don't feel like using `List`, or have non-list-like tables, then `pl.tablex` offers the same facilities. In fact, the `List` methods are implemented using `tablex' functions.)
+
+A common observation is that loopless programming is less efficient, particularly in the way it uses memory. `ls1:map2('*',ls2):reduce '+'` will give you the dot product of two lists, but an unnecessary temporary list is created. But efficiency is relative to the actual situation, it may turn out to be _fast enough_, or may not appear in any crucial inner loops, etc.
+
+Writing loops is 'error-prone and tedious', as Stroustrup says. But any half-decent editor can be taught to do much of that typing for you. The question should actually be: is it tedious to _read_ loops? As with natural language, programmers tend to read chunks at a time. A for-loop causes no surprise, and probably little brain activity. One argument for loopless programming is the loops that you _do_ write stand out more, and signal 'something different happening here'. It should not be an all-or-nothing thing, since most programs require a mixture of idioms that suit the problem. Some languages (like APL) do nearly everything with map and reduce operations on arrays, and so solutions can sometimes seem forced. Wisdom is knowing when a particular idiom makes a particular problem easy to _solve_ and the solution easy to _explain_ afterwards.
+
+### Generally useful functions.
+
+The function `printf` discussed earlier is included in `pl.utils` because it makes properly formatted output easier. (There is an equivalent `fprintf` which also takes a file object parameter, just like the C function.)
+
+Utility functions like `is_callable` and `is_type` help with identifying what kind of animal you are dealing with. Obviously, a function is callable, but an object can be callable as well if it has overriden the `__call` metamethod. The Lua `type` function handles the basic types, but can't distinguish between different kinds of objects, which are all tables. So `is_type` handles both cases, like `is_type(s,"string")` and `is_type(ls,List)`.
+
+A common pattern when working with Lua varargs is capturing all the arguments in a table:
+
+ function t(...)
+ local args = {...}
+ ...
+ end
+
+But this will bite you someday when `nil` is one of the arguments, since this will put a 'hole' in your table. In particular, `#ls` will only give you the size upto the `nil` value. Hence the need for `table.pack` - this is a new Lua 5.2 function which Penlight defines also for Lua 5.1.
+
+ function t(...)
+ local args,n = table.pack(...)
+ for i = 1,n do
+ ...
+ end
+ end
+
+The 'memoize' pattern occurs when you have a function which is expensive to call, but will always return the same value subsequently. `utils.memoize` is given a function, and returns another function. This calls the function the first time, saves the value for that argument, and thereafter for that argument returns the saved value. This is a more flexible alternative to building a table of values upfront, since in general you won't know what values are needed.
+
+ sum = utils.memoize(function(n)
+ local sum = 0
+ for i = 1,n do sum = sum + i end
+ return sum
+ end)
+ ...
+ s = sum(1e8) --takes time!
+ ...
+ s = sum(1e8) --returned saved value!
+
+Penlight is fully compatible with Lua 5.1, 5.2 and LuaJIT 2. To ensure this, `utils` also defines the global Lua 5.2 [load](http://www.lua.org/work/doc/manual.html#pdf-load) function when needed.
+
+ * the input (either a string or a function)
+ * the source name used in debug information
+ * the mode is a string that can have either or both of 'b' or 't', depending on whether the source is a binary chunk or text code (default is 'bt')
+ * the environment for the compiled chunk
+
+Using `load` should reduce the need to call the deprecated function `setfenv`, and make your Lua 5.1 code 5.2-friendly.
+
+### Application Support
+
+`app.parse_args` is a simple command-line argument parser. If called without any arguments, it tries to use the global `arg` array. It returns the _flags_ (options begining with '-') as a table of name/value pairs, and the _arguments_ as an array. It knows about long GNU-style flag names, e.g. `--value`, and groups of short flags are understood, so that `-ab` is short for `-a -b`. The flags result would then look like `{value=true,a=true,b=true}`.
+
+Flags may take values. The command-line `--value=open -n10` would result in `{value='open',n='10'}`; generally you can use '=' or ':' to separate the flag from its value, except in the special case where a short flag is followed by an integer. Or you may specify upfront that some flags have associated values, and then the values will follow the flag.
+
+ > require 'pl'
+ > flags,args = utils.parse_args({'-o','fred','-n10','fred.txt'},{o=true})
+ > pretty.dump(flags)
+ {o='fred',n='10'}
+
+`parse_args` is not intelligent or psychic; it will not convert any flag values or arguments for you, or raise errors. For that, have a look at @{08-additional.md.Command_line_Programs_with_Lapp|Lapp}.
+
+An application which consists of several files usually cannot use `require` to load files in the same directory as the main script. `app.require_here()` ensures that the Lua module path is modified so that files found locally are found first. In the `examples` directory, `test-symbols.lua` uses this function to ensure that it can find `symbols.lua` even if it is not run from this directory.
+
+`app.appfile` will create a filename that your application can use to store its private data, based on the script name. For example, `app.appfile "test.txt"` from a script called `testapp.lua` produces the following file on my Windows machine:
+
+ C:\Documents and Settings\SJDonova\.testapp\test.txt
+
+and the equivalent on my Linux machine:
+
+ /home/sdonovan/.testapp/test.txt
+
+If `.testapp` does not exist, it will be created.
+
+Penlight makes it convenient to save application data in Lua format. You can use `pretty.dump(t,file)` to write a Lua table in a human-readable form to a file, and `pretty.read(file.read(file))` to generate the table again, using the `pretty` module.
+
+
+### Simplifying Object-Oriented Programming in Lua
+
+Lua is similar to JavaScript in that the concept of class is not directly supported by the language. In fact, Lua has a very general mechanism for extending the behaviour of tables which makes it straightforward to implement classes. A table's behaviour is controlled by its metatable. If that metatable has a `__index` function or table, this will handle looking up anything which is not found in the original table. A class is just a table with an `__index` key pointing to itself. Creating an object involves making a table and setting its metatable to the class; then when handling `obj.fun`, Lua first looks up `fun` in the table `obj`, and if not found it looks it up in the class. `obj:fun(a)` is just short for `obj.fun(obj,a)`. So with the metatable mechanism and this bit of syntactic sugar, it is straightforward to implement classic object orientation.
+
+ -- animal.lua
+
+ class = require 'pl.class'
+
+ class.Animal()
+
+ function Animal:_init(name)
+ self.name = name
+ end
+
+ function Animal:__tostring()
+ return self.name..': '..self:speak()
+ end
+
+ class.Dog(Animal)
+
+ function Dog:speak()
+ return 'bark'
+ end
+
+ class.Cat(Animal)
+
+ function Cat:_init(name,breed)
+ self:super(name) -- must init base!
+ self.breed = breed
+ end
+
+ function Cat:speak()
+ return 'meow'
+ end
+
+ class.Lion(Cat)
+
+ function Lion:speak()
+ return 'roar'
+ end
+
+ fido = Dog('Fido')
+ felix = Cat('Felix','Tabby')
+ leo = Lion('Leo','African')
+
+ $ lua -i animal.lua
+ > = fido,felix,leo
+ Fido: bark Felix: meow Leo: roar
+ > = leo:is_a(Animal)
+ true
+ > = leo:is_a(Dog)
+ false
+ > = leo:is_a(Cat)
+ true
+
+All Animal does is define `__tostring`, which Lua will use whenever a string representation is needed of the object. In turn, this relies on `speak`, which is not defined. So it's what C++ people would call an abstract base class; the specific derived classes like Dog define `speak`. (Please note that if derived classes have their own constructors, they must explicitly call the base constructor for their base class; this is conveniently available as the `super` method.)
+
+All such objects will have a `is_a` method, which looks up the inheritance chain to find a match. Another form is `class_of`, which can be safely called on all objects, so instead of `leo:is_a(Animal)` one can say `Animal:class_of(leo)`.
+
+There are two ways to define a class, either `class.Name()` or `Name = class()`; both work identically, except that the first form will always put the class in the current environment (whether global or module); the second form provides more flexibility about where to store the class. The first form does _name_ the class by setting the `_name` field, which can be useful in identifying the objects of this type later. This session illustrates the usefulness of having named classes, if no `__tostring` method is explicitly defined.
+
+ > class.Fred()
+ > a = Fred()
+ > = a
+ Fred: 00459330
+ > Alice = class()
+ > b = Alice()
+ > = b
+ table: 00459AE8
+ > Alice._name = 'Alice'
+ > = b
+ Alice: 00459AE8
+
+So `Alice = class(); Alice._name = 'Alice'` is exactly the same as `class.Alice()`.
+
+This useful notation is borrowed from Hugo Etchegoyen's [classlib](http://lua-users.org/wiki/MultipleInheritanceClasses) which further extends this concept to allow for multiple inheritance.
+
+Penlight provides a number of useful classes; there is `List`, which is a Lua clone of the standard Python list object, and `Set` which represents sets. There are three kinds of _map_ defined: `Map`, `MultiMap` (where a key may have multiple values) and `OrderedMap` (where the order of insertion is remembered.). There is nothing special about these classes and you may inherit from them.
+
+_Properties_ are a useful object-oriented pattern. We wish to control access to a field, but don't wish to force the user of the class to say `obj:get_field()` etc. This excerpt from `tests/test-class.lua` shows how it is done:
+
+
+ local MyProps = class(class.properties)
+ local setted_a, got_b
+
+ function MyProps:_init ()
+ self._a = 1
+ self._b = 2
+ end
+
+ function MyProps:set_a (v)
+ setted_a = true
+ self._a = v
+ end
+
+ function MyProps:get_b ()
+ got_b = true
+ return self._b
+ end
+
+ local mp = MyProps()
+
+ mp.a = 10
+
+ asserteq(mp.a,10)
+ asserteq(mp.b,2)
+ asserteq(setted_a and got_b, true)
+
+The convention is that the internal field name is prefixed with an underscore; when reading `mp.a`, first a check for an explicit _getter_ `get_a` and then only look for `_a`. Simularly, writing `mp.a` causes the _setter_ `set_a` to be used.
+
+This is cool behaviour, but like much Lua metaprogramming, it is not free. Method lookup on such objects goes through `__index` as before, but now `__index` is a function which has to explicitly look up methods in the class, before doing any property indexing, which is not going to be as fast as field lookup. If however, your accessors actually do non-trivial things, then the extra overhead could be worth it.
+
+This is not really intended for _access control_ because external code can write to `mp._a` directly. It is possible to have this kind of control in Lua, but it again comes with run-time costs, and in this case a simple audit of code will reveal any naughty use of 'protected' fields.
--- /dev/null
+## Tables and Arrays
+
+<a id="list"/>
+
+### Python-style Lists
+
+One of the elegant things about Lua is that tables do the job of both lists and dicts (as called in Python) or vectors and maps, (as called in C++), and they do it efficiently. However, if we are dealing with 'tables with numerical indices' we may as well call them lists and look for operations which particularly make sense for lists. The Penlight `List` class was originally written by Nick Trout for Lua 5.0, and translated to 5.1 and extended by myself. It seemed that borrowing from Python was a good idea, and this eventually grew into Penlight.
+
+Here is an example showing `List` in action; it redefines `__tostring`, so that it can print itself out more sensibly:
+
+ > List = require 'pl.List' --> automatic with require 'pl' <---
+ > l = List()
+ > l:append(10)
+ > l:append(20)
+ > = l
+ {10,20}
+ > l:extend {30,40}
+ {10,20,30,40}
+ > l:insert(1,5)
+ {5,10,20,30,40}
+ > = l:pop()
+ 40
+ > = l
+ {5,10,20,30}
+ > = l:index(30)
+ 4
+ > = l:contains(30)
+ true
+ > = l:reverse() ---> note: doesn't make a copy!
+ {30,20,10,5}
+
+Although methods like `sort` and `reverse` operate in-place and change the list, they do return the original list. This makes it possible to do _method chaining_, like `ls = ls:append(10):append(20):reverse():append(1)`. But (and this is an important but) no extra copy is made, so `ls` does not change identity. `List` objects (like tables) are _mutable_, unlike strings. If you want a copy of a list, then `List(ls)` will do the job, i.e. it acts like a copy constructor. However, if passed any other table, `List` will just set the metatable of the table and _not_ make a copy.
+
+A particular feature of Python lists is _slicing_. This is fully supported in this version of `List`, except we use 1-based indexing. So `List.slice` works rather like `string.sub`:
+
+ > l = List {10,20,30,40}
+ > = l:slice(1,1) ---> note: creates a new list!
+ {10}
+ > = l:slice(2,2)
+ {20}
+ > = l:slice(2,3)
+ {20,30}
+ > = l:slice(2,-2)
+ {20,30}
+ > = l:slice_assign(2,2,{21,22,23})
+ {10,21,22,23,30,40}
+ > = l:chop(1,1)
+ {21,22,23,30,40}
+
+Functions like `slice_assign` and `chop` modify the list; the first is equivalent to Python`l[i1:i2] = seq` and the second to `del l[i1:i2]`.
+
+List objects are ultimately just Lua 'list-like' tables, but they have extra operations defined on them, such as equality and concatention. For regular tables, equality is only true if the two tables are _identical objects_, whereas two lists are equal if they have the same contents, i.e. that `l1[i]==l2[i]` for all elements.
+
+ > l1 = List {1,2,3}
+ > l2 = List {1,2,3}
+ > = l1 == l2
+ true
+ > = l1..l2
+ {1,2,3,1,2,3}
+
+The `List` constructor can be passed a function. If so, it's assumed that this is an iterator function that can be repeatedly called to generate a sequence. One such function is `io.lines`; the following short, intense little script counts the number of lines in standard input:
+
+ -- linecount.lua
+ require 'pl'
+ ls = List(io.lines())
+ print(#ls)
+
+`List.iterate` captures what `List` considers a sequence. In particular, it can also iterate over all 'characters' in a string:
+
+ > for ch in List.iterate 'help' do io.write(ch,' ') end
+ h e l p >
+
+Since the function `iterate` is used internally by the `List` constructor, strings can be made into lists of character strings very easily.
+
+There are a number of operations that go beyond the standard Python methods. For instance, you can _partition_ a list into a table of sublists using a function. In the simplest form, you use a predicate (a function returning a boolean value) to partition the list into two lists, one of elements matching and another of elements not matching. But you can use any function; if we use `type` then the keys will be the standard Lua type names.
+
+ > ls = List{1,2,3,4}
+ > ops = require 'pl.operator'
+ > ls:partition(function(x) return x > 2 end)
+ {false={1,2},true={3,4}}
+ > ls = List{'one',math.sin,List{1},10,20,List{1,2}}
+ > ls:partition(type)
+ {function={function: 00369110},string={one},number={10,20},table={{1},{1,2}}}
+
+This is one `List` method which returns a table which is not a `List`. Bear in mind that you can always call a `List` method on a plain table argument, so `List.partition(t,type)` works as expected. But these functions will only operate on the array part of the table.
+
+Stacks occur everywhere in computing. `List` supports stack-like operations; there is already `pop` (remove and return last value) and `append` acts like `push` (add a value to the end). `push` is provided as an alias for `append`, and the other stack operation (size) is simply the size operator `#`. Queues can also be implemented; you use `pop` to take values out of the queue, and `put` to insert a value at the begining.
+
+
+### Map and Set classes
+
+The `Map` class exposes what Python would call a 'dict' interface, and accesses the hash part of the table. The name 'Map' is used to emphasize the interface, not the implementation; it is an object which maps keys onto values; `m['alice']` or the equivalent `m.alice` is the access operation. This class also provides explicit `set` and `get` methods, which are trivial for regular maps but get interesting when `Map` is subclassed. The other operation is `update`, which extends a map by copying the keys and values from another table, perhaps overwriting existing keys:
+
+ > Map = require 'pl.Map'
+ > m = Map{one=1,two=2}
+ > m:update {three=3,four=4,two=20}
+ > = m == M{one=1,two=20,three=3,four=4}
+ true
+
+The method `values` returns a list of the values, and `keys` returns a list of the keys; there is no guarantee of order. `getvalues` is given a list of keys and returns a list of values associated with these keys:
+
+ > m = Map{one=1,two=2,three=3}
+ > = m:getvalues {'one','three'}
+ {1,3}
+ > = m:getvalues(m:keys()) == m:values()
+ true
+
+When querying the value of a `Map`, it is best to use the `get` method:
+
+ > print(m:get 'one', m:get 'two')
+ 1 2
+
+The reason is that `m[key]` can be ambiguous; due to the current implementation, `m["get"]` will always succeed, because if a value is not present in the map, it will be looked up in the `Map` metatable, which contains a method `get`. There is currently no simple solution to this annoying restriction.
+
+There are some useful classes which inherit from `Map`. An `OrderedMap` behaves like a `Map` but keeps its keys in order if you use its `set` method to add keys and values. Like all the 'container' classes in Penlight, it defines an `iter` method for iterating over its values; this will return the keys and values in the order of insertion; the `keys` and `values` methods likewise.
+
+A `MultiMap` allows multiple values to be associated with a given key. So `set` (as before) takes a key and a value, but calling it with the same key and a different value does not overwrite but adds a new value. `get` (or using `[]`) will return a list of values.
+
+A `Set` can be seen as a special kind of `Map`, where all the values are `true`, the keys are the values, and the order is not important. So in this case `Set.values` is defined to return a list of the keys. Sets can display themselves, and the basic operations like `union` (`+`) and `intersection` (`*`) are defined.
+
+ > Set = require 'pl.Set'
+ > = Set{'one','two'} == Set{'two','one'}
+ true
+ > fruit = Set{'apple','banana','orange'}
+ > = fruit['banana']
+ true
+ > = fruit['hazelnut']
+ nil
+ > = fruit:values()
+ {apple,orange,banana}
+ > colours = Set{'red','orange','green','blue'}
+ > = fruit,colours
+ [apple,orange,banana] [blue,green,orange,red]
+ > = fruit+colours
+ [blue,green,apple,red,orange,banana]
+ > = fruit*colours
+ [orange]
+
+There are also the functions `Set.difference` and `Set.symmetric_difference`. The first answers the question 'what fruits are not colours?' and the second 'what are fruits and colours but not both?'
+
+ > = fruit - colours
+ [apple,banana]
+ > = fruit ^ colours
+ [blue,green,apple,red,banana]
+
+Adding elements to a set is simply `fruit['peach'] = true` and removing is `fruit['apple'] = nil` . To make this simplicity properly, the `Set` class has no methods - either you use the operator forms or explicitly use `Set.intersect` etc. In this way we avoid the ambiguity that plagues `Map`.
+
+
+(See `pl.Map` and `pl.Set`)
+
+### Useful Operations on Tables
+
+@lookup pl.tablex
+
+Some notes on terminology: Lua tables are usually _list-like_ (like an array) or _map-like_ (like an associative array or dict); they can of course have a list-like and a map-like part. Some of the table operations only make sense for list-like tables, and some only for map-like tables. (The usual Lua terminology is the array part and the hash part of the table, which reflects the actual implementation used; it is more accurate to say that a Lua table is an associative map which happens to be particularly efficient at acting like an array.)
+
+The functions provided in `table` provide all the basic manipulations on Lua tables, but as we saw with the `List` class, it is useful to build higher-level operations on top of those functions. For instance, to copy a table involves this kind of loop:
+
+ local res = {}
+ for k,v in pairs(T) do
+ res[k] = v
+ end
+ return res
+
+The `tablex` module provides this as `copy`, which does a _shallow_ copy of a table. There is also `deepcopy` which goes further than a simple loop in two ways; first, it also gives the copy the same metatable as the original (so it can copy objects like `List` above) and any nested tables will also be copied, to arbitrary depth. There is also `icopy` which operates on list-like tables, where you can set optionally set the start index of the source and destination as well. It ensures that any left-over elements will be deleted:
+
+ asserteq(icopy({1,2,3,4,5,6},{20,30}),{20,30}) -- start at 1
+ asserteq(icopy({1,2,3,4,5,6},{20,30},2),{1,20,30}) -- start at 2
+ asserteq(icopy({1,2,3,4,5,6},{20,30},2,2),{1,30}) -- start at 2, copy from 2
+
+(This code from the `tablex` test module shows the use of `pl.test.asserteq`)
+
+Whereas, `move` overwrites but does not delete the rest of the destination:
+
+ asserteq(move({1,2,3,4,5,6},{20,30}),{20,30,3,4,5,6})
+ asserteq(move({1,2,3,4,5,6},{20,30},2),{1,20,30,4,5,6})
+ asserteq(move({1,2,3,4,5,6},{20,30},2,2),{1,30,3,4,5,6})
+
+(The difference is somewhat like that between C's `strcpy` and `memmove`.)
+
+To summarize, use `copy` or `deepcopy` to make a copy of an arbitrary table. To copy into a map-like table, use `update`; to copy into a list-like table use `icopy`, and `move` if you are updating a range in the destination.
+
+To complete this set of operations, there is `insertvalues` which works like `table.insert` except that one provides a table of values to be inserted, and `removevalues` which removes a range of values.
+
+ asserteq(insertvalues({1,2,3,4},2,{20,30}),{1,20,30,2,3,4})
+ asserteq(insertvalues({1,2},{3,4}),{1,2,3,4})
+
+Another example:
+
+ > T = require 'pl.tablex'
+ > t = {10,20,30,40}
+ > = T.removevalues(t,2,3)
+ {10,40}
+ > = T.insertvalues(t,2,{20,30})
+ {10,20,30,40}
+
+
+In a similar spirit to `deepcopy`, `deepcompare` will take two tables and return true only if they have exactly the same values and structure.
+
+ > t1 = {1,{2,3},4}
+ > t2 = deepcopy(t1)
+ > = t1 == t2
+ false
+ > = deepcompare(t1,t2)
+ true
+
+`find` will return the index of a given value in a list-like table. Note that like `string.find` you can specify an index to start searching, so that all instances can be found. There is an optional fourth argument, which makes the search start at the end and go backwards, so we could define `rfind` like so:
+
+ function rfind(t,val,istart)
+ return tablex.find(t,val,istart,true)
+ end
+
+`find` does a linear search, so it can slow down code that depends on it. If efficiency is required for large tables, consider using an _index map_. `index_map` will return a table where the keys are the original values of the list, and the associated values are the indices. (It is almost exactly the representation needed for a _set_.)
+
+ > t = {'one','two','three'}
+ > = tablex.find(t,'two')
+ 2
+ > = tablex.find(t,'four')
+ nil
+ > il = tablex.index_map(t)
+ > = il['two']
+ 2
+ > = il.two
+ 2
+
+A version of `index_map` called `makeset` is also provided, where the values are just `true`. This is useful because two such sets can be compared for equality using `deepcompare`:
+
+ > = deepcompare(makeset {1,2,3},makeset {2,1,3})
+ true
+
+Consider the problem of determining the new employees that have joined in a period. Assume we have two files of employee names:
+
+ (last-month.txt)
+ smith,john
+ brady,maureen
+ mongale,thabo
+
+ (this-month.txt)
+ smith,john
+ smit,johan
+ brady,maureen
+ mogale,thabo
+ van der Merwe,Piet
+
+To find out differences, just make the employee lists into sets, like so:
+
+ require 'pl'
+
+ function read_employees(file)
+ local ls = List(io.lines(file)) -- a list of employees
+ return tablex.makeset(ls)
+ end
+
+ last = read_employees 'last-month.txt'
+ this = read_employees 'this-month.txt'
+
+ -- who is in this but not in last?
+ diff = tablex.difference(this,last)
+
+ -- in a set, the keys are the values...
+ for e in pairs(diff) do print(e) end
+
+ -- *output*
+ -- van der Merwe,Piet
+ -- smit,johan
+
+The `difference` operation is easy to write and read:
+
+ for e in pairs(this) do
+ if not last[e] then
+ print(e)
+ end
+ end
+
+Using `difference` here is not that it is a tricky thing to code, it is that you are stating your intentions clearly to other readers of your code. (And naturally to your future self, in six months time.)
+
+`find_if` will search a table using a function. The optional third argument is a value which will be passed as a second argument to the function. `pl.operator` provides the Lua operators conveniently wrapped as functions, so the basic comparison functions are available:
+
+ > ops = require 'pl.operator'
+ > = tablex.find_if({10,20,30,40},ops.gt,20)
+ 3 true
+
+Note that `find_if` will also return the _actual value_ returned by the function, which of course is usually just `true` for a boolean function, but any value which is not `nil` and not `false` can be usefully passed back.
+
+`deepcompare` does a thorough recursive comparison, but otherwise using the default equality operator. `compare` allows you to specify exactly what function to use when comparing two list-like tables, and `compare_no_order` is true if they contain exactly the same elements. Do note that the latter does not need an explicit comparison function - in this case the implementation is actually to compare the two sets, as above:
+
+ > = compare_no_order({1,2,3},{2,1,3})
+ true
+ > = compare_no_order({1,2,3},{2,1,3},'==')
+ true
+
+(Note the special string '==' above; instead of saying `ops.gt` or `ops.eq` we can use the strings '>' or '==' respectively.)
+
+There are several ways to merge tables in PL. If they are list-like, then see the operations defined by `pl.List`, like concatenation. If they are map-like, then `merge` provides two basic operations. If the third arg is false, then the result only contains the keys that are in common between the two tables, and if true, then the result contains all the keys of both tables. These are in fact generalized set union and intersection operations:
+
+ > S1 = {john=27,jane=31,mary=24}
+ > S2 = {jane=31,jones=50}
+ > = tablex.merge(S1, S2, false)
+ {jane=31}
+ > = tablex.merge(S1, S2, true)
+ {mary=24,jane=31,john=27,jones=50}
+
+When working with tables, you will often find yourself writing loops like in the first example. Loops are second nature to programmers, but they are often not the most elegant and self-describing way of expressing an operation. Consider the `map` function, which creates a new table by applying a function to each element of the original:
+
+ > = map(math.sin, {1,2,3,4})
+ { 0.84, 0.91, 0.14, -0.76}
+ > = map(function(x) return x*x end, {1,2,3,4})
+ {1,4,9,16}
+
+`map` saves you from writing a loop, and the resulting code is often clearer, as well as being shorter. This is not to say that 'loops are bad' (although you will hear that from some extremists), just that it's good to capture standard patterns. Then the loops you do write will stand out and acquire more significance.
+
+`pairmap` is interesting, because the function works with both the key and the value.
+
+ > t = {fred=10,bonzo=20,alice=4}
+ > = pairmap(function(k,v) return v end, t)
+ {4,10,20}
+ > = pairmap(function(k,v) return k end, t)
+ {'alice','fred','bonzo'}
+
+(These are common enough operations that the first is defined as `values` and the second as `keys`.) If the function returns two values, then the _second_ value is considered to be the new key:
+
+ > = pairmap(t,function(k,v) return v+10, k:upper() end)
+ {BONZO=30,FRED=20,ALICE=14}
+
+`map2` applies a function to two tables:
+
+ > map2(ops.add,{1,2},{10,20})
+ {11,22}
+ > map2('*',{1,2},{10,20})
+ {10,40}
+
+The various map operations generate tables; `reduce` applies a function of two arguments over a table and returns the result as a scalar:
+
+ > reduce ('+', {1,2,3})
+ 6
+ > reduce ('..', {'one','two','three'})
+ 'onetwothree'
+
+Finally, `zip` sews different tables together:
+
+ > = zip({1,2,3},{10,20,30})
+ {{1,10},{2,20},{3,30}}
+
+Browsing through the documentation, you will find that `tablex` and `List` share methods. For instance, `tablex.imap` and `List.map` are basically the same function; they both operate over the array-part of the table and generate another table. This can also be expressed as a _list comprehension_ `C 'f(x) for x' (t)` which makes the operation more explicit. So why are there different ways to do the same thing? The main reason is that not all tables are Lists: the expression `ls:map('#')` will return a _list_ of the lengths of any elements of `ls`. A list is a thin wrapper around a table, provided by the metatable `List`. Sometimes you may wish to work with ordinary Lua tables; the `List` interface is not a compulsory way to use Penlight table operations.
+
+### Operations on two-dimensional tables
+
+@lookup pl.array2d
+
+Two-dimensional tables are of course easy to represent in Lua, for instance `{{1,2},{3,4}}` where we store rows as subtables and index like so `A[col][row]`. This is the common representation used by matrix libraries like [LuaMatrix](http://lua-users.org/wiki/LuaMatrix). `pl.array2d` does not provide matrix operations, since that is the job for a specialized library, but rather provides generalizations of the higher-level operations provided by `pl.tablex` for one-dimensional arrays.
+
+`iter` is a useful generalization of `ipairs`. (The extra parameter determines whether you want the indices as well.)
+
+ > a = {{1,2},{3,4}}
+ > for i,j,v in array2d.iter(a,true) do print(i,j,v) end
+ 1 1 1
+ 1 2 2
+ 2 1 3
+ 2 2 4
+
+Note that you can always convert an arbitrary 2D array into a 'list of lists' with `List(tablex.map(List,a))`
+
+`map` will apply a function over all elements (notice that extra arguments can be provided, so this operation is in effect `function(x) return x-1 end`)
+
+ > array2d.map('-',a,1)
+ {{0,1},{2,3}}
+
+2D arrays are stored as an array of rows, but columns can be extracted:
+
+ > array2d.column(a,1)
+ {1,3}
+
+There are three equivalents to `tablex.reduce`. You can either reduce along the rows (which is the most efficient) or reduce along the columns. Either one will give you a 1D array. And `reduce2` will apply two operations: the first one reduces the rows, and the second reduces the result.
+
+ > array2d.reduce_rows('+',a)
+ {3,7}
+ > array2d.reduce_cols('+',a)
+ {4,6}
+ > -- same as tablex.reduce('*',array.reduce_rows('+',a))
+ > array2d.reduce2('*','+',a)
+ 21 `
+
+`tablex.map2` applies an operation to two tables, giving another table. `array2d.map2` does this for 2D arrays. Note that you have to provide the _rank_ of the arrays involved, since it's hard to always correctly deduce this from the data:
+
+ > b = {{10,20},{30,40}}
+ > a = {{1,2},{3,4}}
+ > = array2d.map2('+',2,2,a,b) -- two 2D arrays
+ {{11,22},{33,44}}
+ > = array2d.map2('+',1,2,{10,100},a) -- 1D, 2D
+ {{11,102},{13,104}}
+ > = array2d.map2('*',2,1,a,{1,-1}) -- 2D, 1D
+ {{1,-2},{3,-4}}
+
+Of course, you are not limited to simple arithmetic. Say we have a 2D array of strings, and wish to print it out with proper right justification. The first step is to create all the string lengths by mapping `string.len` over the array, the second is to reduce this along the columns using `math.max` to get maximum column widths, and last, apply `stringx.rjust` with these widths.
+
+ maxlens = reduce_cols(math.max,map('#',lines))
+ lines = map2(stringx.rjust,2,1,lines,maxlens)
+
+There is `product` which returns the _Cartesian product_ of two 1D arrays. The result is a 2D array formed from applying the function to all possible pairs from the two arrays.
+
+ > array2d.product('{}',{1,2},{'a','b'})
+ {{{1,'b'},{2,'a'}},{{1,'a'},{2,'b'}}}
+
+There is a set of operations which work in-place on 2D arrays. You can `swap_rows` and `swap_cols`; the first really is a simple one-liner, but the idea here is to give the operation a name. `remove_row` and `remove_col` are generalizations of `table.remove`. Likewise, `extract_rows` and `extract_cols` are given arrays of indices and discard anything else. So, for instance, `extract_cols(A,{2,4})` will leave just columns 2 and 4 in the array.
+
+`List.slice` is often useful on 1D arrays; `slice` does the same thing, but is generally given a start (row,column) and a end (row,column).
+
+ > A = {{1,2,3},{4,5,6},{7,8,9}}
+ > B = slice(A,1,1,2,2)
+ > write(B)
+ 1 2
+ 4 5
+ > B = slice(A,2,2)
+ > write(B,nil,'%4.1f')
+ 5.0 6.0
+ 8.0 9.0
+
+Here `write` is used to print out an array nicely; the second parameter is `nil`, which is the default (stdout) but can be any file object and the third parameter is an optional format (as used in `string.format`).
+
+`parse_range` will take a spreadsheet range like 'A1:B2' or 'R1C1:R2C2' and return the range as four numbers, which can be passed to `slice`. The rule is that `slice` will return an array of the appropriate shape depending on the range; if a range represents a row or a column, the result is 1D, otherwise 2D.
+
+This applies to `iter` as well, which can also optionally be given a range:
+
+
+ > for i,j,v in iter(A,true,2,2) do print(i,j,v) end
+ 2 2 5
+ 2 3 6
+ 3 2 8
+ 3 3 9
+
+`new` will construct a new 2D array with the given dimensions. You provide an initial value for the elements, which is interpreted as a function if it's callable. With `L` being `utils.string_lambda` we then have the following way to make an _identity matrix_:
+
+ asserteq(
+ array.new(3,3,L'|i,j| i==j and 1 or 0'),
+ {{1,0,0},{0,1,0},{0,0,1}}
+ )
+
+Please note that most functions in `array2d` are _covariant_, that is, they return an array of the same type as they receive. In particular, any objects created with `data.new` or `matrix.new` will remain data or matrix objects when reshaped or sliced, etc. Data objects have the `array2d` functions available as methods.
+
+
--- /dev/null
+## Strings. Higher-level operations on strings.
+
+### Extra String Methods
+
+@lookup pl.stringx
+
+These are convenient borrowings from Python, as described in 3.6.1 of the Python reference, but note that indices in Lua always begin at one. `stringx` defines functions like `isalpha` and `isdigit`, which return `true` if s is only composed of letters or digits respectively. `startswith` and `endswith` are convenient ways to find substrings. (`endswith` works as in Python 2.5, so that `f:endswith {'.bat','.exe','.cmd'}` will be true for any filename which ends with these extensions.) There are justify methods and whitespace trimming functions like `strip`.
+
+ > stringx.import()
+ > ('bonzo.dog'):endswith {'.dog','.cat'}
+ true
+ > ('bonzo.txt'):endswith {'.dog','.cat'}
+ false
+ > ('bonzo.cat'):endswith {'.dog','.cat'}
+ true
+ > (' stuff'):ljust(20,'+')
+ '++++++++++++++ stuff'
+ > (' stuff '):lstrip()
+ 'stuff '
+ > (' stuff '):rstrip()
+ ' stuff'
+ > (' stuff '):strip()
+ 'stuff'
+ > for s in ('one\ntwo\nthree\n'):lines() do print(s) end
+ one
+ two
+ three
+
+Most of these can be fairly easily implemented using the Lua string library, which is more general and powerful. But they are convenient operations to have easily at hand. Note that can be injected into the `string` table if you use `stringx.import`, but a simple alias like `local stringx = require 'pl.stringx'` is preferrable. This is the recommended practice when writing modules for consumption by other people, since it is bad manners to change the global state of the rest of the system.
+
+
+### String Templates
+
+@lookup pl.text
+
+Another borrowing from Python, string templates allow you to substitute values looked up in a table:
+
+ local Template = require ('pl.text').Template
+ t = Template('${here} is the $answer')
+ print(t:substitute {here = 'Lua', answer = 'best'})
+ ==>
+ Lua is the best
+
+'$ variables' can optionally have curly braces; this form is useful if you are glueing text together to make variables, e.g `${prefix}_name_${postfix}`. The `substitute` method will throw an error if a $ variable is not found in the table, and the `safe_substitute` method will not.
+
+The Lua implementation has an extra method, `indent_substitute` which is very useful for inserting blocks of text, because it adjusts indentation. Consider this example:
+
+ -- testtemplate.lua
+ local Template = require ('pl.text').Template
+
+ t = Template [[
+ for i = 1,#$t do
+ $body
+ end
+ ]]
+
+ body = Template [[
+ local row = $t[i]
+ for j = 1,#row do
+ fun(row[j])
+ end
+ ]]
+
+ print(t:indent_substitute {body=body,t='tbl'})
+
+And the output is:
+
+ for i = 1,#tbl do
+ local row = tbl[i]
+ for j = 1,#row do
+ fun(row[j])
+ end
+ end
+
+`indent_substitute` can substitute templates, and in which case they themselves will be substituted using the given table. So in this case, `$t` was substituted twice.
+
+`pl.text` also has a number of useful functions like `dedent`, which strips all the initial indentation from a multiline string. As in Python, this is useful for preprocessing multiline strings if you like indenting them with your code. The function `wrap` is passed a long string (a _paragraph_) and returns a list of lines that fit into a desired line width. As an extension, there is also `indent` for indenting multiline strings.
+
+New in Penlight with the 0.9 series is `text.format_operator`. Calling this enables Python-style string formating using the modulo operator `%`:
+
+ > text.format_operator()
+ > = '%s[%d]' % {'dog',1}
+ dog[1]
+
+So in its simplest form it saves the typing involved with `string.format`; it will also expand `$` variables using named fields:
+
+ > = '$animal[$num]' % {animal='dog',num=1}
+ dog[1]
+
+### Another Style of Template
+
+A new module is `template`, which is a version of Rici Lake's [Lua Preprocessor](http://lua-users.org/wiki/SlightlyLessSimpleLuaPreprocessor). This allows you to mix Lua code with your templates in a straightforward way. There are only two rules:
+
+ - Lines begining with `#` are Lua
+ - Otherwise, anything inside `$()` is a Lua expression.
+
+So a template generating an HTML list would look like this:
+
+ <ul>
+ # for i,val in ipairs(T) do
+ <li>$(i) = $(val:upper())</li>
+ # end
+ </ul>
+
+Assume the text is inside `tmpl`, then the template can be expanded using:
+
+ local template = require 'pl.template'
+ res = template.substitute(tmpl,{T = {'one','two','three'}})
+
+and we get
+
+ <ul>
+ <li>1 = ONE</li>
+ <li>2 = TWO</li>
+ <li>3 = THREE</li>
+ </ul>
+
+There is a single function, `template.substitute` which is passed a template string and an environment table. This table may contain some special fields, like `_parent` which can be set to a table representing a 'fallback' environment in case a symbol was not found. `_brackets` is usually '()' and `_escape` is usually '#' but it's sometimes necessary to redefine these if the defaults interfere with the target language - for instance, `$(V)` has another meaning in Make, and `#` means a preprocessor line in C/C++.
+
+Finally, if something goes wrong, passing `_debug` will cause the intermediate Lua code to be dumped if there's a problem.
+
+Here is a C code generation example; something that could easily be extended to be a minimal Lua extension skeleton generator.
+
+ local subst = require 'pl.template'.substitute
+
+ local templ = [[
+ #include <lua.h>
+ #include <lauxlib.h>
+ #include <lualib.h>
+
+ > for _,f in ipairs(mod) do
+ static int l_$(f.name) (lua_State *L) {
+
+ }
+ > end
+
+ static const luaL_reg $(mod.name)[] = {
+ > for _,f in ipairs(mod) do
+ {"$(f.name)",l_$(f.name)},
+ > end
+ {NULL,NULL}
+ };
+
+ int luaopen_$(mod.name) {
+ luaL_register (L, "$(mod.name)", $(mod.name));
+ return 1;
+ }
+ ]]
+
+ print(subst(templ,{
+ _escape = '>',
+ ipairs = ipairs,
+ mod = {
+ name = 'baggins';
+ {name='frodo'},
+ {name='bilbo'}
+ }
+ }))
+
+
+### File-style I/O on Strings
+
+`pl.stringio` provides just three functions; `stringio.open` is passed a string, and returns a file-like object for reading. It supports a `read` method, which takes the same arguments as standard file objects:
+
+ > f = stringio.open 'first line\n10 20 30\n'
+ > = f:read()
+ first line
+ > = f:read('*n','*n','*n')
+ 10 20 30
+
+`lines` and `seek` are also supported.
+
+`stringio.lines` is a useful short-cut for iterating over all the lines in a string.
+
+`stringio.create` creates a writeable file-like object. You then use `write` to this stream, and finally extract the builded string using `value`. This 'string builder' pattern is useful for efficiently creating large strings.
+
--- /dev/null
+## Paths and Directories\r
+\r
+### Working with Paths\r
+\r
+Programs should not depend on quirks of your operating system. They will be harder to read, and need to be ported for other systems. The worst of course is hardcoding paths like 'c:\\' in programs, and wondering why Vista complains so much. But even something like `dir..'\\'..file` is a problem, since Unix can't understand backslashes in this way. `dir..'/'..file` is _usually_ portable, but it's best to put this all into a simple function, `path.join`. If you consistently use `path.join`, then it's much easier to write cross-platform code, since it handles the directory separator for you.\r
+\r
+`pl.path` provides the same functionality as Python's `os.path` module (11.1).\r
+\r
+ > p = 'c:\\bonzo\\DOG.txt'\r
+ > = path.normcase (p) ---> only makes sense on Windows\r
+ c:\bonzo\dog.txt\r
+ > = path.splitext (p)\r
+ c:\bonzo\DOG .txt\r
+ > = path.extension (p)\r
+ .txt\r
+ > = path.basename (p)\r
+ DOG.txt\r
+ > = path.exists(p)\r
+ false\r
+ > = path.join ('fred','alice.txt')\r
+ fred\alice.txt\r
+ > = path.exists 'pretty.lua'\r
+ true\r
+ > = path.getsize 'pretty.lua'\r
+ 2125\r
+ > = path.isfile 'pretty.lua'\r
+ true\r
+ > = path.isdir 'pretty.lua'\r
+ false\r
+\r
+\r
+It is very important for all programmers, not just on Unix, to only write to where they are allowed to write. `path.expanduser` will expand '~' (tilde) into the home directory. Depending on your OS, this will be a guaranteed place where you can create files:\r
+\r
+ > = path.expanduser '~/mydata.txt'\r
+ 'C:\Documents and Settings\SJDonova/mydata.txt'\r
+\r
+ > = path.expanduser '~/mydata.txt'\r
+ /home/sdonovan/mydata.txt\r
+\r
+Under Windows, `os.tmpname` returns a path which leads to your drive root full of temporary files. (And increasingly, you do not have access to this root folder.) This is corrected by `path.tmpname`, which uses the environment variable TMP:\r
+\r
+ > os.tmpname() -- not a good place to put temporary files!\r
+ '\s25g.'\r
+ > path.tmpname()\r
+ 'C:\DOCUME~1\SJDonova\LOCALS~1\Temp\s25g.1'\r
+\r
+\r
+A useful extra function is `pl.path.package_path`, which will tell you the path of a particular Lua module. So on my system, `package_path('pl.path')` returns 'C:\Program Files\Lua\5.1\lualibs\pl\path.lua', and `package_path('ifs')` returns 'C:\Program Files\Lua\5.1\clibs\lfs.dll'. It is implemented in terms of `package.searchpath`, which is a new function in Lua 5.2 which has been implemented for Lua 5.1 in Penlight.\r
+\r
+### File Operations\r
+\r
+`pl.file` is a new module that provides more sensible names for common file operations. For instance, `file.read` and `file.write` are aliases for `utils.readfile` and `utils.writefile`.\r
+\r
+Smaller files can be efficiently read and written in one operation. `file.read` is passed a filename and returns the contents as a string, if successful; if not, then it returns `nil` and the actual error message. There is an optional boolean parameter if you want the file to be read in binary mode (this makes no difference on Unix but remains important with Windows.)\r
+\r
+In previous versions of Penlight, `utils.readfile` would read standard input if the file was not specified, but this can lead to nasty bugs; use `io.read '*a'` to grab all of standard input.\r
+\r
+Similarly, `file.write` takes a filename and a string which will be written to that file.\r
+\r
+For example, this little script converts a file into upper case:\r
+\r
+ require 'pl'\r
+ assert(#arg == 2, 'supply two filenames')\r
+ text = assert(file.read(arg[1]))\r
+ assert(file.write(arg[2],text:upper()))\r
+\r
+Copying files is suprisingly tricky. `file.copy` and `file.move` attempt to use the best implementation possible. On Windows, they link to the API functions `CopyFile` and `MoveFile`, but only if the `alien` package is installed (this is true for Lua for Windows.) Otherwise, the system copy command is used. This can be ugly when writing Windows GUI applications, because of the dreaded flashing black-box problem with launching processes.\r
+\r
+### Directory Operations\r
+\r
+`pl.dir` provides some useful functions for working with directories. `fnmatch` will match a filename against a shell pattern, and `filter` will return any files in the supplied list which match the given pattern, which correspond to the functions in the Python `fnmatch` module. `getdirectories` will return all directories contained in a directory, and `getfiles` will return all files in a directory which match a shell pattern. These functions return the files as a table, unlike `lfs.dir` which returns an iterator.)\r
+\r
+`dir.makepath` can create a full path, creating subdirectories as necessary; `rmtree` is the Nuclear Option of file deleting functions, since it will recursively clear out and delete all directories found begining at a path (there is a similar function with this name in the Python `shutils` module.)\r
+\r
+ > = dir.makepath 't\\temp\\bonzo'\r
+ > = path.isdir 't\\temp\\bonzo'\r
+ true\r
+ > = dir.rmtree 't'\r
+\r
+`dir.rmtree` depends on `dir.walk`, which is a powerful tool for scanning a whole directory tree. Here is the implementation of `dir.rmtree`:\r
+\r
+ --- remove a whole directory tree.\r
+ -- @param path A directory path\r
+ function dir.rmtree(fullpath)\r
+ for root,dirs,files in dir.walk(fullpath) do\r
+ for i,f in ipairs(files) do\r
+ os.remove(path.join(root,f))\r
+ end\r
+ lfs.rmdir(root)\r
+ end\r
+ end\r
+\r
+\r
+`dir.clonetree` clones directory trees. The first argument is a path that must exist, and the second path is the path to be cloned. (Note that this path cannot be _inside_ the first path, since this leads to madness.) By default, it will then just recreate the directory structure. You can in addition provide a function, which will be applied for all files found.\r
+\r
+ -- make a copy of my libs folder\r
+ require 'pl'\r
+ p1 = [[d:\dev\lua\libs]]\r
+ p2 = [[D:\dev\lua\libs\..\tests]]\r
+ dir.clonetree(p1,p2,dir.copyfile)\r
+\r
+A more sophisticated version, which only copies files which have been modified:\r
+\r
+ -- p1 and p2 as before, or from arg[1] and arg[2]\r
+ dir.clonetree(p1,p2,function(f1,f2)\r
+ local res\r
+ local t1,t2 = path.getmtime(f1),path.getmtime(f2)\r
+ -- f2 might not exist, so be careful about t2\r
+ if not t2 or t1 > t2 then\r
+ res = dir.copyfile(f1,f2)\r
+ end\r
+ return res -- indicates successful operation\r
+ end)\r
+\r
+`dir.clonetree` uses `path.common_prefix`. With `p1` and `p2` defined above, the common path is 'd:\dev\lua'. So 'd:\dev\lua\libs\testfunc.lua' is copied to 'd:\dev\lua\test\testfunc.lua', etc.\r
+\r
+If you need to find the common path of list of files, then `tablex.reduce` will do the job:\r
+\r
+ > p3 = [[d:\dev]]\r
+ > = tablex.reduce(path.common_prefix,{p1,p2,p3})\r
+ 'd:\dev'\r
+\r
--- /dev/null
+## Date and Time\r
+\r
+<a id="date"></a>\r
+\r
+### Manipulating Dates\r
+\r
+The `Date` class provides a simplified way to work with [date and time](http://www.lua.org/pil/22.1.html) in Lua; it leans heavily on the functions `os.date` and `os.time`.\r
+\r
+A `Date` object can be constructed from a table, just like with `os.time`. Methods are provided to get and set the various parts of the date.\r
+\r
+ > d = Date {year = 2011, month = 3, day = 2 }\r
+ > = d\r
+ 2011-03-02 12:00:00\r
+ > = d:month(),d:year(),d:day()\r
+ 3 2011 2\r
+ > d:month(4)\r
+ > = d\r
+ 2011-04-02 12:00:00\r
+ > d:add {day=1}\r
+ > = d\r
+ 2011-04-03 12:00:00\r
+\r
+`add` takes a table containing one of the date table fields.\r
+\r
+ > = d:weekday_name()\r
+ Sun\r
+ > = d:last_day()\r
+ 2011-04-30 12:00:00\r
+ > = d:month_name(true)\r
+ April\r
+\r
+There is a default conversion to text for date objects, but `Date.Format` gives you full control of the format for both parsing and displaying dates:\r
+\r
+ > iso = Date.Format 'yyyy-mm-dd'\r
+ > d = iso:parse '2010-04-10'\r
+ > amer = Date.Format 'mm/dd/yyyy'\r
+ > = amer:tostring(d)\r
+ 04/10/2010\r
+\r
+With the 0.9.7 relase, the `Date` constructor has become more flexible. You may omit any of the 'year', 'month' or 'day' fields:\r
+\r
+ > = Date { year = 2008 }\r
+ 2008-01-01 12:00:00\r
+ > = Date { month = 3 }\r
+ 2011-03-01 12:00:00\r
+ > = Date { day = 20 }\r
+ 2011-10-20 12:00:00\r
+ > = Date { hour = 14, min = 30 }\r
+ 2011-10-13 14:30:00\r
+\r
+If 'year' is omitted, then the current year is assumed, and likewise for 'month'.\r
+\r
+To set the time on such a partial date, you can use the fact that the 'setter' methods return the date object and so you can 'chain' these methods.\r
+\r
+ > d = Date { day = 03 }\r
+ > = d:hour(18):min(30)\r
+ 2011-10-03 18:30:00\r
+\r
+Finally, `Date` also now accepts positional arguments:\r
+\r
+ > = Date(2011,10,3)\r
+ 2011-10-03 12:00:00\r
+ > = Date(2011,10,3,18,30,23)\r
+ 2011-10-03 18:30:23\r
+\r
+`Date.format` has been extended. If you construct an instance without a pattern, then it will try to match against a set of known formats. This is useful for human-input dates since keeping to a strict format is not one of the strong points of users. It assumes that there will be a date, and then a date.\r
+\r
+ > df = Date.Format()\r
+ > = df:parse '5.30pm'\r
+ 2011-10-13 17:30:00\r
+ > = df:parse '1730'\r
+ nil day out of range: 1730 is not between 1 and 31\r
+ > = df:parse '17.30'\r
+ 2011-10-13 17:30:00\r
+ > = df:parse 'mar'\r
+ 2011-03-01 12:00:00\r
+ > = df:parse '3 March'\r
+ 2011-03-03 12:00:00\r
+ > = df:parse '15 March'\r
+ 2011-03-15 12:00:00\r
+ > = df:parse '15 March 2008'\r
+ 2008-03-15 12:00:00\r
+ > = df:parse '15 March 2008 1.30pm'\r
+ 2008-03-15 13:30:00\r
+ > = df:parse '2008-10-03 15:30:23'\r
+ 2008-10-03 15:30:23\r
+\r
+ISO date format is of course a good idea if you need to deal with users from different countries. Here is the default behaviour for 'short' dates:\r
+\r
+ > = df:parse '24/02/12'\r
+ 2012-02-24 12:00:00\r
+\r
+That's not what Americans expect! It's tricky to work out in a cross-platform way exactly what the expected format is, so there is an explicit flag:\r
+\r
+ > df:US_order(true)\r
+ > = df:parse '9/11/01'\r
+ 2001-11-09 12:00:00\r
+\r
--- /dev/null
+## Data
+
+### Reading Data Files
+
+The first thing to consider is this: do you actually need to write a custom file reader? And if the answer is yes, the next question is: can you write the reader in as clear a way as possible? Correctness, Robustness, and Speed; pick the first two and the third can be sorted out later, _if necessary_.
+
+A common sort of data file is the configuration file format commonly used on Unix systems. This format is often called a _property_ file in the Java world.
+
+ # Read timeout in seconds
+ read.timeout=10
+
+ # Write timeout in seconds
+ write.timeout=10
+
+Here is a simple Lua implementation:
+
+ -- property file parsing with Lua string patterns
+ props = []
+ for line in io.lines() do
+ if line:find('#,1,true) ~= 1 and not line:find('^%s*$') then
+ local var,value = line:match('([^=]+)=(.*)')
+ props[var] = value
+ end
+ end
+
+Very compact, but it suffers from a similar disease in equivalent Perl programs; it uses odd string patterns which are 'lexically noisy'. Noisy code like this slows the casual reader down. (For an even more direct way of doing this, see the next section, 'Reading Configuration Files')
+
+Another implementation, using the Penlight libraries:
+
+ -- property file parsing with extended string functions
+ require 'pl'
+ stringx.import()
+ props = []
+ for line in io.lines() do
+ if not line:startswith('#') and not line:isspace() then
+ local var,value = line:splitv('=')
+ props[var] = value
+ end
+ end
+
+This is more self-documenting; it is generally better to make the code express the _intention_, rather than having to scatter comments everywhere - comments are necessary, of course, but mostly to give the higher view of your intention that cannot be expressed in code. It is slightly slower, true, but in practice the speed of this script is determined by I/O, so further optimization is unnecessary.
+
+### Reading Unstructured Text Data
+
+Text data is sometimes unstructured, for example a file containing words. The `pl.input` module has a number of functions which makes processing such files easier. For example, a script to count the number of words in standard input using `import.words`:
+
+ -- countwords.lua
+ require 'pl'
+ local k = 1
+ for w in input.words(io.stdin) do
+ k = k + 1
+ end
+ print('count',k)
+
+Or this script to calculate the average of a set of numbers using `input.numbers`:
+
+ -- average.lua
+ require 'pl'
+ local k = 1
+ local sum = 0
+ for n in input.numbers(io.stdin) do
+ sum = sum + n
+ k = k + 1
+ end
+ print('average',sum/k)
+
+These scripts can be improved further by _eliminating loops_ In the last case, there is a perfectly good function `seq.sum` which can already take a sequence of numbers and calculate these numbers for us:
+
+ -- average2.lua
+ require 'pl'
+ local total,n = seq.sum(input.numbers())
+ print('average',total/n)
+
+A further simplification here is that if `numbers` or `words` are not passed an argument, they will grab their input from standard input. The first script can be rewritten:
+
+ -- countwords2.lua
+ require 'pl'
+ print('count',seq.count(input.words()))
+
+A useful feature of a sequence generator like `numbers` is that it can read from a string source. Here is a script to calculate the sums of the numbers on each line in a file:
+
+ -- sums.lua
+ for line in io.lines() do
+ print(seq.sum(input.numbers(line))
+ end
+
+### Reading Columnar Data
+
+It is very common to find data in columnar form, either space or comma-separated, perhaps with an initial set of column headers. Here is a typical example:
+
+ EventID Magnitude LocationX LocationY LocationZ
+ 981124001 2.0 18988.4 10047.1 4149.7
+ 981125001 0.8 19104.0 9970.4 5088.7
+ 981127003 0.5 19012.5 9946.9 3831.2
+ ...
+
+`input.fields` is designed to extract several columns, given some delimiter (default to whitespace). Here is a script to calculate the average X location of all the events:
+
+ -- avg-x.lua
+ require 'pl'
+ io.read() -- skip the header line
+ local sum,count = seq.sum(input.fields {3})
+ print(sum/count)
+
+`input.fields` is passed either a field count, or a list of column indices, starting at one as usual. So in this case we're only interested in column 3. If you pass it a field count, then you get every field up to that count:
+
+ for id,mag,locX,locY,locZ in input.fields (5) do
+ ....
+ end
+
+`input.fields` by default tries to convert each field to a number. It will skip lines which clearly don't match the pattern, but will abort the script if there are any fields which cannot be converted to numbers.
+
+The second parameter is a delimiter, by default spaces. ' ' is understood to mean 'any number of spaces', i.e. '%s+'. Any Lua string pattern can be used.
+
+The third parameter is a _data source_, by default standard input (defined by `input.create_getter`.) It assumes that the data source has a `read` method which brings in the next line, i.e. it is a 'file-like' object. As a special case, a string will be split into its lines:
+
+ > for x,y in input.fields(2,' ','10 20\n30 40\n') do print(x,y) end
+ 10 20
+ 30 40
+
+Note the default behaviour for bad fields, which is to show the offending line number:
+
+ > for x,y in input.fields(2,' ','10 20\n30 40x\n') do print(x,y) end
+ 10 20
+ line 2: cannot convert '40x' to number
+
+This behaviour of `input.fields` is appropriate for a script which you want to fail immediately with an appropriate _user_ error message if conversion fails. The fourth optional parameter is an options table: `{no_fail=true}` means that conversion is attempted but if it fails it just returns the string, rather as AWK would operate. You are then responsible for checking the type of the returned field. `{no_convert=true}` switches off conversion altogether and all fields are returned as strings.
+
+@lookup pl.data
+
+Sometimes it is useful to bring a whole dataset into memory, for operations such as extracting columns. Penlight provides a flexible reader specifically for reading this kind of data, using the `data` module. Given a file looking like this:
+
+ x,y
+ 10,20
+ 2,5
+ 40,50
+
+Then `data.read` will create a table like this, with each row represented by a sublist:
+
+ > t = data.read 'test.txt'
+ > pretty.dump(t)
+ {{10,20},{2,5},{40,50},fieldnames={'x','y'},delim=','}
+
+You can now analyze this returned table using the supplied methods. For instance, the method `column_by_name` returns a table of all the values of that column.
+
+ -- testdata.lua
+ require 'pl'
+ d = data.read('fev.txt')
+ for _,name in ipairs(d.fieldnames) do
+ local col = d:column_by_name(name)
+ if type(col[1]) == 'number' then
+ local total,n = seq.sum(col)
+ utils.printf("Average for %s is %f\n",name,total/n)
+ end
+ end
+
+`data.read` tries to be clever when given data; by default it expects a first line of column names, unless any of them are numbers. It tries to deduce the column delimiter by looking at the first line. Sometimes it guesses wrong; these things can be specified explicitly. The second optional parameter is an options table: can override `delim` (a string pattern), `fieldnames` (a list or comma-separated string), specify `no_convert` (default is to convert), numfields (indices of columns known to be numbers, as a list) and `thousands_dot` (when the thousands separator in Excel CSV is '.')
+
+A very powerful feature is a way to execute SQL-like queries on such data:
+
+ -- queries on tabular data
+ require 'pl'
+ local d = data.read('xyz.txt')
+ local q = d:select('x,y,z where x > 3 and z < 2 sort by y')
+ for x,y,z in q do
+ print(x,y,z)
+ end
+
+Please note that the format of queries is restricted to the following syntax:
+
+ FIELDLIST [ 'where' CONDITION ] [ 'sort by' FIELD [asc|desc]]
+
+Any valid Lua code can appear in `CONDITION`; remember it is _not_ SQL and you have to use `==` (this warning comes from experience.)
+
+For this to work, _field names must be Lua identifiers_. So `read` will massage fieldnames so that all non-alphanumeric chars are replaced with underscores.
+
+`read` can handle standard CSV files fine, although doesn't try to be a full-blown CSV parser. Spreadsheet programs are not always the best tool to process such data, strange as this might seem to some people. This is a toy CSV file; to appreciate the problem, imagine thousands of rows and dozens of columns like this:
+
+ Department Name,Employee ID,Project,Hours Booked
+ sales,1231,overhead,4
+ sales,1255,overhead,3
+ engineering,1501,development,5
+ engineering,1501,maintenance,3
+ engineering,1433,maintenance,10
+
+The task is to reduce the dataset to a relevant set of rows and columns, perhaps do some processing on row data, and write the result out to a new CSV file. The `write_row` method uses the delimiter to write the row to a file; `Data.select_row` is like `Data.select`, except it iterates over _rows_, not fields; this is necessary if we are dealing with a lot of columns!
+
+ names = {[1501]='don',[1433]='dilbert'}
+ keepcols = {'Employee_ID','Hours_Booked'}
+ t:write_row (outf,{'Employee','Hours_Booked'})
+ q = t:select_row {
+ fields=keepcols,
+ where=function(row) return row[1]=='engineering' end
+ }
+ for row in q do
+ row[1] = names[row[1]]
+ t:write_row(outf,row)
+ end
+
+`Data.select_row` and `Data.select` can be passed a table specifying the query; a list of field names, a function defining the condition and an optional parameter `sort_by`. It isn't really necessary here, but if we had a more complicated row condition (such as belonging to a specified set) then it is not generally possible to express such a condition as a query string, without resorting to hackery such as global variables.
+
+Data does not have to come from files, nor does it necessarily come from the lab or the accounts department. On Linux, `ps aux` gives you a full listing of all processes running on your machine. It is straightforward to feed the output of this command into `data.read` and perform useful queries on it. Notice that non-identifier characters like '%' get converted into underscores:
+
+ require 'pl'
+ f = io.popen 'ps aux'
+ s = data.read (f,{last_field_collect=true})
+ f:close()
+ print(s.fieldnames)
+ print(s:column_by_name 'USER')
+ qs = 'COMMAND,_MEM where _MEM > 5 and USER=="steve"'
+ for name,mem in s:select(qs) do
+ print(mem,name)
+ end
+
+I've always been an admirer of the AWK programming language; with `filter` you can get Lua programs which are just as compact:
+
+ -- printxy.lua
+ require 'pl'
+ data.filter 'x,y where x > 3'
+
+It is common enough to have data files without headers of field names. `data.read` makes a special exception for such files if all fields are numeric. Since there are no column names to use in query expressions, you can use AWK-like column indexes, e.g. '$1,$2 where $1 > 3'. I have a little executable script on my system called `lf` which looks like this:
+
+ #!/usr/bin/env lua
+ require 'pl.data'.filter(arg[1])
+
+And it can be used generally as a filter command to extract columns from data. (The column specifications may be expressions or even constants.)
+
+ $ lf '$1,$5/10' < test.dat
+
+(As with AWK, please note the single-quotes used in this command; this prevents the shell trying to expand the column indexes. If you are on Windows, then you are fine, but it is still necessary to quote the expression in double-quotes so it is passed as one argument to your batch file.)
+
+As a tutorial resource, have a look at `test-data.lua` in the PL tests directory for other examples of use, plus comments.
+
+The data returned by `read` or constructed by `Data.copy_select` from a query is basically just an array of rows: `{{1,2},{3,4}}`. So you may use `read` to pull in any array-like dataset, and process with any function that expects such a implementation. In particular, the functions in `array2d` will work fine with this data. In fact, these functions are available as methods; e.g. `array2d.flatten` can be called directly like so to give us a one-dimensional list:
+
+ v = data.read('dat.txt'):flatten()
+
+The data is also in exactly the right shape to be treated as matrices by [LuaMatrix](http://lua-users.org/wiki/LuaMatrix):
+
+ > matrix = require 'matrix'
+ > m = matrix(data.read 'mat.txt')
+ > = m
+ 1 0.2 0.3
+ 0.2 1 0.1
+ 0.1 0.2 1
+ > = m^2 -- same as m*m
+ 1.07 0.46 0.62
+ 0.41 1.06 0.26
+ 0.24 0.42 1.05
+
+`write` will write matrices back to files for you.
+
+Finally, for the curious, the global variable `_DEBUG` can be used to print out the actual iterator function which a query generates and dynamically compiles. By using code generation, we can get pretty much optimal performance out of arbitrary queries.
+
+ > lua -lpl -e "_DEBUG=true" -e "data.filter 'x,y where x > 4 sort by x'" < test.txt
+ return function (t)
+ local i = 0
+ local v
+ local ls = {}
+ for i,v in ipairs(t) do
+ if v[1] > 4 then
+ ls[#ls+1] = v
+ end
+ end
+ table.sort(ls,function(v1,v2)
+ return v1[1] < v2[1]
+ end)
+ local n = #ls
+ return function()
+ i = i + 1
+ v = ls[i]
+ if i > n then return end
+ return v[1],v[2]
+ end
+ end
+
+ 10,20
+ 40,50
+
+### Reading Configuration Files
+
+The `config` module provides a simple way to convert several kinds of configuration files into a Lua table. Consider the simple example:
+
+ # test.config
+ # Read timeout in seconds
+ read.timeout=10
+
+ # Write timeout in seconds
+ write.timeout=5
+
+ #acceptable ports
+ ports = 1002,1003,1004
+
+This can be easily brought in using `config.read` and the result shown using `pretty.write`:
+
+ -- readconfig.lua
+ local config = require 'pl.config'
+ local pretty= require 'pl.pretty'
+
+ local t = config.read(arg[1])
+ print(pretty.write(t))
+
+and the output of `lua readconfig.lua test.config` is:
+
+ {
+ ports = {
+ 1002,
+ 1003,
+ 1004
+ },
+ write_timeout = 5,
+ read_timeout = 10
+ }
+
+That is, `config.read` will bring in all key/value pairs, ignore # comments, and ensure that the key names are proper Lua identifiers by replacing non-identifier characters with '_'. If the values are numbers, then they will be converted. (So the value of `t.write_timeout` is the number 5). In addition, any values which are separated by commas will be converted likewise into an array.
+
+Any line can be continued with a backslash. So this will all be considered one line:
+
+ names=one,two,three, \
+ four,five,six,seven, \
+ eight,nine,ten
+
+
+Windows-style INI files are also supported. The section structure of INI files translates naturally to nested tables in Lua:
+
+ ; test.ini
+ [timeouts]
+ read=10 ; Read timeout in seconds
+ write=5 ; Write timeout in seconds
+ [portinfo]
+ ports = 1002,1003,1004
+
+ The output is:
+
+ {
+ portinfo = {
+ ports = {
+ 1002,
+ 1003,
+ 1004
+ }
+ },
+ timeouts = {
+ write = 5,
+ read = 10
+ }
+ }
+
+You can now refer to the write timeout as `t.timeouts.write`.
+
+As a final example of the flexibility of `config.read`, if passed this simple comma-delimited file
+
+ one,two,three
+ 10,20,30
+ 40,50,60
+ 1,2,3
+
+it will produce the following table:
+
+ {
+ { "one", "two", "three" },
+ { 10, 20, 30 },
+ { 40, 50, 60 },
+ { 1, 2, 3 }
+ }
+
+`config.read` isn't designed to read all CSV files in general, but intended to support some Unix configuration files not structured as key-value pairs, such as '/etc/passwd'.
+
+This function is intended to be a Swiss Army Knife of configuration readers, but it does have to make assumptions, and you may not like them. So there is an optional extra parameter which allows some control, which is table that may have the following fields:
+
+ {
+ variablilize = true,
+ convert_numbers = true,
+ trim_space = true,
+ list_delim = ','
+ }
+
+`variablilize` is the option that converted `write.timeout` in the first example to the valid Lua identifier `write_timeout`. If `convert_numbers` is true, then an attempt is made to convert any string that starts like a number. `trim_space` ensures that there is no starting or trailing whitespace with values, and `list_delim` is the character that will be used to decide whether to split a value up into a list (it may be a Lua string pattern such as '%s+'.)
+
+For instance, the password file in Unix is colon-delimited:
+
+ t = config.read('/etc/passwd',{list_delim=':'})
+
+This produces the following output on my system (only last two lines shown):
+
+ {
+ ...
+ {
+ "user",
+ "x",
+ "1000",
+ "1000",
+ "user,,,",
+ "/home/user",
+ "/bin/bash"
+ },
+ {
+ "sdonovan",
+ "x",
+ "1001",
+ "1001",
+ "steve donovan,28,,",
+ "/home/sdonovan",
+ "/bin/bash"
+ }
+ }
+
+You can get this into a more sensible format, where the usernames are the keys, with:
+
+ t = tablex.pairmap(function(k,v) return v,v[1] end,t)
+
+and you get:
+
+ { ...
+ sdonovan = {
+ "sdonovan",
+ "x",
+ "1001",
+ "1001",
+ "steve donovan,28,,",
+ "/home/sdonovan",
+ "/bin/bash"
+ }
+ ...
+ }
+
+
+<a id="lexer"/>
+
+### Lexical Scanning
+
+Although Lua's string pattern matching is very powerful, there are times when something more powerful is needed. `pl.lexer.scan` provides lexical scanners which _tokenizes_ a string, classifying tokens into numbers, strings, etc.
+
+ > lua -lpl
+ Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
+ > tok = lexer.scan 'alpha = sin(1.5)'
+ > = tok()
+ iden alpha
+ > = tok()
+ = =
+ > = tok()
+ iden sin
+ > = tok()
+ ( (
+ > = tok()
+ number 1.5
+ > = tok()
+ ) )
+ > = tok()
+ (nil)
+
+The scanner is a function, which is repeatedly called and returns the _type_ and _value_ of the token. Recognized basic types are 'iden','string','number', and 'space'. and everything else is represented by itself. Note that by default the scanner will skip any 'space' tokens.
+
+'comment' and 'keyword' aren't applicable to the plain scanner, which is not language-specific, but a scanner which understands Lua is available. It recognizes the Lua keywords, and understands both short and long comments and strings.
+
+ > for t,v in lexer.lua 'for i=1,n do' do print(t,v) end
+ keyword for
+ iden i
+ = =
+ number 1
+ , ,
+ iden n
+ keyword do
+
+A lexical scanner is useful where you have highly-structured data which is not nicely delimited by newlines. For example, here is a snippet of a in-house file format which it was my task to maintain:
+
+ points (818344.1,-20389.7,-0.1),(818337.9,-20389.3,-0.1),(818332.5,-20387.8,-0.1)
+ ,(818327.4,-20388,-0.1),(818322,-20387.7,-0.1),(818316.3,-20388.6,-0.1)
+ ,(818309.7,-20389.4,-0.1),(818303.5,-20390.6,-0.1),(818295.8,-20388.3,-0.1)
+ ,(818290.5,-20386.9,-0.1),(818285.2,-20386.1,-0.1),(818279.3,-20383.6,-0.1)
+ ,(818274,-20381.2,-0.1),(818274,-20380.7,-0.1);
+
+Here is code to extract the points using `pl.lexer`:
+
+ -- assume 's' contains the text above...
+ local lexer = require 'pl.lexer'
+ local expecting = lexer.expecting
+ local append = table.insert
+
+ local tok = lexer.scan(s)
+
+ local points = {}
+ local t,v = tok() -- should be 'iden','points'
+
+ while t ~= ';' do
+ c = {}
+ expecting(tok,'(')
+ c.x = expecting(tok,'number')
+ expecting(tok,',')
+ c.y = expecting(tok,'number')
+ expecting(tok,',')
+ c.z = expecting(tok,'number')
+ expecting(tok,')')
+ t,v = tok() -- either ',' or ';'
+ append(points,c)
+ end
+
+The `expecting` function grabs the next token and if the type doesn't match, it throws an error. (`pl.lexer`, unlike other PL libraries, raises errors if something goes wrong, so you should wrap your code in `pcall` to catch the error gracefully.)
+
+The scanners all have a second optional argument, which is a table which controls whether you want to exclude spaces and/or comments. The default for `lexer.lua` is `{space=true,comments=true}`. There is a third optional argument which determines how string and number tokens are to be processsed.
+
+The ultimate highly-structured data is of course, program source. Here is a snippet from 'text-lexer.lua':
+
+ require 'pl'
+
+ lines = [[
+ for k,v in pairs(t) do
+ if type(k) == 'number' then
+ print(v) -- array-like case
+ else
+ print(k,v)
+ end
+ end
+ ]]
+
+ ls = List()
+ for tp,val in lexer.lua(lines,{space=true,comments=true}) do
+ assert(tp ~= 'space' and tp ~= 'comment')
+ if tp == 'keyword' then ls:append(val) end
+ end
+ test.asserteq(ls,List{'for','in','do','if','then','else','end','end'})
+
+Here is a useful little utility that identifies all common global variables found in a lua module:
+
+ -- testglobal.lua
+ require 'pl'
+
+ local txt,err = utils.readfile(arg[1])
+ if not txt then return print(err) end
+
+ local globals = List()
+ for t,v in lexer.lua(txt) do
+ if t == 'iden' and _G[v] then
+ globals:append(v)
+ end
+ end
+ pretty.dump(seq.count_map(globals))
+
+Rather then dumping the whole list, with its duplicates, we pass it through `seq.count_map` which turns the list into a table where the keys are the values, and the associated values are the number of times those values occur in the sequence. Typical output looks like this:
+
+ {
+ type = 2,
+ pairs = 2,
+ table = 2,
+ print = 3,
+ tostring = 2,
+ require = 1,
+ ipairs = 4
+ }
+
+You could further pass this through `tablex.keys` to get a unique list of symbols. This can be useful when writing 'strict' Lua modules, where all global symbols must be defined as locals at the top of the file.
+
+For a more detailed use of `lexer.scan`, please look at `testxml.lua` in the examples directory.
+
+### XML
+
+New in the 0.9.7 release is some support for XML. This is a large topic, and Penlight does not provide a full XML stack, which is properly the task of a more specialized library.
+
+#### Parsing and Pretty-Printing
+
+The semi-standard XML parser in the Lua universe is [lua-expat](). In particular, it has a function called `lxp.lom.parse` which will parse XML into the Lua Object Model (LOM) format. However, it does not provide a way to convert this data back into XML text. `xml.parse` will use this function, _if_ `lua-expat` is available, and otherwise switches back to a pure Lua parser originally written by Roberto Ierusalimschy.
+
+The resulting document object knows how to render itself as a string, which is useful for debugging:
+
+ > d = xml.parse "<nodes><node id='1'>alice</node></nodes>"
+ > = d
+ <nodes><node id='1'>alice</node></nodes>
+ > pretty.dump (d)
+ {
+ {
+ "alice",
+ attr = {
+ "id",
+ id = "1"
+ },
+ tag = "node"
+ },
+ attr = {
+ },
+ tag = "nodes"
+ }
+
+Looking at the actual shape of the data reveals the structure of LOM:
+
+ * every element has a `tag` field with its name
+ * plus a `attr` field which is a table containing the attributes as fields, and also as an array. It is always present.
+ * the children of the element are the array part of the element, so `d[1]` is the first child of `d`, etc.
+
+It could be argued that having attributes also as the array part of `attr` is not essential (you generally cannot depend on attribute order in XML) but that's how it goes with this standard.
+
+`lua-expat` is another _soft dependency_ of Penlight; generally, the fallback parser is good enough for straightforward XML as is commonly found in configuration files, etc. `doc.basic_parse` is not intended to be a proper conforming parser (it's only sixty lines) but it handles simple kinds of documents that do not have comments or DTD directives. It is intelligent enough to ignore the `<?xml` directive and that is about it.
+
+You can get pretty-printing by explicitly calling `xml.tostring` and passing it the initial indent and the per-element indent:
+
+ > = xml.tostring(d,'',' ')
+
+ <nodes>
+ <node id='1'>alice</node>
+ </nodes>
+
+There is a fourth argument which is the _attribute indent_:
+
+ > a = xml.parse "<frodo name='baggins' age='50' type='hobbit'/>"
+ > = xml.tostring(a,'',' ',' ')
+
+ <frodo
+ type='hobbit'
+ name='baggins'
+ age='50'
+ />
+
+#### Parsing and Working with Configuration Files
+
+It's common to find configurations expressed with XML these days. It's straightforward to 'walk' the LOM data and extract the data in the form you want:
+
+ require 'pl'
+
+ local config = [[
+ <config>
+ <alpha>1.3</alpha>
+ <beta>10</beta>
+ <name>bozo</name>
+ </config>
+ ]]
+ local d,err = xml.parse(config)
+
+ local t = {}
+ for item in d:childtags() do
+ t[item.tag] = item[1]
+ end
+
+ pretty.dump(t)
+ --->
+ {
+ beta = "10",
+ alpha = "1.3",
+ name = "bozo"
+ }
+
+The only gotcha is that here we must use the `Doc:childtags` method, which will skip over any text elements.
+
+A more involved example is this excerpt from `serviceproviders.xml`, which is usually found at `/usr/share/mobile-broadband-provider-info/serviceproviders.xml` on Debian/Ubuntu Linux systems.
+
+ d = xml.parse [[
+ <serviceproviders format="2.0">
+ <country code="za">
+ <provider>
+ <name>Cell-c</name>
+ <gsm>
+ <network-id mcc="655" mnc="07"/>
+ <apn value="internet">
+ <username>Cellcis</username>
+ <dns>196.7.0.138</dns>
+ <dns>196.7.142.132</dns>
+ </apn>
+ </gsm>
+ </provider>
+ <provider>
+ <name>MTN</name>
+ <gsm>
+ <network-id mcc="655" mnc="10"/>
+ <apn value="internet">
+ <dns>196.11.240.241</dns>
+ <dns>209.212.97.1</dns>
+ </apn>
+ </gsm>
+ </provider>
+ <provider>
+ <name>Vodacom</name>
+ <gsm>
+ <network-id mcc="655" mnc="01"/>
+ <apn value="internet">
+ <dns>196.207.40.165</dns>
+ <dns>196.43.46.190</dns>
+ </apn>
+ <apn value="unrestricted">
+ <name>Unrestricted</name>
+ <dns>196.207.32.69</dns>
+ <dns>196.43.45.190</dns>
+ </apn>
+ </gsm>
+ </provider>
+ <provider>
+ <name>Virgin Mobile</name>
+ <gsm>
+ <apn value="vdata">
+ <dns>196.7.0.138</dns>
+ <dns>196.7.142.132</dns>
+ </apn>
+ </gsm>
+ </provider>
+ </country>
+
+ </serviceproviders>
+ ]]
+
+Getting the names of the providers per-country is straightforward:
+
+ local t = {}
+ for country in d:childtags() do
+ local providers = {}
+ t[country.tag] = providers
+ for provider in country:childtags() do
+ table.insert(providers,provider:child_with_name('name'):get_text())
+ end
+ end
+
+ pretty.dump(t)
+ -->
+ {
+ country = {
+ "Cell-c",
+ "MTN",
+ "Vodacom",
+ "Virgin Mobile"
+ }
+ }
+
+#### Generating XML with 'xmlification'
+
+This feature is inspired by the `htmlify` function used by [Orbit](http://keplerproject.github.com/orbit/) to simplify HTML generation, except that no function environment magic is used; the `tags` function returns a set of _constructors_ for elements of the given tag names.
+
+ > nodes, node = xml.tags 'nodes, node'
+ > = node 'alice'
+ <node>alice</node>
+ > = nodes { node {id='1','alice'}}
+ <nodes><node id='1'>alice</node></nodes>
+
+The flexibility of Lua tables is very useful here, since both the attributes and the children of an element can be encoded naturally. The argument to these tag constructors is either a single value (like a string) or a table where the attributes are the named keys and the children are the array values.
+
+#### Generating XML using Templates
+
+A template is a little XML document which contains dollar-variables. The `subst` method on a document is fed an array of tables containing values for these variables. Note how the parent tag name is specified:
+
+ > templ = xml.parse "<node id='$id'>$name</node>"
+ > = templ:subst {tag='nodes', {id=1,name='alice'},{id=2,name='john'}}
+ <nodes><node id='1'>alice</node><node id='2'>john</node></nodes>
+
+#### Extracting Data using Templates
+
+Matching goes in the opposite direction. We have a document, and would like to extract values from it using a pattern.
+
+A common use of this is parsing the XML result of API queries. The [(undocumented) Google Weather API](http://blog.programmableweb.com/2010/02/08/googles-secret-weather-api/) is a good example. Grabbing the result of `http://www.google.com/ig/api?weather=Johannesburg,ZA" we get something like this, after pretty-printing:
+
+ <xml_api_reply version='1'>
+ <weather module_id='0' tab_id='0' mobile_zipped='1' section='0' row='0' mobile_row='0'>
+ <forecast_information>
+ <city data='Johannesburg, Gauteng'/>
+ <postal_code data='Johannesburg,ZA'/>
+ <latitude_e6 data=''/>
+ <longitude_e6 data=''/>
+ <forecast_date data='2010-10-02'/>
+ <current_date_time data='2010-10-02 18:30:00 +0000'/>
+ <unit_system data='US'/>
+ </forecast_information>
+ <current_conditions>
+ <condition data='Clear'/>
+ <temp_f data='75'/>
+ <temp_c data='24'/>
+ <humidity data='Humidity: 19%'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <wind_condition data='Wind: NW at 7 mph'/>
+ </current_conditions>
+ <forecast_conditions>
+ <day_of_week data='Sat'/>
+ <low data='60'/>
+ <high data='89'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <condition data='Clear'/>
+ </forecast_conditions>
+ ....
+ </weather>
+ </xml_api_reply>
+
+Assume that the above XML has been read into `google`. The idea is to write a pattern looking like a template, and use it to extract some values of interest:
+
+ t = [[
+ <weather>
+ <current_conditions>
+ <condition data='$condition'/>
+ <temp_c data='$temp'/>
+ </current_conditions>
+ </weather>
+ ]]
+
+ local res, ret = google:match(t)
+ pretty.dump(res)
+
+And the output is:
+
+ {
+ condition = "Clear",
+ temp = "24"
+ }
+
+The `match` method can be passed a LOM document or some text, which will be parsed first. Note that `$NUMBER` is treated specially as a numerical index, so that `$1` is the first element of the resulting array, etc.
+
+
--- /dev/null
+## Functional Programming\r
+\r
+### Sequences\r
+\r
+@lookup pl.seq\r
+\r
+A Lua iterator (in its simplest form) is a function which can be repeatedly called to return a set of one or more values. The `for in` statement understands these iterators, and loops until the function returns `nil`. There are standard sequence adapters for tables in Lua (`ipairs` and `pairs`), and `io.lines` returns an iterator over all the lines in a file. In the Penlight libraries, such iterators are also called _sequences_. A sequence of single values (say from `io.lines`) is called _single-valued_, whereas the sequence defined by `pairs` is _double-valued_.\r
+\r
+`pl.seq` provides a number of useful iterators, and some functions which operate on sequences. At first sight this example looks like an attempt to write Python in Lua, (with the sequence being inclusive):\r
+\r
+ > for i in seq.range(1,4) do print(i) end\r
+ 1\r
+ 2\r
+ 3\r
+ 4\r
+\r
+But `range` is actually equivalent to Python's `xrange`, since it generates a sequence, not a list. To get a list, use `seq.copy(seq.range(1,10))`, which takes any single-value sequence and makes a table from the result. `seq.list` is like `ipairs` except that it does not give you the index, just the value.\r
+\r
+ > for x in seq.list {1,2,3} do print(x) end\r
+ 1\r
+ 2\r
+ 3\r
+\r
+`enum` takes a sequence and turns it into a double-valued sequence consisting of a sequence number and the value, so `enum(list(ls))` is actually equivalent to `ipairs`. A more interesting example prints out a file with line numbers:\r
+\r
+ for i,v in seq.enum(io.lines(fname)) do print(i..' '..v) end\r
+\r
+Sequences can be _combined_, either by 'zipping' them or by concatenating them.\r
+\r
+ > for x,y in seq.zip(l1,l2) do print(x,y) end\r
+ 10 1\r
+ 20 2\r
+ 30 3\r
+ > for x in seq.splice(l1,l2) do print(x) end\r
+ 10\r
+ 20\r
+ 30\r
+ 1\r
+ 2\r
+ 3\r
+\r
+`seq.printall` is useful for printing out single-valued sequences, and provides some finer control over formating, such as a delimiter, the number of fields per line, and a format string to use (@see string.format)\r
+\r
+ > seq.printall(seq.random(10))\r
+ 0.0012512588885159 0.56358531449324 0.19330423902097 ....\r
+ > seq.printall(seq.random(10), ',', 4, '%4.2f')\r
+ 0.17,0.86,0.71,0.51\r
+ 0.30,0.01,0.09,0.36\r
+ 0.15,0.17,\r
+\r
+`map` will apply a function to a sequence.\r
+\r
+ > seq.printall(seq.map(string.upper, {'one','two'}))\r
+ ONE TWO\r
+ > seq.printall(seq.map('+', {10,20,30}, 1))\r
+ 11 21 31\r
+\r
+`filter` will filter a sequence using a boolean function (often called a _predicate_). For instance, this code only prints lines in a file which are composed of digits:\r
+\r
+ for l in seq.filter(io.lines(file), stringx.isdigit) do print(l) end\r
+\r
+The following returns a table consisting of all the positive values in the original table (equivalent to `tablex.filter(ls, '>', 0)`)\r
+\r
+ ls = seq.copy(seq.filter(ls, '>', 0))\r
+\r
+We're already encounted `seq.sum` when discussing `input.numbers`. This can also be expressed with `seq.reduce`:\r
+\r
+ > seq.reduce(function(x,y) return x + y end, seq.list{1,2,3,4})\r
+ 10\r
+\r
+`seq.reduce` applies a binary function in a recursive fashion, so that:\r
+\r
+ reduce(op,{1,2,3}) => op(1,reduce(op,{2,3}) => op(1,op(2,3))\r
+\r
+it's now possible to easily generate other cumulative operations; the standard operations declared in `pl.operator` are useful here:\r
+\r
+ > ops = require 'pl.operator'\r
+ > -- can also say '*' instead of ops.mul\r
+ > = seq.reduce(ops.mul,input.numbers '1 2 3 4')\r
+ 24\r
+\r
+There are functions to extract statistics from a sequence of numbers:\r
+\r
+ > l1 = List {10,20,30}\r
+ > l2 = List {1,2,3}\r
+ > = seq.minmax(l1)\r
+ 10 30\r
+ > = seq.sum(l1)\r
+ 60 3\r
+\r
+It is common to get sequences where values are repeated, say the words in a file. `count_map` will take such a sequence and count the values, returning a table where the _keys_ are the unique values, and the value associated with each key is the number of times they occurred:\r
+\r
+ > t = seq.count_map {'one','fred','two','one','two','two'}\r
+ > = t\r
+ {one=2,fred=1,two=3}\r
+\r
+This will also work on numerical sequences, but you cannot expect the result to be a proper list, i.e. having no 'holes'. Instead, you always need to use `pairs` to iterate over the result - note that there is a hole at index 5:\r
+\r
+ > t = seq.count_map {1,2,4,2,2,3,4,2,6}\r
+ > for k,v in pairs(t) do print(k,v) end\r
+ 1 1\r
+ 2 4\r
+ 3 1\r
+ 4 2\r
+ 6 1\r
+\r
+`unique` uses `count_map` to return a list of the unique values, that is, just the keys of the resulting table.\r
+\r
+`last` turns a single-valued sequence into a double-valued sequence with the current value and the last value:\r
+\r
+ > for current,last in seq.last {10,20,30,40} do print (current,last) end\r
+ 20 10\r
+ 30 20\r
+ 40 30\r
+\r
+This makes it easy to do things like identify repeated lines in a file, or construct differences between values. `filter` can handle double-valued sequences as well, so one could filter such a sequence to only return cases where the current value is less than the last value by using `operator.lt` or just '<'. This code then copies the resulting code into a table.\r
+\r
+ > ls = {10,9,10,3}\r
+ > = seq.copy(seq.filter(seq.last(s),'<'))\r
+ {9,3}\r
+\r
+\r
+### Sequence Wrappers\r
+\r
+The functions in `pl.seq` cover the common patterns when dealing with sequences, but chaining these functions together can lead to ugly code. Consider the last example of the previous section; `seq` is repeated three times and the resulting expression has to be read right-to-left. The first issue can be helped by local aliases, so that the expression becomes `copy(filter(last(s),'<'))` but the second issue refers to the somewhat unnatural order of functional application. We tend to prefer reading operations from left to right, which is one reason why object-oriented notation has become popular. Sequence adapters allow this expression to be written like so:\r
+\r
+ seq(s):last():filter('<'):copy()\r
+\r
+With this notation, the operation becomes a chain of method calls running from left to right.\r
+\r
+'Sequence' is not a basic Lua type, they are generally functions or callable objects. The expression `seq(s)` wraps a sequence in a _sequence wrapper_, which is an object which understands all the functions in `pl.seq` as methods. This object then explicitly represents sequences.\r
+\r
+As a special case, the constructor (which is when you call the table `seq`) will make a wrapper for a plain list-like table. Here we apply the length operator to a sequence of strings, and print them out.\r
+\r
+ > seq{'one','tw','t'} :map '#' :printall()\r
+ 3 2 1\r
+\r
+As a convenience, there is a function `seq.lines` which behaves just like `io.lines` except it wraps the result as an explicit sequence type. This takes the first 10 lines from standard input, makes it uppercase, turns it into a sequence with a count and the value, glues these together with the concatenation operator, and finally prints out the sequence delimited by a newline.\r
+\r
+ seq.lines():take(10):upper():enum():map('..'):printall '\n'\r
+\r
+Note the method `upper`, which is not a `seq` function. if an unknown method is called, sequence wrappers apply that method to all the values in the sequence (this is implicit use of `mapmethod`)\r
+\r
+It is straightforward to create custom sequences that can be used in this way. On Unix, `/dev/random` gives you an _endless_ sequence of random bytes, so we use `take` to limit the sequence, and then `map` to scale the result into the desired range. The key step is to use `seq` to wrap the iterator function:\r
+\r
+ -- random.lua\r
+ local seq = require 'pl.seq'\r
+\r
+ function dev_random()\r
+ local f = io.open('/dev/random')\r
+ local byte = string.byte\r
+ return seq(function()\r
+ -- read two bytes into a string and convert into a 16-bit number\r
+ local s = f:read(2)\r
+ return byte(s,1) + 256*byte(s,2)\r
+ end)\r
+ end\r
+\r
+ -- print 10 random numbers from 0 to 1 !\r
+ dev_random():take(10):map('%',100):map('/',100):printall ','\r
+\r
+\r
+Another Linux one-liner depends on the `/proc` filesystem and makes a list of all the currently running processes:\r
+\r
+ pids = seq(lfs.dir '/proc'):filter(stringx.isdigit):map(tonumber):copy()\r
+\r
+This version of Penlight has an experimental feature which relies on the fact that _all_ Lua types can have metatables, including functions. This makes _implicit sequence wrapping_ possible:\r
+\r
+ > seq.import()\r
+ > seq.random(5):printall(',',5,'%4.1f')\r
+ 0.0, 0.1, 0.4, 0.1, 0.2\r
+\r
+This avoids the awkward `seq(seq.random(5))` construction. Or the iterator can come from somewhere else completely:\r
+\r
+ > ('one two three'):gfind('%a+'):printall(',')\r
+ one,two,three,\r
+\r
+After `seq.import`, it is no longer necessary to explicitly wrap sequence functions.\r
+\r
+But there is a price to pay for this convenience. _Every_ function is affected, so that any function can be used, appropriate or not:\r
+\r
+ > math.sin:printall()\r
+ ..seq.lua:287: bad argument #1 to '(for generator)' (number expected, got nil)\r
+ > a = tostring\r
+ > = a:find(' ')\r
+ function: 0042C920\r
+\r
+What function is returned? It's almost certain to be something that makes no sense in the current context. So implicit sequences may make certain kinds of programming mistakes harder to catch - they are best used for interactive exploration and small scripts.\r
+\r
+<a id="comprehensions"/>\r
+\r
+### List Comprehensions\r
+\r
+List comprehensions are a compact way to create tables by specifying their elements. In Python, you can say this:\r
+\r
+ ls = [x for x in range(5)] # == [0,1,2,3,4]\r
+\r
+In Lua, using `pl.comprehension`:\r
+\r
+ > C = require('pl.comprehension').new()\r
+ > = C ('x for x=1,10') ()\r
+ {1,2,3,4,5,6,7,8,9,10}\r
+\r
+`C` is a function which compiles a list comprehension _string_ into a _function_. In this case, the function has no arguments. The parentheses are redundant for a function taking a string argument, so this works as well:\r
+\r
+ > = C 'x^2 for x=1,4' ()\r
+ {1,4,9,16}\r
+ > = C '{x,x^2} for x=1,4' ()\r
+ {{1,1},{2,4},{3,9},{4,16}}\r
+\r
+Note that the expression can be _any_ function of the variable `x`!\r
+\r
+The basic syntax so far is `<expr> for <set>`, where `<set>` can be anything that the Lua `for` statement understands. `<set>` can also just be the variable, in which case the values will come from the _argument_ of the comprehension. Here I'm emphasizing that a comprehension is a function which can take a list argument:\r
+\r
+ > = C '2*x for x' {1,2,3}\r
+ {2,4,6}\r
+ > dbl = C '2*x for x'\r
+ > = dbl {10,20,30}\r
+ {20,40,60}\r
+\r
+Here is a somewhat more explicit way of saying the same thing; `_1` is a _placeholder_ refering to the _first_ argument passed to the comprehension.\r
+\r
+ > = C '2*x for _,x in pairs(_1)' {10,20,30}\r
+ {20,40,60}\r
+ > = C '_1(x) for x'(tostring,{1,2,3,4})\r
+ {'1','2','3','4'}\r
+\r
+This extended syntax is useful when you wish to collect the result of some iterator, such as `io.lines`. This comprehension creates a function which creates a table of all the lines in a file:\r
+\r
+ > f = io.open('array.lua')\r
+ > lines = C 'line for line in _1:lines()' (f)\r
+ > = #lines\r
+ 118\r
+\r
+There are a number of functions that may be applied to the result of a comprehension:\r
+\r
+ > = C 'min(x for x)' {1,44,0}\r
+ 0\r
+ > = C 'max(x for x)' {1,44,0}\r
+ 44\r
+ > = C 'sum(x for x)' {1,44,0}\r
+ 45\r
+\r
+(These are equivalent to a reduce operation on a list.)\r
+\r
+After the `for` part, there may be a condition, which filters the output. This comprehension collects the even numbers from a list:\r
+\r
+ > = C 'x for x if x % 2 == 0' {1,2,3,4,5}\r
+ {2,4}\r
+\r
+There may be a number of `for` parts:\r
+\r
+ > = C '{x,y} for x = 1,2 for y = 1,2' ()\r
+ {{1,1},{1,2},{2,1},{2,2}}\r
+ > = C '{x,y} for x for y' ({1,2},{10,20})\r
+ {{1,10},{1,20},{2,10},{2,20}}\r
+\r
+These comprehensions are useful when dealing with functions of more than one variable, and are not so easily achieved with the other Penlight functional forms.\r
+\r
+<a id="func"/>\r
+\r
+### Creating Functions from Functions\r
+\r
+@lookup pl.func\r
+\r
+Lua functions may be treated like any other value, although of course you cannot multiply or add them. One operation that makes sense is _function composition_, which chains function calls (so `(f * g)(x)` is `f(g(x))`.)\r
+\r
+ > func = require 'pl.func'\r
+ > printf = func.compose(io.write,string.format)\r
+ > printf("hello %s\n",'world')\r
+ hello world\r
+ true\r
+\r
+Many functions require you to pass a function as an argument, say to apply to all values of a sequence or as a callback. Often useful functions have the wrong number of arguments. So there is a need to construct a function of one argument from one of two arguments, _binding_ the extra argument to a given value.\r
+\r
+_currying_ takes a function of n arguments and returns a function of n-1 arguments where the first argument is bound to some value:\r
+\r
+ > p2 = func.curry(print,'start>')\r
+ > p2('hello',2)\r
+ start> hello 2\r
+ > ops = require 'pl.operator'\r
+ > = tablex.filter({1,-2,10,-1,2},curry(ops.gt,0))\r
+ {-2,-1}\r
+ > tablex.filter({1,-2,10,-1,2},curry(ops.le,0))\r
+ {1,10,2}\r
+\r
+The last example unfortunately reads backwards, because `curry` alway binds the first argument!\r
+\r
+Currying is a specialized form of function argument binding. Here is another way to say the `print` example:\r
+\r
+ > p2 = func.bind(print,'start>',func._1,func._2)\r
+ > p2('hello',2)\r
+ start> hello 2\r
+\r
+where `_1` and `_2` are _placeholder variables_, corresponding to the first and second argument respectively.\r
+\r
+Having `func` all over the place is distracting, so it's useful to pull all of `pl.func` into the local context. Here is the filter example, this time the right way around:\r
+\r
+ > utils.import 'pl.func'\r
+ > tablex.filter({1,-2,10,-1,2},bind(ops.gt, _1, 0))\r
+ {1,10,2}\r
+\r
+\r
+`tablex.merge` does a general merge of two tables. This example shows the usefulness of binding the last argument of a function.\r
+\r
+ > S1 = {john=27, jane=31, mary=24}\r
+ > S2 = {jane=31, jones=50}\r
+ > intersection = bind(tablex.merge, _1, _2, false)\r
+ > union = bind(tablex.merge, _1, _2, true)\r
+ > = intersection(S1,S2)\r
+ {jane=31}\r
+ > = union(S1,S2)\r
+ {mary=24,jane=31,john=27,jones=50}\r
+\r
+When using `bind` to curry `print`, we got a function of precisely two arguments, whereas we really want our function to use varargs like `print`. This is the role of `_0`:\r
+\r
+ > _DEBUG = true\r
+ > p = bind(print,'start>', _0)\r
+ return function (fn,_v1)\r
+ return function(...) return fn(_v1,...) end\r
+ end\r
+\r
+ > p(1,2,3,4,5)\r
+ start> 1 2 3 4 5\r
+\r
+I've turned on the global `_DEBUG` flag, so that the function generated is printed out. It is actually a function which _generates_ the required function; the first call _binds the value_ of `_v1` to 'start>'.\r
+\r
+### Placeholder Expressions\r
+\r
+A common pattern in Penlight is a function which applies another function to all elements in a table or a sequence, such as `tablex.map` or `seq.filter`. Lua does anonymous functions well, although they can be a bit tedious to type:\r
+\r
+ > = tablex.map(function(x) return x*x end, {1,2,3,4})\r
+ {1,4,9,16}\r
+\r
+`pl.func` allows you to define _placeholder expressions_, which can cut down on the typing required, and also make your intent clearer. First, we bring contents of `pl.func` into our context, and then supply an expression using placeholder variables, such as `_1`,`_2`,etc. (C++ programmers will recognize this from the Boost libraries.)\r
+\r
+ > utils.import 'pl.func'\r
+ > = tablex.map(_1*_1, {1,2,3,4})\r
+ {1,4,9,16}\r
+\r
+Functions of up to 5 arguments can be generated.\r
+\r
+ > = tablex.map2(_1+_2,{1,2,3}, {10,20,30})\r
+ {11,22,33}\r
+\r
+These expressions can use arbitrary functions, altho they must first be registered with the functional library. `func.register` brings in a single function, and `func.import` brings in a whole table of functions, such as `math`.\r
+\r
+ > sin = register(math.sin)\r
+ > = tablex.map(sin(_1), {1,2,3,4})\r
+ {0.8414709848079,0.90929742682568,0.14112000805987,-0.75680249530793}\r
+ > import 'math'\r
+ > = tablex.map(cos(2*_1),{1,2,3,4})\r
+ {-0.41614683654714,-0.65364362086361,0.96017028665037,-0.14550003380861}\r
+\r
+A common operation is calling a method of a set of objects:\r
+\r
+ > = tablex.map(_1:sub(1,1), {'one','four','x'})\r
+ {'o','f','x'}\r
+\r
+There are some restrictions on what operators can be used in PEs. For instance, because the `__len` metamethod cannot be overriden by plain Lua tables, we need to define a special function to express `#_1':\r
+\r
+ > = tablex.map(Len(_1), {'one','four','x'})\r
+ {3,4,1}\r
+\r
+Likewise for comparison operators, which cannot be overloaded for _different_ types, and thus also have to be expressed as a special function:\r
+\r
+ > = tablex.filter(Gt(_1,0), {1,-1,2,4,-3})\r
+ {1,2,4}\r
+\r
+It is useful to express the fact that a function returns multiple values. For instance, `tablex.pairmap` expects a function that will be called with the key and the value, and returns the new value and the key, in that order.\r
+\r
+ > = pairmap(Args(_2,_1:upper()),{fred=1,alice=2})\r
+ {ALICE=2,FRED=1}\r
+\r
+PEs cannot contain `nil` values, since PE function arguments are represented as an array. Instead, a special value called `Nil` is provided. So say `_1:f(Nil,1)` instead of `_1:f(nil,1)`.\r
+\r
+A placeholder expression cannot be automatically used as a Lua function. The technical reason is that the call operator must be overloaded to construct function calls like `_1(1)`. If you want to force a PE to return a function, use `func.I`.\r
+\r
+ > = tablex.map(_1(10),{I(2*_1),I(_1*_1),I(_1+2)})\r
+ {20,100,12}\r
+\r
+Here we make a table of functions taking a single argument, and then call them all with a value of 10.\r
+\r
+The essential idea with PEs is to 'quote' an expression so that it is not immediately evaluated, but instead turned into a function that can be applied later to some arguments. The basic mechanism is to wrap values and placeholders so that the usual Lua operators have the effect of building up an _expression tree_. (It turns out that you can do _symbolic algebra_ using PEs, see `symbols.lua` in the examples directory, and its test runner `testsym.lua`, which demonstrates symbolic differentiation.)\r
+\r
+The rule is that if any operator has a PE operand, the result will be quoted. Sometimes we need to quote things explicitly. For instance, say we want to pass a function to a filter that must return true if the element value is in a set. `set[_1]` is the obvious expression, but it does not give the desired result, since it evaluates directly, giving `nil`. Indexing works differently than a binary operation like addition (set+_1 _is_ properly quoted) so there is a need for an explicit quoting or wrapping operation. This is the job of the `_` function; the PE in this case should be `_(set)[_1]`. This works for functions as well, as a convenient alternative to registering functions: `_(math.sin)(_1)`. This is equivalent to using the `lines' method:\r
+\r
+ for line in I(_(f):read()) do print(line) end\r
+\r
+Now this will work for _any_ 'file-like' object which which has a `read` method returning the next line. If you had a LuaSocket client which was being 'pushed' by lines sent from a server, then `_(s):receive '*l'` would create an iterator for accepting input. These forms can be convenient for adapting your data flow so that it can be passed to the sequence functions in `pl.seq'.\r
+\r
+Placeholder expressions can be mixed with sequence wrapper expressions. `lexer.lua` will give us a double-valued sequence of tokens, where the first value is a type, and the second is a value. We filter out only the values where the type is 'iden', extract the actual value using `map`, get the unique values and finally copy to a list.\r
+\r
+ > str = 'for i=1,10 do for j = 1,10 do print(i,j) end end'\r
+ > = seq(lexer.lua(str)):filter('==','iden'):map(_2):unique():copy()\r
+ {i,print,j}\r
+\r
+This is a particularly intense line (and I don't always suggest making everything a one-liner!); the key is the behaviour of `map`, which will take both values of the sequence, so `_2` returns the value part. (Since `filter` here takes extra arguments, it only operates on the type values.)\r
+\r
+There are some performance considerations to using placeholder expressions. Instantiating a PE requires constructing and compiling a function, which is not such a fast operation. So to get best performance, factor out PEs from loops like this;\r
+\r
+ local fn = I(_1:f() + _2:g())\r
+ for i = 1,n do\r
+ res[i] = tablex.map2(fn,first[i],second[i])\r
+ end\r
+\r
+\r
--- /dev/null
+## Additional Libraries\r
+\r
+Libraries in this section are no longer considered to be part of the Penlight core, but still provide specialized functionality when needed.\r
+\r
+<a id="sip"/>\r
+\r
+### Simple Input Patterns\r
+\r
+Lua string pattern matching is very powerful, and usually you will not need a traditional regular expression library. Even so, sometimes Lua code ends up looking like Perl, which happens because string patterns are not always the easiest things to read, especially for the casual reader. Here is a program which needs to understand three distinct date formats:\r
+\r
+ -- parsing dates using Lua string patterns\r
+ months={Jan=1,Feb=2,Mar=3,Apr=4,May=5,Jun=6,\r
+ Jul=7,Aug=8,Sep=9,Oct=10,Nov=11,Dec=12}\r
+\r
+ function check_and_process(d,m,y)\r
+ d = tonumber(d)\r
+ m = tonumber(m)\r
+ y = tonumber(y)\r
+ ....\r
+ end\r
+\r
+ for line in f:lines() do\r
+ -- ordinary (English) date format\r
+ local d,m,y = line:match('(%d+)/(%d+)/(%d+)')\r
+ if d then\r
+ check_and_process(d,m,y)\r
+ else -- ISO date??\r
+ y,m,d = line:match('(%d+)%-(%d+)%-(%d+)')\r
+ if y then\r
+ check_and_process(d,m,y)\r
+ else -- <day> <month-name> <year>?\r
+ d,mm,y = line:match('%(d+)%s+(%a+)%s+(%d+)')\r
+ m = months[mm]\r
+ check_and_process(d,m,y)\r
+ end\r
+ end\r
+ end\r
+\r
+These aren't particularly difficult patterns, but already typical issues are appearing, such as having to escape '-'. Also, `string.match` returns its captures, so that we're forced to use a slightly awkward nested if-statement.\r
+\r
+Verification issues will further cloud the picture, since regular expression people try to enforce constraints (like year cannot be more than four digits) using regular expressions, on the usual grounds that one shouldn't stop using a hammer when one is enjoying oneself.\r
+\r
+`pl.sip` provides a simple, intuitive way to detect patterns in strings and extract relevant parts.\r
+\r
+ > sip = require 'pl.sip'\r
+ > dump = require('pl.pretty').dump\r
+ > res = {}\r
+ > c = sip.compile 'ref=$S{file}:$d{line}'\r
+ > = c('ref=hello.c:10',res)\r
+ true\r
+ > dump(res)\r
+ {\r
+ line = 10,\r
+ file = "hello.c"\r
+ }\r
+ > = c('ref=long name, no line',res)\r
+ false\r
+\r
+`sip.compile` creates a pattern matcher function, which is given a string and a table. If it matches the string, then `true` is returned and the table is populated according to the _named fields_ in the pattern.\r
+\r
+Here is another version of the date parser:\r
+\r
+ -- using SIP patterns\r
+ function check(t)\r
+ check_and_process(t.day,t.month,t.year)\r
+ end\r
+\r
+ shortdate = sip.compile('$d{day}/$d{month}/$d{year}')\r
+ longdate = sip.compile('$d{day} $v{mon} $d{year}')\r
+ isodate = sip.compile('$d{year}-$d{month}-$d{day}')\r
+\r
+ for line in f:lines() do\r
+ local res = {}\r
+ if shortdate(str,res) then\r
+ check(res)\r
+ elseif isodate(str,res) then\r
+ check(res)\r
+ elseif longdate(str,res) then\r
+ res.month = months[res.mon]\r
+ check(res)\r
+ end\r
+ end\r
+\r
+SIP patterns start with '$', then a one-letter type, and then an optional variable in curly braces.\r
+\r
+ Type Meaning\r
+ v variable, or identifier.\r
+ i possibly signed integer\r
+ f floating-point number\r
+ r 'rest of line'\r
+ q quoted string (either ' or ")\r
+ p a path name\r
+ ( anything inside (...)\r
+ [ anything inside [...]\r
+ { anything inside {...}\r
+ < anything inside <...>\r
+ [---------------------------------]\r
+ S non-space\r
+ d digits\r
+ ...\r
+\r
+If a type is not one of v,i,f,r or q, then it's assumed to be one of the standard Lua character classes. Any spaces you leave in your pattern will match any number of spaces. And any 'magic' string characters will be escaped.\r
+\r
+SIP captures (like `$v{mon}`) do not have to be named. You can use just `$v`, but you have to be consistent; if a pattern contains unnamed captures, then all captures must be unnamed. In this case, the result table is a simple list of values.\r
+\r
+`sip.match` is a useful shortcut if you like your matches to be 'in place'. (It caches the result, so it is not much slower than explicitly using `sip.compile`.)\r
+\r
+ > sip.match('($q{first},$q{second})','("john","smith")',res)\r
+ true\r
+ > res\r
+ {second='smith',first='john'}\r
+ > res = {}\r
+ > sip.match('($q,$q)','("jan","smit")',res) -- unnamed captures\r
+ true\r
+ > res\r
+ {'jan','smit'}\r
+ > sip.match('($q,$q)','("jan", "smit")',res)\r
+ false ---> oops! Can't handle extra space!\r
+ > sip.match('( $q , $q )','("jan", "smit")',res)\r
+ true\r
+\r
+As a general rule, allow for whitespace in your patterns.\r
+\r
+Finally, putting a ' $' at the end of a pattern means 'capture the rest of the line, starting at the first non-space'.\r
+\r
+ > sip.match('( $q , $q ) $','("jan", "smit") and a string',res)\r
+ true\r
+ > res\r
+ {'jan','smit','and a string'}\r
+ > res = {}\r
+ > sip.match('( $q{first} , $q{last} ) $','("jan", "smit") and a string',res)\r
+ true\r
+ > res\r
+ {first='jan',rest='and a string',last='smit'}\r
+\r
+\r
+<a id="lapp"/>\r
+\r
+### Command-line Programs with Lapp\r
+\r
+`pl.lapp` is a small and focused Lua module which aims to make standard command-line parsing easier and intuitive. It implements the standard GNU style, i.e. short flags with one letter start with '-', and there may be an additional long flag which starts with '--'. Generally options which take an argument expect to find it as the next parameter (e.g. 'gcc test.c -o test') but single short options taking a numerical parameter can dispense with the space (e.g. 'head -n4 test.c')\r
+\r
+As far as possible, Lapp will convert parameters into their equivalent Lua types, i.e. convert numbers and convert filenames into file objects. If any conversion fails, or a required parameter is missing, an error will be issued and the usage text will be written out. So there are two necessary tasks, supplying the flag and option names and associating them with a type.\r
+\r
+For any non-trivial script, even for personal consumption, it's necessary to supply usage text. The novelty of Lapp is that it starts from that point and defines a loose format for usage strings which can specify the names and types of the parameters.\r
+\r
+An example will make this clearer:\r
+\r
+ -- scale.lua\r
+ lapp = require 'pl.lapp'\r
+ local args = lapp [[\r
+ Does some calculations\r
+ -o,--offset (default 0.0) Offset to add to scaled number\r
+ -s,--scale (number) Scaling factor\r
+ <number> (number ) Number to be scaled\r
+ ]]\r
+\r
+ print(args.offset + args.scale * args.number)\r
+\r
+Here is a command-line session using this script:\r
+\r
+ $ lua scale.lua\r
+ scale.lua:missing required parameter: scale\r
+\r
+ Does some calculations\r
+ -o,--offset (default 0.0) Offset to add to scaled number\r
+ -s,--scale (number) Scaling factor\r
+ <number> (number ) Number to be scaled\r
+\r
+ $ lua scale.lua -s 2.2 10\r
+ 22\r
+\r
+ $ lua scale.lua -s 2.2 x10\r
+ scale.lua:unable to convert to number: x10\r
+\r
+ ....(usage as before)\r
+\r
+There are two kinds of lines in Lapp usage strings which are meaningful; option and parameter lines. An option line gives the short option, optionally followed by the corresponding long option. A type specifier in parentheses may follow. Similarly, a parameter line starts with '<' PARAMETER '>', followed by a type specifier. Type specifiers are either of the form '(default ' VALUE ')' or '(' TYPE ')'; the default specifier means that the parameter or option has a default value and is not required. TYPE is one of 'string','number','file-in' or 'file-out'; VALUE is a number, one of ('stdin','stdout','stderr') or a token. The rest of the line is not parsed and can be used for explanatory text.\r
+\r
+This script shows the relation between the specified parameter names and the fields in the output table.\r
+\r
+ -- simple.lua\r
+ local args = require ('pl.lapp') [[\r
+ Various flags and option types\r
+ -p A simple optional flag, defaults to false\r
+ -q,--quiet A simple flag with long name\r
+ -o (string) A required option with argument\r
+ <input> (default stdin) Optional input file parameter\r
+ ]]\r
+\r
+ for k,v in pairs(args) do\r
+ print(k,v)\r
+ end\r
+\r
+I've just dumped out all values of the args table; note that args.quiet has become true, because it's specified; args.p defaults to false. If there is a long name for an option, that will be used in preference as a field name. A type or default specifier is not necessary for simple flags, since the default type is boolean.\r
+\r
+ $ simple -o test -q simple.lua\r
+ p false\r
+ input file (781C1BD8)\r
+ quiet true\r
+ o test\r
+ input_name simple.lua\r
+ D:\dev\lua\lapp>simple -o test simple.lua one two three\r
+ 1 one\r
+ 2 two\r
+ 3 three\r
+ p false\r
+ quiet false\r
+ input file (781C1BD8)\r
+ o test\r
+ input_name simple.lua\r
+\r
+The parameter input has been set to an open read-only file object - we know it must be a read-only file since that is the type of the default value. The field input_name is automatically generated, since it's often useful to have access to the original filename.\r
+\r
+Notice that any extra parameters supplied will be put in the result table with integer indices, i.e. args[i] where i goes from 1 to #args.\r
+\r
+Files don't really have to be closed explicitly for short scripts with a quick well-defined mission, since the result of garbage-collecting file objects is to close them.\r
+\r
+#### Enforcing a Range for a Parameter\r
+\r
+The type specifier can also be of the form '(' MIN '..' MAX ')'.\r
+\r
+ local lapp = require 'pl.lapp'\r
+ local args = lapp [[\r
+ Setting ranges\r
+ <x> (1..10) A number from 1 to 10\r
+ <y> (-5..1e6) Bigger range\r
+ ]]\r
+\r
+ print(args.x,args.y)\r
+\r
+Here the meaning is that the value is greater or equal to MIN and less or equal to MAX; there is no provision for forcing a parameter to be a whole number.\r
+\r
+You may also define custom types that can be used in the type specifier:\r
+\r
+ lapp = require ('pl.lapp')\r
+\r
+ lapp.add_type('integer','number',\r
+ function(x)\r
+ lapp.assert(math.ceil(x) == x, 'not an integer!')\r
+ end\r
+ )\r
+\r
+ local args = lapp [[\r
+ <ival> (integer) Process PID\r
+ ]]\r
+\r
+ print(args.ival)\r
+\r
+`lapp.add_type` takes three parameters, a type name, a converter and a constraint function. The constraint function is expected to throw an assertion if some condition is not true; we use lapp.assert because it fails in the standard way for a command-line script. The converter argument can either be a type name known to Lapp, or a function which takes a string and generates a value.\r
+\r
+#### 'varargs' Parameter Arrays\r
+\r
+ lapp = require 'pl.lapp'\r
+ local args = lapp [[\r
+ Summing numbers\r
+ <numbers...> (number) A list of numbers to be summed\r
+ ]]\r
+\r
+ local sum = 0\r
+ for i,x in ipairs(args.numbers) do\r
+ sum = sum + x\r
+ end\r
+ print ('sum is '..sum)\r
+\r
+The parameter number has a trailing '...', which indicates that this parameter is a 'varargs' parameter. It must be the last parameter, and args.number will be an array.\r
+\r
+Consider this implementation of the head utility from Mac OS X:\r
+\r
+ -- implements a BSD-style head\r
+ -- (see http://www.manpagez.com/man/1/head/osx-10.3.php)\r
+\r
+ lapp = require ('pl.lapp')\r
+\r
+ local args = lapp [[\r
+ Print the first few lines of specified files\r
+ -n (default 10) Number of lines to print\r
+ <files...> (default stdin) Files to print\r
+ ]]\r
+\r
+ -- by default, lapp converts file arguments to an actual Lua file object.\r
+ -- But the actual filename is always available as <file>_name.\r
+ -- In this case, 'files' is a varargs array, so that 'files_name' is\r
+ -- also an array.\r
+ local nline = args.n\r
+ local nfile = #args.files\r
+ for i = 1,nfile do\r
+ local file = args.files[i]\r
+ if nfile > 1 then\r
+ print('==> '..args.files_name[i]..' <==')\r
+ end\r
+ local n = 0\r
+ for line in file:lines() do\r
+ print(line)\r
+ n = n + 1\r
+ if n == nline then break end\r
+ end\r
+ end\r
+\r
+Note how we have access to all the filenames, because the auto-generated field `files_name` is also an array!\r
+\r
+(This is probably not a very considerate script, since Lapp will open all the files provided, and only close them at the end of the script. See the `xhead.lua` example for another implementation.)\r
+\r
+Flags and options may also be declared as vararg arrays, and can occur anywhere. Bear in mind that short options can be combined (like 'tar -xzf'), so it's perfectly legal to have '-vvv'. But normally the value of args.v is just a simple `true` value.\r
+\r
+ local args = require ('pl.lapp') [[\r
+ -v... Verbosity level; can be -v, -vv or -vvv\r
+ ]]\r
+ vlevel = not args.v[1] and 0 or #args.v\r
+ print(vlevel)\r
+\r
+The vlevel assigment is a bit of Lua voodoo, so consider the cases:\r
+\r
+ * No -v flag, v is just { false }\r
+ * One -v flags, v is { true }\r
+ * Two -v flags, v is { true, true }\r
+ * Three -v flags, v is { true, true, true }\r
+\r
+#### Defining a Parameter Callback\r
+\r
+If a script implements `lapp.callback`, then Lapp will call it after each argument is parsed. The callback is passed the parameter name, the raw unparsed value, and the result table. It is called immediately after assignment of the value, so the corresponding field is available.\r
+\r
+ lapp = require ('pl.lapp')\r
+\r
+ function lapp.callback(parm,arg,args)\r
+ print('+',parm,arg)\r
+ end\r
+\r
+ local args = lapp [[\r
+ Testing parameter handling\r
+ -p Plain flag (defaults to false)\r
+ -q,--quiet Plain flag with GNU-style optional long name\r
+ -o (string) Required string option\r
+ -n (number) Required number option\r
+ -s (default 1.0) Option that takes a number, but will default\r
+ <start> (number) Required number argument\r
+ <input> (default stdin) A parameter which is an input file\r
+ <output> (default stdout) One that is an output file\r
+ ]]\r
+ print 'args'\r
+ for k,v in pairs(args) do\r
+ print(k,v)\r
+ end\r
+\r
+This produces the following output:\r
+\r
+ $ args -o name -n 2 10 args.lua\r
+ + o name\r
+ + n 2\r
+ + start 10\r
+ + input args.lua\r
+ args\r
+ p false\r
+ s 1\r
+ input_name args.lua\r
+ quiet false\r
+ output file (781C1B98)\r
+ start 10\r
+ input file (781C1BD8)\r
+ o name\r
+ n 2\r
+\r
+Callbacks are needed when you want to take action immediately on parsing an argument.\r
+\r
--- /dev/null
+## Technical Choices\r
+\r
+### Modularity and Granularity\r
+\r
+In an ideal world, a program should only load the libraries it needs. Penlight is intended to work in situations where an extra 100Kb of bytecode could be a problem. It is straightforward but tedious to load exactly what you need:\r
+\r
+ local data = require 'pl.data'\r
+ local List = require 'pl.List'\r
+ local array2d = require 'pl.array2d'\r
+ local seq = require 'pl.seq'\r
+ local utils = require 'pl.utils'\r
+\r
+This is the style that I follow in Penlight itself, so that modules don't mess with the global environment; also, `stringx.import()` is not used because it will update the global `string` table.\r
+\r
+But `require 'pl'` is more convenient in scripts; the question is how to ensure that one doesn't load the whole kitchen sink as the price of convenience. The strategy is to only load modules when they are referenced. In 'init.lua' (which is loaded by `require 'pl'`) a metatable is attached to the global table with an `__index` metamethod. Any unknown name is looked up in the list of modules, and if found, we require it and make that module globally available. So when `tablex.deepcompare` is encountered, looking up `tablex` causes 'pl.tablex' to be required. .\r
+\r
+Modifying the behaviour of the global table has consequences. For instance, there is the famous module `strict` which comes with Lua itself (perhaps the only standard Lua module written in Lua itself) which also does this modification so that global variiables must be defined before use. So the implementation in 'init.lua' allows for a 'not found' hook, which 'pl.strict.lua' uses. Other libraries may install their own metatables for `_G`, but Penlight will now forward any unknown name to the `__index` defined by the original metatable.\r
+\r
+But the strategy is worth the effort: the old 'kitchen sink' 'init.lua' would pull in about 260K of bytecode, whereas now typical programs use about 100K less, and short scripts even better - for instance, if they were only needing functionality in `utils`.\r
+\r
+There are some functions which mark their output table with a special metatable, when it seems particularly appropriate. For instance, `tablex.makeset` creates a `Set`, and `seq.copy` creates a `List`. But this does not automatically result in the loading of `pl.Set` and `pl.List`; only if you try to access any of these methods. In 'utils.lua', there is an exported table called `stdmt`:\r
+\r
+ stdmt = { List = {}, Map = {}, Set = {}, MultiMap = {} }\r
+\r
+If you go through 'init.lua', then these plain little 'identity' tables get an `__index` metamethod which forces the loading of the full functionality. Here is the code from 'list.lua' which starts the ball rolling for lists:\r
+\r
+ List = utils.stdmt.List\r
+ List.__index = List\r
+ List._name = "List"\r
+ List._class = List\r
+\r
+The 'load-on-demand' strategy helps to modularize the library. Especially for more casual use, `require 'pl'` is a good compromise between convenience and modularity.\r
+\r
+In this current version, I have generally reduced the amount of trickery involved. Previously, `Map` was defined in `pl.class`; now it is sensibly defined in `pl.Map`; `pl.class` only contains the basic class mechanism (and returns that function.) For consistency, `List` is returned directly by `require 'pl.List'` (note the uppercase 'L'), Also, the amount of module dependencies in the non-core libraries like `pl.config` have been reduced.\r
+\r
+### Defining what is Callable\r
+\r
+'utils.lua' exports `function_arg` which is used extensively throughout Penlight. It defines what is meant by 'callable'. Obviously true functions are immediately passed back. But what about strings? The first option is that it represents an operator in 'operator.lua', so that '<' is just an alias for `operator.lt`.\r
+\r
+We then check whether there is a _function factory_ defined for the metatable of the value.\r
+\r
+(It is true that strings can be made callable, but in practice this turns out to be a cute but dubious idea, since _all_ strings share the same metatable. A common programming error is to pass the wrong kind of object to a function, and it's better to get a nice clean 'attempting to call a string' message rather than some obscure trace from the bowels of your library.)\r
+\r
+The other module that registers a function factory is `pl.func`. Placeholder expressions cannot be directly calleable, and so need to be instantiated and cached in as efficient way as possible.\r
+\r
+(An inconsistency is that `utils.is_callable` does not do this thorough check.)\r
+\r
+\r
--- /dev/null
+-- shows how replacing '@see module' in the Markdown documentation\r
+-- can be done more elegantly using PL.\r
+-- We either have something like 'pl.config' (a module reference)\r
+-- or 'pl.seq.map' (a function reference); these cases must be distinguished\r
+-- and a Markdown link generated pointing to the LuaDoc file.\r
+\r
+require 'pl'\r
+\r
+local res = {}\r
+s = [[\r
+(@see pl.bonzo.dog)\r
+remember about @see pl.bonzo\r
+\r
+]]\r
+\r
+local _gsub_patterns = {}\r
+\r
+function gsub (s,pat,subst,start)\r
+ local fpat = _gsub_patterns[pat]\r
+ if not fpat then\r
+ -- use SIP to generate a proper string pattern.\r
+ -- the _whole thing_ is a capture, to get the whole match\r
+ -- and the unnamed capture.\r
+ fpat = '('..sip.create_pattern(pat)..')'\r
+ _gsub_patterns[pat] = fpat\r
+ end\r
+ return s:gsub(fpat,subst,start)\r
+end\r
+\r
+\r
+local mod = sip.compile '$v.$v'\r
+local fun = sip.compile '$v.$v.$v'\r
+\r
+for line in stringx.lines(s) do\r
+ line = gsub(line,'@see $p',function(see,path)\r
+ if fun(path,res) or mod(path,res) then\r
+ local ret = ('[see %s](%s.%s.html'):format(path,res[1],res[2])\r
+ if res[3] then\r
+ return ret..'#'..res[3]..')'\r
+ else\r
+ return ret..')'\r
+ end\r
+ end\r
+ end)\r
+ print(line)\r
+end\r
+\r
+\r
+\r
+\r
+\r
+\r
+\r
+\r
--- /dev/null
+-- another SIP example, shows how an awkward log file format\r
+-- can be parsed. It also prints out the actual Lua string\r
+-- pattern generated:\r
+-- SYNC%s*%[([+%-%d]%d*)%]%s*([+%-%d]%d*)%s*([+%-%d]%d*)\r
+\r
+require 'pl'\r
+\r
+s = [[\r
+SYNC [1] 0 547 (14679 sec)\r
+SYNC [2] 0 555 (14679 sec)\r
+SYNC [3] 0 563 (14679 sec)\r
+SYNC [4] 0 571 (14679 sec)\r
+SYNC [5] -1 580 (14679 sec)\r
+SYNC [6] 0 587 (14679 sec)\r
+]]\r
+\r
+\r
+local first = true\r
+local start\r
+local res = {}\r
+local pat = 'SYNC [$i{seq}] $i{diff} $i{val}'\r
+print(sip.create_pattern(pat))\r
+local match = sip.compile(pat)\r
+for line in stringx.lines(s) do\r
+ if match(line,res) then\r
+ if first then\r
+ expected = res.val\r
+ first = false\r
+ end\r
+ print(res.val,expected - res.val)\r
+ expected = expected + 8\r
+ end\r
+end\r
--- /dev/null
+require 'pl'\r
+utils.import 'pl.func'\r
+local ops = require 'pl.operator'\r
+local List = require 'pl.List'\r
+local append,concat = table.insert,table.concat\r
+local compare,find_if,compare_no_order,imap,reduce,count_map = tablex.compare,tablex.find_if,tablex.compare_no_order,tablex.imap,tablex.reduce,tablex.count_map\r
+\r
+function bindval (self,val)\r
+ rawset(self,'value',val)\r
+end\r
+\r
+local optable = ops.optable\r
+\r
+function sexpr (e)\r
+ if isPE(e) then\r
+ if e.op ~= 'X' then\r
+ local args = tablex.imap(sexpr,e)\r
+ return '('..e.op..' '..table.concat(args,' ')..')'\r
+ else\r
+ return e.repr\r
+ end\r
+ else\r
+ return tostring(e)\r
+ end\r
+end\r
+\r
+\r
+psexpr = compose(print,sexpr)\r
+\r
+\r
+\r
+function equals (e1,e2)\r
+ local p1,p2 = isPE(e1),isPE(e2)\r
+ if p1 ~= p2 then return false end -- different kinds of animals!\r
+ if p1 and p2 then -- both PEs\r
+ -- operators must be the same\r
+ if e1.op ~= e2.op then return false end\r
+ -- PHs are equal if their representations are equal\r
+ if e1.op == 'X' then return e1.repr == e2.repr\r
+ -- commutative operators\r
+ elseif e1.op == '+' or e1.op == '*' then\r
+ return compare_no_order(e1,e2,equals)\r
+ else\r
+ -- arguments must be the same\r
+ return compare(e1,e2,equals)\r
+ end\r
+ else -- fall back on simple equality for non PEs\r
+ return e1 == e2\r
+ end\r
+end\r
+\r
+-- run down an unbalanced operator chain (like a+b+c) and return the arguments {a,b,c}\r
+function tcollect (op,e,ls)\r
+ if isPE(e) and e.op == op then\r
+ for i = 1,#e do\r
+ tcollect(op,e[i],ls)\r
+ end\r
+ else\r
+ ls:append(e)\r
+ return\r
+ end\r
+end\r
+\r
+function rcollect (e)\r
+ local res = List()\r
+ tcollect(e.op,e,res)\r
+ return res\r
+end\r
+\r
+\r
+-- balance ensures that +/* chains are collected together, operates in-place.\r
+-- thus (+(+ a b) c) or (+ a (+ b c)) becomes (+ a b c), order immaterial\r
+function balance (e)\r
+ if isPE(e) and e.op ~= 'X' then\r
+ local op,args = e.op\r
+ if op == '+' or op == '*' then\r
+ args = rcollect(e)\r
+ else\r
+ args = imap(balance,e)\r
+ end\r
+ for i = 1,#args do\r
+ e[i] = args[i]\r
+ end\r
+ end\r
+ return e\r
+end\r
+\r
+-- fold constants in an expression\r
+function fold (e)\r
+ if isPE(e) then\r
+ if e.op == 'X' then\r
+ -- there could be _bound values_!\r
+ local val = rawget(e,'value')\r
+ return val and val or e\r
+ else\r
+ local op = e.op\r
+ local addmul = op == '*' or op == '+'\r
+ -- first fold all arguments\r
+ local args = imap(fold,e)\r
+ if not addmul and not find_if(args,isPE) then\r
+ -- no placeholders in these args, we can fold the expression.\r
+ local opfn = optable[op]\r
+ if opfn then\r
+ return opfn(unpack(args))\r
+ else\r
+ return '?'\r
+ end\r
+ elseif addmul then\r
+ -- enforce a few rules for + and *\r
+ -- split the args into two classes, PE args and non-PE args.\r
+ local classes = List.partition(args,isPE)\r
+ local pe,npe = classes[true],classes[false]\r
+ if npe then -- there's at least one non PE argument\r
+ -- so fold them\r
+ if #npe == 1 then npe = npe[1]\r
+ else npe = npe:reduce(optable[op])\r
+ end\r
+ -- if the result is a constant, return it\r
+ if not pe then return npe end\r
+\r
+ -- either (* 1 x) => x or (* 1 x y ...) => (* x y ...)\r
+ if op == '*' then\r
+ if npe == 0 then return 0\r
+ elseif npe == 1 then -- identity\r
+ if #pe == 1 then return pe[1] else npe = nil end\r
+ end\r
+ else -- special cases for +\r
+ if npe == 0 then -- identity\r
+ if #pe == 1 then return pe[1] else npe = nil end\r
+ end\r
+ end\r
+ end\r
+ -- build up the final arguments\r
+ local res = {}\r
+ if npe then append(res,npe) end\r
+ for val,count in pairs(count_map(pe,equals)) do\r
+ if count > 1 then\r
+ if op == '*' then val = val ^ count\r
+ else val = val * count\r
+ end\r
+ end\r
+ append(res,val)\r
+ end\r
+ if #res == 1 then return res[1] end\r
+ return PE{op=op,unpack(res)}\r
+ elseif op == '^' then\r
+ if args[2] == 1 then return args[1] end -- identity\r
+ if args[2] == 0 then return 1 end\r
+ end\r
+ return PE{op=op,unpack(args)}\r
+ end\r
+ else\r
+ return e\r
+ end\r
+end\r
+\r
+function expand (e)\r
+ if isPE(e) and e.op == '*' and isPE(e[2]) and e[2].op == '+' then\r
+ local a,b = e[1],e[2]\r
+ return expand(b[1]*a) + expand(b[2]*a)\r
+ else\r
+ return e\r
+ end\r
+end\r
+\r
+function isnumber (x)\r
+ return type(x) == 'number'\r
+end\r
+\r
+-- does this PE contain a reference to x?\r
+function references (e,x)\r
+ if isPE(e) then\r
+ if e.op == 'X' then return x.repr == e.repr\r
+ else\r
+ return find_if(e,references,x)\r
+ end\r
+ else\r
+ return false\r
+ end\r
+end\r
+\r
+local function muli (args)\r
+ return PE{op='*',unpack(args)}\r
+end\r
+\r
+local function addi (args)\r
+ return PE{op='+',unpack(args)}\r
+end\r
+\r
+function diff (e,x)\r
+ if isPE(e) and references(e,x) then\r
+ local op = e.op\r
+ if op == 'X' then\r
+ return 1\r
+ else\r
+ local a,b = e[1],e[2]\r
+ if op == '+' then -- differentiation is linear\r
+ local args = imap(diff,e,x)\r
+ return balance(addi(args))\r
+ elseif op == '*' then -- product rule\r
+ local res,d,ee = {}\r
+ for i = 1,#e do\r
+ d = fold(diff(e[i],x))\r
+ if d ~= 0 then\r
+ ee = {unpack(e)}\r
+ ee[i] = d\r
+ append(res,balance(muli(ee)))\r
+ end\r
+ end\r
+ if #res > 1 then return addi(res)\r
+ else return res[1] end\r
+ elseif op == '^' and isnumber(b) then -- power rule\r
+ return b*x^(b-1)\r
+ end\r
+ end\r
+ else\r
+ return 0\r
+ end\r
+end\r
+\r
+\r
+\r
--- /dev/null
+A = require 'pl.tablex'\r
+ops = require 'pl.operator'\r
+print(A.compare_no_order({1,2,3},{2,1,3}))\r
+print(A.compare_no_order({1,2,3},{2,1,3},'=='))\r
--- /dev/null
+-- demonstrates how to use a list of callbacks\r
+require 'pl'\r
+actions = List()\r
+L = utils.string_lambda\r
+\r
+actions:append(function() print 'hello' end)\r
+actions:append(L '|| print "yay"')\r
+\r
+-- '()' is a shortcut for operator.call or function(x) return x() end\r
+actions:foreach '()'\r
--- /dev/null
+local pretty = require 'pl.pretty'\r
+\r
+tb = {\r
+ 'one','two','three',{1,2,3},\r
+ alpha=1,beta=2,gamma=3,['&']=true,[0]=false,\r
+ _fred = {true,true},\r
+ s = [[\r
+hello dolly\r
+you're so fine\r
+]]\r
+}\r
+\r
+print(pretty.write(tb))\r
--- /dev/null
+require 'pl'\r
+-- force us to look in the script's directory when requiring...\r
+app.require_here()\r
+require 'symbols'\r
+\r
+local MT = getmetatable(_1)\r
+\r
+add = MT.__add\r
+mul = MT.__mul\r
+pow = MT.__pow\r
+\r
+\r
+function testeq (e1,e2)\r
+ if not equals(e1,e2) then\r
+ print ('Not equal',repr(e1),repr(e2))\r
+ end\r
+end\r
+\r
+sin = register(math.sin,'sin')\r
+\r
+f = register(function(x,y,z) end)\r
+\r
+--[[\r
+testeq (_1,_1)\r
+testeq (_1+_2,_1+_2)\r
+testeq (_1 + 3*_2,_1 + 3*_2)\r
+testeq (_2+_1,_1+_2)\r
+testeq (sin(_1),sin(_1))\r
+testeq (1+f(10,20,'ok'),f(10,20,'ok')+1)\r
+--]]\r
+\r
+\r
+function testexpand (e)\r
+ print(repr(fold(expand(e)))) --fold\r
+end\r
+\r
+--[[\r
+testexpand (a*(a+1))\r
+\r
+testexpand ((x+2)*(b+1))\r
+]]--\r
+\r
+function testfold (e)\r
+ print(repr(fold(e)))\r
+end\r
+\r
+a,b,c,x,y = Var 'a,b,c,x,y'\r
+\r
+--~ testfold(_1 + _2)\r
+--~ testfold(add(10,20))\r
+--~ testfold(add(mul(2,_1),mul(3,_2)))\r
+--[[\r
+testfold(sin(a))\r
+e = a^(b+2)\r
+testfold(e)\r
+bindval(b,1)\r
+testfold(e)\r
+bindval(a,2)\r
+testfold(e)\r
+\r
+bindval(a)\r
+bindval(b)\r
+]]\r
+\r
+\r
+\r
+function testdiff (e)\r
+ balance(e)\r
+ e = diff(e,x)\r
+ balance(e)\r
+ print('+ ',e)\r
+ e = fold(e)\r
+ print('- ',e)\r
+end\r
+\r
+\r
+testdiff(x^2+1)\r
+testdiff(3*x^2)\r
+testdiff(x^2 + 2*x^3)\r
+testdiff(x^2 + 2*a*x^3 + x^4)\r
+testdiff(2*a*x^3)\r
+testdiff(x*x*x)\r
+\r
+\r
+\r
--- /dev/null
+-- shows how a script can get a private file path\r
+-- the output on my Windows machine is:\r
+-- C:\Documents and Settings\steve\.testapp\test.txt\r
+require 'pl'\r
+print(app.appfile 'test.txt')\r
--- /dev/null
+--cloning a directory tree.\r
+\r
+require 'pl'\r
+p1 = [[examples]]\r
+p2 = [[copy/of/examples]]\r
+\r
+if not path.isfile 'examples/testclone.lua' then\r
+ return print 'please run this in the penlight folder (below examples)'\r
+end\r
+\r
+-- make a copy of the examples folder\r
+dir.clonetree(p1,p2,dir.copyfile)\r
+\r
+assert(path.isdir 'copy')\r
+\r
+print '---'\r
+t = os.time()\r
+print(lfs.touch('examples/testclone.lua',t,t+10))\r
+\r
+-- this should only update this file\r
+dir.clonetree(p1,p2,\r
+function(f1,f2)\r
+ local t1 = path.getmtime(f1)\r
+ local t2 = path.getmtime(f2)\r
+ --print(f1,t1,f2,t2)\r
+ if t1 > t2 then\r
+ dir.copyfile(f1,f2)\r
+ print(f1,f2,t1,t2)\r
+ end\r
+ return true\r
+end)\r
+\r
+-- and get rid of the whole copy directory, with subdirs\r
+dir.rmtree 'copy'\r
+\r
+assert(not path.exists 'copy')\r
+\r
+\r
--- /dev/null
+local stringio = require 'pl.stringio'
+local config = require 'pl.config'
+
+function dump(t,indent)
+ if type(t) == 'table' then
+ io.write(indent,'{\n')
+ local newindent = indent..' '
+ for k,v in pairs(t) do
+ io.write(newindent,k,'=')
+ dump(v,indent)
+ io.write('\n')
+ end
+ io.write(newindent,'},\n')
+ else
+ io.write(indent,t,'(',type(t),')')
+ end
+end
+
+
+function testconfig(test)
+ local f = stringio.open(test)
+ local c = config.read(f)
+ f:close()
+ dump(c,' ')
+ print '-----'
+end
+
+testconfig [[
+ ; comment 2 (an ini file)
+[section!]
+bonzo.dog=20,30
+config_parm=here we go again
+depth = 2
+[another]
+felix="cat"
+]]
+
+testconfig [[
+# this is a more Unix-y config file
+fred = 1
+alice = 2
+home = /bonzo/dog/etc
+]]
+
+testconfig [[
+# this is just a set of comma-separated values
+1000,444,222
+44,555,224
+]]
+
+
--- /dev/null
+-- very simple lexer program which looks at all identifiers in a Lua\r
+-- file and checks whether they're in the global namespace.\r
+-- At the end, we dump out the result of count_map, which will give us\r
+-- unique identifiers with their usage count.\r
+-- (an example of a program which itself needs to be careful about what\r
+-- goes into the global namespace)\r
+\r
+local utils = require 'pl.utils'\r
+local file = require 'pl.file'\r
+local lexer = require 'pl.lexer'\r
+local List = require 'pl.List'\r
+local pretty = require 'pl.pretty'\r
+local seq = require 'pl.seq'\r
+\r
+utils.on_error 'quit'\r
+\r
+local txt,err = file.read(arg[1] or 'testglobal.lua')\r
+local globals = List()\r
+for t,v in lexer.lua(txt) do\r
+ if t == 'iden' and rawget(_G,v) then\r
+ globals:append(v)\r
+ end\r
+end\r
+\r
+pretty.dump(seq.count_map(globals))\r
+\r
+\r
--- /dev/null
+require 'pl'\r
+local sum = 0.0\r
+local count = 0\r
+local text = [[\r
+ 981124001 2.0 18988.4 10047.1 4149.7\r
+ 981125001 0.8 19104.0 9970.4 5088.7\r
+ 981127003 0.5 19012.5 9946.9 3831.2\r
+]]\r
+for id,magn,x in input.fields(3,' ',text) do\r
+ sum = sum + x\r
+ count = count + 1\r
+end\r
+print('average x coord is ',sum/count)\r
--- /dev/null
+require 'pl'\r
+local text = [[\r
+ 981124001 2.0 18988.4 10047.1 4149.7\r
+ 981125001 0.8 19104.0 9970.4 5088.7\r
+ 981127003 0.5 19012.5 9946.9 3831.2\r
+]]\r
+local sum,count = seq.sum(input.fields ({3},' ',text))\r
+print(sum/count)\r
--- /dev/null
+-- an example showing 'pl.lexer' doing some serious work.\r
+-- The resulting Lua table is in the same LOM format used by luaexpat.\r
+-- This is (clearly) not a professional XML parser, so don't use it\r
+-- on your homework!\r
+\r
+require 'pl'\r
+\r
+local append = table.insert\r
+local skipws,expecting = lexer.skipws,lexer.expecting\r
+\r
+function parse_element (tok,tag)\r
+ local tbl,t,v,attrib\r
+ tbl = {}\r
+ tbl.tag = tag -- LOM 'tag' is the element tag\r
+ t,v = skipws(tok)\r
+ while v ~= '/' and v ~= '>' do\r
+ if t ~= 'iden' then error('expecting attribute identifier') end\r
+ attrib = v\r
+ expecting(tok,'=')\r
+ v = expecting(tok,'string')\r
+ -- LOM: 'attr' subtable contains attrib/value pairs and an ordered list of attribs\r
+ if not tbl.attr then tbl.attr = {} end\r
+ tbl.attr[attrib] = v\r
+ append(tbl.attr,attrib)\r
+ t,v = skipws(tok)\r
+ end\r
+ if v == '/' then\r
+ expecting(tok,'>')\r
+ return tbl\r
+ end\r
+ -- pick up element data\r
+ t,v = tok()\r
+ while true do\r
+ if t == '<' then\r
+ t,v = skipws(tok)\r
+ if t == '/' then -- element end tag\r
+ t,v = tok()\r
+ if t == '>' then return tbl end\r
+ if t == 'iden' and v == tag then\r
+ if tok() == '>' then return tbl end\r
+ end\r
+ error('expecting end tag '..tag)\r
+ else\r
+ append(tbl,parse_element(tok,v)) -- LOM: child elements added to table\r
+ t,v = skipws(tok)\r
+ end\r
+ else\r
+ append(tbl,v) -- LOM: text added to table\r
+ t,v = skipws(tok)\r
+ end\r
+ end\r
+end\r
+\r
+function parse_xml (tok)\r
+ local t,v = skipws(tok)\r
+ while t == '<' do\r
+ t,v = tok()\r
+ if t == '?' or t == '!' then\r
+ -- skip meta stuff and commentary\r
+ repeat t = tok() until t == '>'\r
+ t,v = expecting(tok,'<')\r
+ else\r
+ return parse_element(tok,v)\r
+ end\r
+ end\r
+end\r
+\r
+s = [[\r
+<?xml version="1.0" encoding="UTF-8"?>\r
+<sensor name="closure-meter-2" id="7D7D0600006F0D00" loc="100,100,0" device="closure-meter" init="true">\r
+<detector name="closure-meter" phenomenon="closure" units="mm" id="1"\r
+ vmin="0" vmax="5000" device="closure-meter" calib="0,0;5000,5000"\r
+ sampling_interval="25000" measurement_interval="600000"\r
+/>\r
+</sensor>\r
+]]\r
+\r
+local tok = lexer.scan(s,nil,{space=false},{string=true})\r
+local res = parse_xml(tok)\r
+print(pretty.write(res))\r
+\r
--- /dev/null
+-- a simple implementation of the which command. This looks for
+-- the given file on the path. On windows, it will assume an extension
+-- of .exe if no extension is given.
+local List = require 'pl.List'
+local path = require 'pl.path'
+local app = require 'pl.app'
+
+local pathl = List.split(os.getenv 'PATH',path.dirsep)
+
+function which (file)
+ local res = pathl:map(path.join,file)
+ res = res:filter(path.exists)
+ if res then return res[1] end
+end
+
+local _,lua = app.lua()
+local file = arg[1] or lua -- i.e. location of lua executable
+local try
+
+if not file then return print 'must provide a filename' end
+
+if path.extension(file) == '' and path.is_windows then
+ try = which(file..'.exe')
+else
+ try = which(file)
+end
+
+if try then print(try) else print 'cannot find on path' end
+
+
--- /dev/null
+--- Date and Date Format classes.
+-- See @{05-dates.md|the Guide}.
+--
+-- Dependencies: `pl.class`, `pl.stringx`
+-- @module pl.Date
+-- @pragma nostrip
+
+local class = require 'pl.class'
+local os_time, os_date = os.time, os.date
+local stringx = require 'pl.stringx'
+
+local Date = class()
+Date.Format = class()
+
+--- Date constructor.
+-- @param t this can be either <ul>
+-- <li>nil - use current date and time</li>
+-- <li>number - seconds since epoch (as returned by @{os.time})</li>
+-- <li>Date - copy constructor</li>
+-- <li>table - table containing year, month, etc as for os.time()
+-- You may leave out year, month or day, in which case current values will be used.
+-- </li>
+-- <li> two to six numbers: year, month, day, hour, min, sec
+-- </ul>
+-- @function Date
+function Date:_init(t,...)
+ local time
+ if select('#',...) > 0 then
+ local extra = {...}
+ local year = t
+ t = {
+ year = year,
+ month = extra[1],
+ day = extra[2],
+ hour = extra[3],
+ min = extra[4],
+ sec = extra[5]
+ }
+ end
+ if t == nil then
+ time = os_time()
+ elseif type(t) == 'number' then
+ time = t
+ elseif type(t) == 'table' then
+ if getmetatable(t) == Date then -- copy ctor
+ time = t.time
+ else
+ if not (t.year and t.month and t.year) then
+ local lt = os.date('*t')
+ if not t.year and not t.month and not t.day then
+ t.year = lt.year
+ t.month = lt.month
+ t.day = lt.day
+ else
+ t.year = t.year or lt.year
+ t.month = t.month or (t.day and lt.month or 1)
+ t.day = t.day or 1
+ end
+ end
+ time = os_time(t)
+ end
+ end
+ self:set(time)
+end
+
+local tzone_
+
+--- get the time zone offset from UTC.
+-- @return seconds ahead of UTC
+function Date.tzone ()
+ if not tzone_ then
+ local now = os.time()
+ local utc = os.date('!*t',now)
+ local lcl = os.date('*t',now)
+ local unow = os.time(utc)
+ tzone_ = os.difftime(now,unow)
+ if lcl.isdst then
+ if tzone_ > 0 then
+ tzone_ = tzone_ - 3600
+ else
+ tzone_ = tzone_ + 3600
+ end
+ end
+ end
+ return tzone_
+end
+
+--- convert this date to UTC.
+function Date:toUTC ()
+ self:add { sec = -Date.tzone() }
+end
+
+--- convert this UTC date to local.
+function Date:toLocal ()
+ self:add { sec = Date.tzone() }
+end
+
+--- set the current time of this Date object.
+-- @param t seconds since epoch
+function Date:set(t)
+ self.time = t
+ self.tab = os_date('*t',self.time)
+end
+
+--- set the year.
+-- @param y Four-digit year
+-- @class function
+-- @name Date:year
+
+--- set the month.
+-- @param m month
+-- @class function
+-- @name Date:month
+
+--- set the day.
+-- @param d day
+-- @class function
+-- @name Date:day
+
+--- set the hour.
+-- @param h hour
+-- @class function
+-- @name Date:hour
+
+--- set the minutes.
+-- @param min minutes
+-- @class function
+-- @name Date:min
+
+--- set the seconds.
+-- @param sec seconds
+-- @class function
+-- @name Date:sec
+
+--- set the day of year.
+-- @class function
+-- @param yday day of year
+-- @name Date:yday
+
+--- get the year.
+-- @param y Four-digit year
+-- @class function
+-- @name Date:year
+
+--- get the month.
+-- @class function
+-- @name Date:month
+
+--- get the day.
+-- @class function
+-- @name Date:day
+
+--- get the hour.
+-- @class function
+-- @name Date:hour
+
+--- get the minutes.
+-- @class function
+-- @name Date:min
+
+--- get the seconds.
+-- @class function
+-- @name Date:sec
+
+--- get the day of year.
+-- @class function
+-- @name Date:yday
+
+
+for _,c in ipairs{'year','month','day','hour','min','sec','yday'} do
+ Date[c] = function(self,val)
+ if val then
+ self.tab[c] = val
+ self:set(os_time(self.tab))
+ return self
+ else
+ return self.tab[c]
+ end
+ end
+end
+
+--- name of day of week.
+-- @param full abbreviated if true, full otherwise.
+-- @return string name
+function Date:weekday_name(full)
+ return os_date(full and '%A' or '%a',self.time)
+end
+
+--- name of month.
+-- @param full abbreviated if true, full otherwise.
+-- @return string name
+function Date:month_name(full)
+ return os_date(full and '%B' or '%b',self.time)
+end
+
+--- is this day on a weekend?.
+function Date:is_weekend()
+ return self.tab.wday == 0 or self.tab.wday == 6
+end
+
+--- add to a date object.
+-- @param t a table containing one of the following keys and a value:<br>
+-- year,month,day,hour,min,sec
+-- @return this date
+function Date:add(t)
+ local key,val = next(t)
+ self.tab[key] = self.tab[key] + val
+ self:set(os_time(self.tab))
+ return self
+end
+
+--- last day of the month.
+-- @return int day
+function Date:last_day()
+ local d = 28
+ local m = self.tab.month
+ while self.tab.month == m do
+ d = d + 1
+ self:add{day=1}
+ end
+ self:add{day=-1}
+ return self
+end
+
+--- difference between two Date objects.
+-- Note: currently the result is a regular @{Date} object,
+-- but also has `interval` field set, which means a more
+-- appropriate string rep is used.
+-- @param other Date object
+-- @return a Date object
+function Date:diff(other)
+ local dt = self.time - other.time
+ if dt < 0 then error("date difference is negative!",2) end
+ local date = Date(dt)
+ date.interval = true
+ return date
+end
+
+--- long numerical ISO data format version of this date.
+function Date:__tostring()
+ if not self.interval then
+ return os_date('%Y-%m-%d %H:%M:%S',self.time)
+ else
+ local t, res = self.tab, ''
+ local y,m,d = t.year - 1970, t.month - 1, t.day - 1
+ if y > 0 then res = res .. y .. ' years ' end
+ if m > 0 then res = res .. m .. ' months ' end
+ if d > 0 then res = res .. d .. ' days ' end
+ if y == 0 and m == 0 then
+ local h = t.hour
+ if h > 0 then res = res .. h .. ' hours ' end
+ if t.min > 0 then res = res .. t.min .. ' min ' end
+ if t.sec > 0 then res = res .. t.sec .. ' sec ' end
+ end
+ return res
+ end
+end
+
+--- equality between Date objects.
+function Date:__eq(other)
+ return self.time == other.time
+end
+
+--- equality between Date objects.
+function Date:__lt(other)
+ return self.time < other.time
+end
+
+
+------------ Date.Format class: parsing and renderinig dates ------------
+
+-- short field names, explicit os.date names, and a mask for allowed field repeats
+local formats = {
+ d = {'day',{true,true}},
+ y = {'year',{false,true,false,true}},
+ m = {'month',{true,true}},
+ H = {'hour',{true,true}},
+ M = {'min',{true,true}},
+ S = {'sec',{true,true}},
+}
+
+--
+
+--- Date.Format constructor.
+-- @param fmt. A string where the following fields are significant: <ul>
+-- <li>d day (either d or dd)</li>
+-- <li>y year (either yy or yyy)</li>
+-- <li>m month (either m or mm)</li>
+-- <li>H hour (either H or HH)</li>
+-- <li>M minute (either M or MM)</li>
+-- <li>S second (either S or SS)</li>
+-- </ul>
+-- Alternatively, if fmt is nil then this returns a flexible date parser
+-- that tries various date/time schemes in turn:
+-- <ol>
+-- <li> <a href="http://en.wikipedia.org/wiki/ISO_8601">ISO 8601</a>,
+-- like 2010-05-10 12:35:23Z or 2008-10-03T14:30+02<li>
+-- <li> times like 15:30 or 8.05pm (assumed to be today's date)</li>
+-- <li> dates like 28/10/02 (European order!) or 5 Feb 2012 </li>
+-- <li> month name like march or Mar (case-insensitive, first 3 letters);
+-- here the day will be 1 and the year this current year </li>
+-- </ol>
+-- A date in format 3 can be optionally followed by a time in format 2.
+-- Please see test-date.lua in the tests folder for more examples.
+-- @usage df = Date.Format("yyyy-mm-dd HH:MM:SS")
+-- @class function
+-- @name Date.Format
+function Date.Format:_init(fmt)
+ if not fmt then return end
+ local append = table.insert
+ local D,PLUS,OPENP,CLOSEP = '\001','\002','\003','\004'
+ local vars,used = {},{}
+ local patt,outf = {},{}
+ local i = 1
+ while i < #fmt do
+ local ch = fmt:sub(i,i)
+ local df = formats[ch]
+ if df then
+ if used[ch] then error("field appeared twice: "..ch,2) end
+ used[ch] = true
+ -- this field may be repeated
+ local _,inext = fmt:find(ch..'+',i+1)
+ local cnt = not _ and 1 or inext-i+1
+ if not df[2][cnt] then error("wrong number of fields: "..ch,2) end
+ -- single chars mean 'accept more than one digit'
+ local p = cnt==1 and (D..PLUS) or (D):rep(cnt)
+ append(patt,OPENP..p..CLOSEP)
+ append(vars,ch)
+ if ch == 'y' then
+ append(outf,cnt==2 and '%y' or '%Y')
+ else
+ append(outf,'%'..ch)
+ end
+ i = i + cnt
+ else
+ append(patt,ch)
+ append(outf,ch)
+ i = i + 1
+ end
+ end
+ -- escape any magic characters
+ fmt = table.concat(patt):gsub('[%-%.%+%[%]%(%)%$%^%%%?%*]','%%%1')
+ -- replace markers with their magic equivalents
+ fmt = fmt:gsub(D,'%%d'):gsub(PLUS,'+'):gsub(OPENP,'('):gsub(CLOSEP,')')
+ self.fmt = fmt
+ self.outf = table.concat(outf)
+ self.vars = vars
+
+end
+
+local parse_date
+
+--- parse a string into a Date object.
+-- @param str a date string
+-- @return date object
+function Date.Format:parse(str)
+ if not self.fmt then
+ return parse_date(str,self.us)
+ end
+ local res = {str:match(self.fmt)}
+ if #res==0 then return nil, 'cannot parse '..str end
+ local tab = {}
+ for i,v in ipairs(self.vars) do
+ local name = formats[v][1] -- e.g. 'y' becomes 'year'
+ tab[name] = tonumber(res[i])
+ end
+ -- os.date() requires these fields; if not present, we assume
+ -- that the time set is for the current day.
+ if not (tab.year and tab.month and tab.year) then
+ local today = Date()
+ tab.year = tab.year or today:year()
+ tab.month = tab.month or today:month()
+ tab.day = tab.day or today:month()
+ end
+ local Y = tab.year
+ if Y < 100 then -- classic Y2K pivot
+ tab.year = Y + (Y < 35 and 2000 or 1999)
+ elseif not Y then
+ tab.year = 1970
+ end
+ --dump(tab)
+ return Date(tab)
+end
+
+--- convert a Date object into a string.
+-- @param d a date object, or a time value as returned by @{os.time}
+-- @return string
+function Date.Format:tostring(d)
+ local tm = type(d) == 'number' and d or d.time
+ if self.outf then
+ return os.date(self.outf,tm)
+ else
+ return tostring(Date(d))
+ end
+end
+
+function Date.Format:US_order(yesno)
+ self.us = yesno
+end
+
+local months = {jan=1,feb=2,mar=3,apr=4,may=5,jun=6,jul=7,aug=8,sep=9,oct=10,nov=11,dec=12}
+
+--[[
+Allowed patterns:
+- [day] [monthname] [year] [time]
+- [day]/[month][/year] [time]
+
+]]
+
+
+local is_word = stringx.isalpha
+local is_number = stringx.isdigit
+local function tonum(s,l1,l2,kind)
+ kind = kind or ''
+ local n = tonumber(s)
+ if not n then error(("%snot a number: '%s'"):format(kind,s)) end
+ if n < l1 or n > l2 then
+ error(("%s out of range: %s is not between %d and %d"):format(kind,s,l1,l2))
+ end
+ return n
+end
+
+local function parse_iso_end(p,ns,sec)
+ -- may be fractional part of seconds
+ local _,nfrac,secfrac = p:find('^%.%d+',ns+1)
+ if secfrac then
+ sec = sec .. secfrac
+ p = p:sub(nfrac+1)
+ else
+ p = p:sub(ns+1)
+ end
+ -- ISO 8601 dates may end in Z (for UTC) or [+-][isotime]
+ -- (we're working with the date as lower case, hence 'z')
+ if p:match 'z$' then return sec, {h=0,m=0} end -- we're UTC!
+ p = p:gsub(':','') -- turn 00:30 to 0030
+ local _,_,sign,offs = p:find('^([%+%-])(%d+)')
+ if not sign then return sec, nil end -- not UTC
+
+ if #offs == 2 then offs = offs .. '00' end -- 01 to 0100
+ local tz = { h = tonumber(offs:sub(1,2)), m = tonumber(offs:sub(3,4)) }
+ if sign == '-' then tz.h = -tz.h; tz.m = -tz.m end
+ return sec, tz
+end
+
+local function parse_date_unsafe (s,US)
+ s = s:gsub('T',' ') -- ISO 8601
+ local parts = stringx.split(s:lower())
+ local i,p = 1,parts[1]
+ local function nextp() i = i + 1; p = parts[i] end
+ local year,min,hour,sec,apm
+ local tz
+ local _,nxt,day, month = p:find '^(%d+)/(%d+)'
+ if day then
+ -- swop for US case
+ if US then
+ day, month = month, day
+ end
+ _,_,year = p:find('^/(%d+)',nxt+1)
+ nextp()
+ else -- ISO
+ year,month,day = p:match('^(%d+)%-(%d+)%-(%d+)')
+ if year then
+ nextp()
+ end
+ end
+ if p and not year and is_number(p) then -- has to be date
+ day = p
+ nextp()
+ end
+ if p and is_word(p) then
+ p = p:sub(1,3)
+ local mon = months[p]
+ if mon then
+ month = mon
+ else error("not a month: " .. p) end
+ nextp()
+ end
+ if p and not year and is_number(p) then
+ year = p
+ nextp()
+ end
+
+ if p then -- time is hh:mm[:ss], hhmm[ss] or H.M[am|pm]
+ _,nxt,hour,min = p:find '^(%d+):(%d+)'
+ local ns
+ if nxt then -- are there seconds?
+ _,ns,sec = p:find ('^:(%d+)',nxt+1)
+ --if ns then
+ sec,tz = parse_iso_end(p,ns or nxt,sec)
+ --end
+ else -- might be h.m
+ _,ns,hour,min = p:find '^(%d+)%.(%d+)'
+ if ns then
+ apm = p:match '[ap]m$'
+ else -- or hhmm[ss]
+ local hourmin
+ _,nxt,hourmin = p:find ('^(%d+)')
+ if nxt then
+ hour = hourmin:sub(1,2)
+ min = hourmin:sub(3,4)
+ sec = hourmin:sub(5,6)
+ if #sec == 0 then sec = nil end
+ sec,tz = parse_iso_end(p,nxt,sec)
+ end
+ end
+ end
+ end
+ local today
+ if not (year and month and day) then
+ today = Date()
+ end
+ day = day and tonum(day,1,31,'day') or (month and 1 or today:day())
+ month = month and tonum(month,1,12,'month') or today:month()
+ year = year and tonumber(year) or today:year()
+ if year < 100 then -- two-digit year pivot around year < 2035
+ year = year + (year < 35 and 2000 or 1900)
+ end
+ hour = hour and tonum(hour,0,apm and 12 or 24,'hour') or 12
+ if apm == 'pm' then
+ hour = hour + 12
+ end
+ min = min and tonum(min,0,59) or 0
+ sec = sec and tonum(sec,0,60) or 0 --60 used to indicate leap second
+ local res = Date {year = year, month = month, day = day, hour = hour, min = min, sec = sec}
+ if tz then -- ISO 8601 UTC time
+ res:add {hour = -tz.h}
+ if tz.m ~= 0 then res:add {min = -tz.m} end
+ -- we're in UTC, so let's go local...
+ res:toLocal()
+ end
+ return res
+end
+
+function parse_date (s)
+ local ok, d = pcall(parse_date_unsafe,s)
+ if not ok then -- error
+ d = d:gsub('.-:%d+: ','')
+ return nil, d
+ else
+ return d
+ end
+end
+
+
+return Date
+
--- /dev/null
+--- Python-style list class.
+--
+-- **Please Note**: methods that change the list will return the list.
+-- This is to allow for method chaining, but please note that `ls = ls:sort()`
+-- does not mean that a new copy of the list is made. In-place (mutable) methods
+-- are marked as returning 'the list' in this documentation.
+--
+-- See the Guide for further @{02-arrays.md.Python_style_Lists|discussion}
+--
+-- See <a href="http://www.python.org/doc/current/tut/tut.html">http://www.python.org/doc/current/tut/tut.html</a>, section 5.1
+--
+-- **Note**: The comments before some of the functions are from the Python docs
+-- and contain Python code.
+--
+-- Written for Lua version Nick Trout 4.0; Redone for Lua 5.1, Steve Donovan.
+--
+-- Dependencies: `pl.utils`, `pl.tablex`
+-- @module pl.List
+-- @pragma nostrip
+
+local tinsert,tremove,concat,tsort = table.insert,table.remove,table.concat,table.sort
+local setmetatable, getmetatable,type,tostring,assert,string,next = setmetatable,getmetatable,type,tostring,assert,string,next
+local write = io.write
+local tablex = require 'pl.tablex'
+local filter,imap,imap2,reduce,transform,tremovevalues = tablex.filter,tablex.imap,tablex.imap2,tablex.reduce,tablex.transform,tablex.removevalues
+local tablex = tablex
+local tsub = tablex.sub
+local utils = require 'pl.utils'
+local function_arg = utils.function_arg
+local is_type = utils.is_type
+local split = utils.split
+local assert_arg = utils.assert_arg
+local normalize_slice = tablex._normalize_slice
+
+--[[
+module ('pl.List',utils._module)
+]]
+
+local Multimap = utils.stdmt.MultiMap
+-- metatable for our list objects
+local List = utils.stdmt.List
+List.__index = List
+List._class = List
+
+local iter
+
+-- we give the metatable its own metatable so that we can call it like a function!
+setmetatable(List,{
+ __call = function (tbl,arg)
+ return List.new(arg)
+ end,
+})
+
+local function makelist (t,obj)
+ local klass = List
+ if obj then
+ klass = getmetatable(obj)
+ end
+ return setmetatable(t,klass)
+end
+
+local function is_list(t)
+ return getmetatable(t) == List
+end
+
+local function simple_table(t)
+ return type(t) == 'table' and not is_list(t) and #t > 0
+end
+
+function List:_init (src)
+ if src then
+ for v in iter(src) do
+ tinsert(self,v)
+ end
+ end
+end
+
+--- Create a new list. Can optionally pass a table;
+-- passing another instance of List will cause a copy to be created
+-- we pass anything which isn't a simple table to iterate() to work out
+-- an appropriate iterator @see List.iterate
+-- @param t An optional list-like table
+-- @return a new List
+-- @usage ls = List(); ls = List {1,2,3,4}
+function List.new(t)
+ local ls
+ if not simple_table(t) then
+ ls = {}
+ List._init(ls,t)
+ else
+ ls = t
+ end
+ makelist(ls)
+ return ls
+end
+
+function List:clone()
+ local ls = makelist({},self)
+ List._init(ls,self)
+ return ls
+end
+
+function List.default_map_with(T)
+ return function(self,name)
+ local f = T[name]
+ if f then
+ return function(self,...)
+ return self:map(f,...)
+ end
+ else
+ error("method not found: "..name,2)
+ end
+ end
+end
+
+
+---Add an item to the end of the list.
+-- @param i An item
+-- @return the list
+function List:append(i)
+ tinsert(self,i)
+ return self
+end
+
+List.push = tinsert
+
+--- Extend the list by appending all the items in the given list.
+-- equivalent to 'a[len(a):] = L'.
+-- @param L Another List
+-- @return the list
+function List:extend(L)
+ assert_arg(1,L,'table')
+ for i = 1,#L do tinsert(self,L[i]) end
+ return self
+end
+
+--- Insert an item at a given position. i is the index of the
+-- element before which to insert.
+-- @param i index of element before whichh to insert
+-- @param x A data item
+-- @return the list
+function List:insert(i, x)
+ assert_arg(1,i,'number')
+ tinsert(self,i,x)
+ return self
+end
+
+--- Insert an item at the begining of the list.
+-- @param x a data item
+-- @return the list
+function List:put (x)
+ return self:insert(1,x)
+end
+
+--- Remove an element given its index.
+-- (equivalent of Python's del s[i])
+-- @param i the index
+-- @return the list
+function List:remove (i)
+ assert_arg(1,i,'number')
+ tremove(self,i)
+ return self
+end
+
+--- Remove the first item from the list whose value is given.
+-- (This is called 'remove' in Python; renamed to avoid confusion
+-- with table.remove)
+-- Return nil if there is no such item.
+-- @param x A data value
+-- @return the list
+function List:remove_value(x)
+ for i=1,#self do
+ if self[i]==x then tremove(self,i) return self end
+ end
+ return self
+ end
+
+--- Remove the item at the given position in the list, and return it.
+-- If no index is specified, a:pop() returns the last item in the list.
+-- The item is also removed from the list.
+-- @param i An index
+-- @return the item
+function List:pop(i)
+ if not i then i = #self end
+ assert_arg(1,i,'number')
+ return tremove(self,i)
+end
+
+List.get = List.pop
+
+--- Return the index in the list of the first item whose value is given.
+-- Return nil if there is no such item.
+-- @class function
+-- @name List:index
+-- @param x A data value
+-- @param idx where to start search (default 1)
+-- @return the index, or nil if not found.
+
+local tfind = tablex.find
+List.index = tfind
+
+--- does this list contain the value?.
+-- @param x A data value
+-- @return true or false
+function List:contains(x)
+ return tfind(self,x) and true or false
+end
+
+--- Return the number of times value appears in the list.
+-- @param x A data value
+-- @return number of times x appears
+function List:count(x)
+ local cnt=0
+ for i=1,#self do
+ if self[i]==x then cnt=cnt+1 end
+ end
+ return cnt
+end
+
+--- Sort the items of the list, in place.
+-- @param cmp an optional comparison function, default '<'
+-- @return the list
+function List:sort(cmp)
+ if cmp then cmp = function_arg(1,cmp) end
+ tsort(self,cmp)
+ return self
+end
+
+--- return a sorted copy of this list.
+-- @param cmp an optional comparison function, default '<'
+-- @return a new list
+function List:sorted(cmp)
+ return List(self):sort(cmp)
+end
+
+--- Reverse the elements of the list, in place.
+-- @return the list
+function List:reverse()
+ local t = self
+ local n = #t
+ local n2 = n/2
+ for i = 1,n2 do
+ local k = n-i+1
+ t[i],t[k] = t[k],t[i]
+ end
+ return self
+end
+
+--- return the minimum and the maximum value of the list.
+-- @return minimum value
+-- @return maximum value
+function List:minmax()
+ local vmin,vmax = 1e70,-1e70
+ for i = 1,#self do
+ local v = self[i]
+ if v < vmin then vmin = v end
+ if v > vmax then vmax = v end
+ end
+ return vmin,vmax
+end
+
+--- Emulate list slicing. like 'list[first:last]' in Python.
+-- If first or last are negative then they are relative to the end of the list
+-- eg. slice(-2) gives last 2 entries in a list, and
+-- slice(-4,-2) gives from -4th to -2nd
+-- @param first An index
+-- @param last An index
+-- @return a new List
+function List:slice(first,last)
+ return tsub(self,first,last)
+end
+
+--- empty the list.
+-- @return the list
+function List:clear()
+ for i=1,#self do tremove(self) end
+ return self
+end
+
+local eps = 1.0e-10
+
+--- Emulate Python's range(x) function.
+-- Include it in List table for tidiness
+-- @param start A number
+-- @param finish A number greater than start; if zero, then 0..start-1
+-- @param incr an optional increment (may be less than 1)
+-- @usage List.range(0,3) == List {0,1,2,3}
+function List.range(start,finish,incr)
+ if not finish then
+ start = 0
+ finish = finish - 1
+ end
+ if incr then
+ if not utils.is_integer(incr) then finish = finish + eps end
+ else
+ incr = 1
+ end
+ assert_arg(1,start,'number')
+ assert_arg(2,finish,'number')
+ local t = List.new()
+ for i=start,finish,incr do tinsert(t,i) end
+ return t
+end
+
+--- list:len() is the same as #list.
+function List:len()
+ return #self
+end
+
+-- Extended operations --
+
+--- Remove a subrange of elements.
+-- equivalent to 'del s[i1:i2]' in Python.
+-- @param i1 start of range
+-- @param i2 end of range
+-- @return the list
+function List:chop(i1,i2)
+ return tremovevalues(self,i1,i2)
+end
+
+--- Insert a sublist into a list
+-- equivalent to 's[idx:idx] = list' in Python
+-- @param idx index
+-- @param list list to insert
+-- @return the list
+-- @usage l = List{10,20}; l:splice(2,{21,22}); assert(l == List{10,21,22,20})
+function List:splice(idx,list)
+ assert_arg(1,idx,'number')
+ idx = idx - 1
+ local i = 1
+ for v in iter(list) do
+ tinsert(self,i+idx,v)
+ i = i + 1
+ end
+ return self
+end
+
+--- general slice assignment s[i1:i2] = seq.
+-- @param i1 start index
+-- @param i2 end index
+-- @param seq a list
+-- @return the list
+function List:slice_assign(i1,i2,seq)
+ assert_arg(1,i1,'number')
+ assert_arg(1,i2,'number')
+ i1,i2 = normalize_slice(self,i1,i2)
+ if i2 >= i1 then self:chop(i1,i2) end
+ self:splice(i1,seq)
+ return self
+end
+
+--- concatenation operator.
+-- @param L another List
+-- @return a new list consisting of the list with the elements of the new list appended
+function List:__concat(L)
+ assert_arg(1,L,'table')
+ local ls = self:clone()
+ ls:extend(L)
+ return ls
+end
+
+--- equality operator ==. True iff all elements of two lists are equal.
+-- @param L another List
+-- @return true or false
+function List:__eq(L)
+ if #self ~= #L then return false end
+ for i = 1,#self do
+ if self[i] ~= L[i] then return false end
+ end
+ return true
+end
+
+--- join the elements of a list using a delimiter. <br>
+-- This method uses tostring on all elements.
+-- @param delim a delimiter string, can be empty.
+-- @return a string
+function List:join (delim)
+ delim = delim or ''
+ assert_arg(1,delim,'string')
+ return concat(imap(tostring,self),delim)
+end
+
+--- join a list of strings. <br>
+-- Uses table.concat directly.
+-- @class function
+-- @name List:concat
+-- @param delim a delimiter
+-- @return a string
+List.concat = concat
+
+local function tostring_q(val)
+ local s = tostring(val)
+ if type(val) == 'string' then
+ s = '"'..s..'"'
+ end
+ return s
+end
+
+--- how our list should be rendered as a string. Uses join().
+-- @see List:join
+function List:__tostring()
+ return '{'..self:join(',',tostring_q)..'}'
+end
+
+--[[
+-- NOTE: this works, but is unreliable. If you leave the loop before finishing,
+-- then the iterator is not reset.
+--- can iterate over a list directly.
+-- @usage for v in ls do print(v) end
+function List:__call()
+ if not self.key then self.key = 1 end
+ local value = self[self.key]
+ self.key = self.key + 1
+ if not value then self.key = nil end
+ return value
+end
+--]]
+
+--[[
+function List.__call(t,v,i)
+ i = (i or 0) + 1
+ v = t[i]
+ if v then return i, v end
+end
+--]]
+
+local MethodIter = {}
+
+function MethodIter:__index (name)
+ return function(mm,...)
+ return self.list:foreachm(name,...)
+ end
+end
+
+--- call the function for each element of the list.
+-- @param fun a function or callable object
+-- @param ... optional values to pass to function
+function List:foreach (fun,...)
+ if fun==nil then
+ return setmetatable({list=self},MethodIter)
+ end
+ fun = function_arg(1,fun)
+ for i = 1,#self do
+ fun(self[i],...)
+ end
+end
+
+function List:foreachm (name,...)
+ for i = 1,#self do
+ local obj = self[i]
+ local f = assert(obj[name],"method not found on object")
+ f(obj,...)
+ end
+end
+
+--- create a list of all elements which match a function.
+-- @param fun a boolean function
+-- @param arg optional argument to be passed as second argument of the predicate
+-- @return a new filtered list.
+function List:filter (fun,arg)
+ return makelist(filter(self,fun,arg),self)
+end
+
+--- split a string using a delimiter.
+-- @param s the string
+-- @param delim the delimiter (default spaces)
+-- @return a List of strings
+-- @see pl.utils.split
+function List.split (s,delim)
+ assert_arg(1,s,'string')
+ return makelist(split(s,delim))
+end
+
+local MethodMapper = {}
+
+function MethodMapper:__index (name)
+ return function(mm,...)
+ return self.list:mapm(name,...)
+ end
+end
+
+--- apply a function to all elements.
+-- Any extra arguments will be passed to the function; if the function
+-- is `nil` then `map` returns a mapper object that maps over a method
+-- of the items
+-- @param fun a function of at least one argument
+-- @param ... arbitrary extra arguments.
+-- @return a new list: {f(x) for x in self}
+-- @usage List{'one','two'}:map(string.upper) == {'ONE','TWO'}
+-- @usage List{'one','two'}:map():sub(1,2) == {'on','tw'}
+-- @see pl.tablex.imap
+function List:map (fun,...)
+ if fun==nil then
+ return setmetatable({list=self},MethodMapper)
+ end
+ return makelist(imap(fun,self,...),self)
+end
+
+--- apply a function to all elements, in-place.
+-- Any extra arguments are passed to the function.
+-- @param fun A function that takes at least one argument
+-- @param ... arbitrary extra arguments.
+function List:transform (fun,...)
+ transform(fun,self,...)
+end
+
+--- apply a function to elements of two lists.
+-- Any extra arguments will be passed to the function
+-- @param fun a function of at least two arguments
+-- @param ls another list
+-- @param ... arbitrary extra arguments.
+-- @return a new list: {f(x,y) for x in self, for x in arg1}
+-- @see pl.tablex.imap2
+function List:map2 (fun,ls,...)
+ return makelist(imap2(fun,self,ls,...),self)
+end
+
+--- apply a named method to all elements.
+-- Any extra arguments will be passed to the method.
+-- @param name name of method
+-- @param ... extra arguments
+-- @return a new list of the results
+-- @see pl.seq.mapmethod
+function List:mapm (name,...)
+ local res = {}
+ local t = self
+ for i = 1,#t do
+ local val = t[i]
+ local fn = val[name]
+ if not fn then error(type(val).." does not have method "..name,2) end
+ res[i] = fn(val,...)
+ end
+ return makelist(res,self)
+end
+
+--- 'reduce' a list using a binary function.
+-- @param fun a function of two arguments
+-- @return result of the function
+-- @see pl.tablex.reduce
+function List:reduce (fun)
+ return reduce(fun,self)
+end
+
+--- partition a list using a classifier function.
+-- The function may return nil, but this will be converted to the string key '<nil>'.
+-- @param fun a function of at least one argument
+-- @param ... will also be passed to the function
+-- @return a table where the keys are the returned values, and the values are Lists
+-- of values where the function returned that key. It is given the type of Multimap.
+-- @see pl.MultiMap
+function List:partition (fun,...)
+ fun = function_arg(1,fun)
+ local res = {}
+ for i = 1,#self do
+ local val = self[i]
+ local klass = fun(val,...)
+ if klass == nil then klass = '<nil>' end
+ if not res[klass] then res[klass] = List() end
+ res[klass]:append(val)
+ end
+ return setmetatable(res,Multimap)
+end
+
+--- return an iterator over all values.
+function List:iter ()
+ return iter(self)
+end
+
+--- Create an iterator over a seqence.
+-- This captures the Python concept of 'sequence'.
+-- For tables, iterates over all values with integer indices.
+-- @param seq a sequence; a string (over characters), a table, a file object (over lines) or an iterator function
+-- @usage for x in iterate {1,10,22,55} do io.write(x,',') end ==> 1,10,22,55
+-- @usage for ch in iterate 'help' do do io.write(ch,' ') end ==> h e l p
+function List.iterate(seq)
+ if type(seq) == 'string' then
+ local idx = 0
+ local n = #seq
+ local sub = string.sub
+ return function ()
+ idx = idx + 1
+ if idx > n then return nil
+ else
+ return sub(seq,idx,idx)
+ end
+ end
+ elseif type(seq) == 'table' then
+ local idx = 0
+ local n = #seq
+ return function()
+ idx = idx + 1
+ if idx > n then return nil
+ else
+ return seq[idx]
+ end
+ end
+ elseif type(seq) == 'function' then
+ return seq
+ elseif type(seq) == 'userdata' and io.type(seq) == 'file' then
+ return seq:lines()
+ end
+end
+iter = List.iterate
+
+return List
+
--- /dev/null
+--- A Map class.
+--
+-- > Map = require 'pl.Map'
+-- > m = Map{one=1,two=2}
+-- > m:update {three=3,four=4,two=20}
+-- > = m == M{one=1,two=20,three=3,four=4}
+-- true
+--
+-- Dependencies: `pl.utils`, `pl.class`, `pl.tablex`, `pl.pretty`
+-- @module pl.Map
+
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+local stdmt = utils.stdmt
+local is_callable = utils.is_callable
+local tmakeset,deepcompare,merge,keys,difference,tupdate = tablex.makeset,tablex.deepcompare,tablex.merge,tablex.keys,tablex.difference,tablex.update
+
+local pretty_write = require 'pl.pretty' . write
+local Map = stdmt.Map
+local Set = stdmt.Set
+local List = stdmt.List
+
+local class = require 'pl.class'
+
+-- the Map class ---------------------
+class(nil,nil,Map)
+
+local function makemap (m)
+ return setmetatable(m,Map)
+end
+
+function Map:_init (t)
+ local mt = getmetatable(t)
+ if mt == Set or mt == Map then
+ self:update(t)
+ else
+ return t -- otherwise assumed to be a map-like table
+ end
+end
+
+
+local function makelist(t)
+ return setmetatable(t,List)
+end
+
+--- list of keys.
+Map.keys = tablex.keys
+
+--- list of values.
+Map.values = tablex.values
+
+--- return an iterator over all key-value pairs.
+function Map:iter ()
+ return pairs(self)
+end
+
+--- return a List of all key-value pairs, sorted by the keys.
+function Map:items()
+ local ls = makelist(tablex.pairmap (function (k,v) return makelist {k,v} end, self))
+ ls:sort(function(t1,t2) return t1[1] < t2[1] end)
+ return ls
+end
+
+-- Will return the existing value, or if it doesn't exist it will set
+-- a default value and return it.
+function Map:setdefault(key, defaultval)
+ return self[key] or self:set(key,defaultval) or defaultval
+end
+
+--- size of map.
+-- note: this is a relatively expensive operation!
+-- @class function
+-- @name Map:len
+Map.len = tablex.size
+
+--- put a value into the map.
+-- @param key the key
+-- @param val the value
+function Map:set (key,val)
+ self[key] = val
+end
+
+--- get a value from the map.
+-- @param key the key
+-- @return the value, or nil if not found.
+function Map:get (key)
+ return rawget(self,key)
+end
+
+local index_by = tablex.index_by
+
+--- get a list of values indexed by a list of keys.
+-- @param keys a list-like table of keys
+-- @return a new list
+function Map:getvalues (keys)
+ return makelist(index_by(self,keys))
+end
+
+--- update the map using key/value pairs from another table.
+-- @param table
+-- @function Map:update
+Map.update = tablex.update
+
+function Map:__eq (m)
+ -- note we explicitly ask deepcompare _not_ to use __eq!
+ return deepcompare(self,m,true)
+end
+
+function Map:__tostring ()
+ return pretty_write(self,'')
+end
+
+return Map
--- /dev/null
+--- MultiMap, a Map which has multiple values per key.
+--
+-- Dependencies: `pl.utils`, `pl.class`, `pl.tablex`, `pl.List`
+-- @module pl.MultiMap
+
+local classes = require 'pl.class'
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+local List = require 'pl.List'
+
+local index_by,tsort,concat = tablex.index_by,table.sort,table.concat
+local append,extend,slice = List.append,List.extend,List.slice
+local append = table.insert
+local is_type = utils.is_type
+
+local class = require 'pl.class'
+local Map = require 'pl.Map'
+
+-- MultiMap is a standard MT
+local MultiMap = utils.stdmt.MultiMap
+
+class(Map,nil,MultiMap)
+MultiMap._name = 'MultiMap'
+
+function MultiMap:_init (t)
+ if not t then return end
+ self:update(t)
+end
+
+--- update a MultiMap using a table.
+-- @param t either a Multimap or a map-like table.
+-- @return the map
+function MultiMap:update (t)
+ utils.assert_arg(1,t,'table')
+ if Map:class_of(t) then
+ for k,v in pairs(t) do
+ self[k] = List()
+ self[k]:append(v)
+ end
+ else
+ for k,v in pairs(t) do
+ self[k] = List(v)
+ end
+ end
+end
+
+--- add a new value to a key. Setting a nil value removes the key.
+-- @param key the key
+-- @param val the value
+-- @return the map
+function MultiMap:set (key,val)
+ if val == nil then
+ self[key] = nil
+ else
+ if not self[key] then
+ self[key] = List()
+ end
+ self[key]:append(val)
+ end
+end
+
+return MultiMap
--- /dev/null
+--- OrderedMap, a map which preserves ordering.
+--
+-- Derived from `pl.Map`.
+--
+-- Dependencies: `pl.utils`, `pl.tablex`, `pl.List`
+-- @module pl.OrderedMap
+
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+local List = require 'pl.List'
+local index_by,tsort,concat = tablex.index_by,table.sort,table.concat
+
+local class = require 'pl.class'
+local Map = require 'pl.Map'
+
+local OrderedMap = class(Map)
+OrderedMap._name = 'OrderedMap'
+
+--- construct an OrderedMap.
+-- Will throw an error if the argument is bad.
+-- @param t optional initialization table, same as for @{OrderedMap:update}
+function OrderedMap:_init (t)
+ self._keys = List()
+ if t then
+ local map,err = self:update(t)
+ if not map then error(err,2) end
+ end
+end
+
+local assert_arg,raise = utils.assert_arg,utils.raise
+
+--- update an OrderedMap using a table. <br>
+-- If the table is itself an OrderedMap, then its entries will be appended. <br>
+-- if it s a table of the form <code>{{key1=val1},{key2=val2},...}</code> these will be appended. <br>
+-- Otherwise, it is assumed to be a map-like table, and order of extra entries is arbitrary.
+-- @param t a table.
+-- @return the map, or nil in case of error
+-- @return the error message
+function OrderedMap:update (t)
+ assert_arg(1,t,'table')
+ if OrderedMap:class_of(t) then
+ for k,v in t:iter() do
+ self:set(k,v)
+ end
+ elseif #t > 0 then -- an array must contain {key=val} tables
+ if type(t[1]) == 'table' then
+ for _,pair in ipairs(t) do
+ local key,value = next(pair)
+ if not key then return raise 'empty pair initialization table' end
+ self:set(key,value)
+ end
+ else
+ return raise 'cannot use an array to initialize an OrderedMap'
+ end
+ else
+ for k,v in pairs(t) do
+ self:set(k,v)
+ end
+ end
+ return self
+end
+
+--- set the key's value. This key will be appended at the end of the map. <br>
+-- If the value is nil, then the key is removed.
+-- @param key the key
+-- @param val the value
+-- @return the map
+function OrderedMap:set (key,val)
+ if not self[key] and val ~= nil then -- ensure that keys are unique
+ self._keys:append(key)
+ elseif val == nil then -- removing a key-value pair
+ self._keys:remove_value(key)
+ end
+ self[key] = val
+ return self
+end
+
+--- insert a key/value pair before a given position.
+-- Note: if the map already contains the key, then this effectively
+-- moves the item to the new position by first removing at the old position.
+-- Has no effect if the key does not exist and val is nil
+-- @param pos a position starting at 1
+-- @param key the key
+-- @param val the value; if nil use the old value
+function OrderedMap:insert (pos,key,val)
+ local oldval = self[key]
+ val = val or oldval
+ if oldval then
+ self._keys:remove_value(key)
+ end
+ if val then
+ self._keys:insert(pos,key)
+ self[key] = val
+ end
+ return self
+end
+
+--- return the keys in order.
+-- (Not a copy!)
+-- @return List
+function OrderedMap:keys ()
+ return self._keys
+end
+
+--- return the values in order.
+-- this is relatively expensive.
+-- @return List
+function OrderedMap:values ()
+ return List(index_by(self,self._keys))
+end
+
+--- sort the keys.
+-- @param cmp a comparison function as for @{table.sort}
+-- @return the map
+function OrderedMap:sort (cmp)
+ tsort(self._keys,cmp)
+ return self
+end
+
+--- iterate over key-value pairs in order.
+function OrderedMap:iter ()
+ local i = 0
+ local keys = self._keys
+ local n,idx = #keys
+ return function()
+ i = i + 1
+ if i > #keys then return nil end
+ idx = keys[i]
+ return idx,self[idx]
+ end
+end
+
+function OrderedMap:__tostring ()
+ local res = {}
+ for i,v in ipairs(self._keys) do
+ local val = self[v]
+ local vs = tostring(val)
+ if type(val) ~= 'number' then
+ vs = '"'..vs..'"'
+ end
+ res[i] = tostring(v)..'='..vs
+ end
+ return '{'..concat(res,',')..'}'
+end
+
+return OrderedMap
+
+
+
--- /dev/null
+--- A Set class.
+--
+-- > Set = require 'pl.Set'
+-- > = Set{'one','two'} == Set{'two','one'}
+-- true
+-- > fruit = Set{'apple','banana','orange'}
+-- > = fruit['banana']
+-- true
+-- > = fruit['hazelnut']
+-- nil
+-- > colours = Set{'red','orange','green','blue'}
+-- > = fruit,colours
+-- [apple,orange,banana] [blue,green,orange,red]
+-- > = fruit+colours
+-- [blue,green,apple,red,orange,banana]
+-- > = fruit*colours
+-- [orange]
+--
+-- Depdencies: `pl.utils`, `pl.tablex`, `pl.class`
+-- @module pl.Set
+
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+local stdmt = utils.stdmt
+local tmakeset,deepcompare,merge,keys,difference,tupdate = tablex.makeset,tablex.deepcompare,tablex.merge,tablex.keys,tablex.difference,tablex.update
+local Map = require 'pl.Map'
+local Set = stdmt.Set
+local List = stdmt.List
+local class = require 'pl.class'
+
+-- the Set class --------------------
+class(Map,nil,Set)
+
+-- note: Set has _no_ methods!
+Set.__index = nil
+
+local function makeset (t)
+ return setmetatable(t,Set)
+end
+
+--- create a set. <br>
+-- @param t may be a Set, Map or list-like table.
+-- @class function
+-- @name Set
+function Set:_init (t)
+ local mt = getmetatable(t)
+ if mt == Set or mt == Map then
+ for k in pairs(t) do self[k] = true end
+ else
+ for _,v in ipairs(t) do self[v] = true end
+ end
+end
+
+function Set:__tostring ()
+ return '['..Set.values(self):join ','..']'
+end
+
+--- get a list of the values in a set.
+-- @param self a Set
+-- @function Set.values
+Set.values = Map.keys
+
+--- map a function over the values of a set.
+-- @param self a Set
+-- @param fn a function
+-- @param ... extra arguments to pass to the function.
+-- @return a new set
+function Set.map (self,fn,...)
+ fn = utils.function_arg(1,fn)
+ local res = {}
+ for k in pairs(self) do
+ res[fn(k,...)] = true
+ end
+ return makeset(res)
+end
+
+--- union of two sets (also +).
+-- @param self a Set
+-- @param set another set
+-- @return a new set
+function Set.union (self,set)
+ return merge(self,set,true)
+end
+Set.__add = Set.union
+
+--- intersection of two sets (also *).
+-- @param self a Set
+-- @param set another set
+-- @return a new set
+function Set.intersection (self,set)
+ return merge(self,set,false)
+end
+Set.__mul = Set.intersection
+
+--- new set with elements in the set that are not in the other (also -).
+-- @param self a Set
+-- @param set another set
+-- @return a new set
+function Set.difference (self,set)
+ return difference(self,set,false)
+end
+Set.__sub = Set.difference
+
+-- a new set with elements in _either_ the set _or_ other but not both (also ^).
+-- @param self a Set
+-- @param set another set
+-- @return a new set
+function Set.symmetric_difference (self,set)
+ return difference(self,set,true)
+end
+Set.__pow = Set.symmetric_difference
+
+--- is the first set a subset of the second (also <)?.
+-- @param self a Set
+-- @param set another set
+-- @return true or false
+function Set.issubset (self,set)
+ for k in pairs(self) do
+ if not set[k] then return false end
+ end
+ return true
+end
+Set.__lt = Set.subset
+
+--- is the set empty?.
+-- @param self a Set
+-- @return true or false
+function Set.issempty (self)
+ return next(self) == nil
+end
+
+--- are the sets disjoint? (no elements in common).
+-- Uses naive definition, i.e. that intersection is empty
+-- @param s1 a Set
+-- @param s2 another set
+-- @return true or false
+function Set.isdisjoint (s1,s2)
+ return Set.isempty(Set.intersection(s1,s2))
+end
+
+function Set.__eq (s1,s2)
+ return Set.issubset(s1,s2) and Set.issubset(s2,s1)
+end
+
+return Set
--- /dev/null
+--- Application support functions.
+-- See @{01-introduction.md.Application_Support|the Guide}
+--
+-- Dependencies: `pl.utils`, `pl.path`, `lfs`
+-- @module pl.app
+
+local io,package,require = _G.io, _G.package, _G.require
+local utils = require 'pl.utils'
+local path = require 'pl.path'
+local lfs = require 'lfs'
+
+
+local app = {}
+
+local function check_script_name ()
+ if _G.arg == nil then error('no command line args available\nWas this run from a main script?') end
+ return _G.arg[0]
+end
+
+--- add the current script's path to the Lua module path.
+-- Applies to both the source and the binary module paths. It makes it easy for
+-- the main file of a multi-file program to access its modules in the same directory.
+-- `base` allows these modules to be put in a specified subdirectory, to allow for
+-- cleaner deployment and resolve potential conflicts between a script name and its
+-- library directory.
+-- @param base optional base directory.
+-- @return the current script's path with a trailing slash
+function app.require_here (base)
+ local p = path.dirname(check_script_name())
+ if not path.isabs(p) then
+ p = path.join(lfs.currentdir(),p)
+ end
+ if p:sub(-1,-1) ~= path.sep then
+ p = p..path.sep
+ end
+ if base then
+ p = p..base..path.sep
+ end
+ local so_ext = path.is_windows and 'dll' or 'so'
+ local lsep = package.path:find '^;' and '' or ';'
+ local csep = package.cpath:find '^;' and '' or ';'
+ package.path = ('%s?.lua;%s?%sinit.lua%s%s'):format(p,p,path.sep,lsep,package.path)
+ package.cpath = ('%s?.%s%s%s'):format(p,so_ext,csep,package.cpath)
+ return p
+end
+
+--- return a suitable path for files private to this application.
+-- These will look like '~/.SNAME/file', with '~' as with expanduser and
+-- SNAME is the name of the script without .lua extension.
+-- @param file a filename (w/out path)
+-- @return a full pathname, or nil
+-- @return 'cannot create' error
+function app.appfile (file)
+ local sname = path.basename(check_script_name())
+ local name,ext = path.splitext(sname)
+ local dir = path.join(path.expanduser('~'),'.'..name)
+ if not path.isdir(dir) then
+ local ret = lfs.mkdir(dir)
+ if not ret then return utils.raise ('cannot create '..dir) end
+ end
+ return path.join(dir,file)
+end
+
+--- return string indicating operating system.
+-- @return 'Windows','OSX' or whatever uname returns (e.g. 'Linux')
+function app.platform()
+ if path.is_windows then
+ return 'Windows'
+ else
+ local f = io.popen('uname')
+ local res = f:read()
+ if res == 'Darwin' then res = 'OSX' end
+ f:close()
+ return res
+ end
+end
+
+--- return the full command-line used to invoke this script
+-- any extra flags occupy slots, so that 'lua -lpl' gives us {[-2]='lua',[-1]='-lpl')
+-- @return command-line
+-- @return name of Lua program used
+function app.lua ()
+ local args = _G.arg or error "not in a main program"
+ local imin = 0
+ for i in pairs(args) do
+ if i < imin then imin = i end
+ end
+ local cmd, append = {}, table.insert
+ for i = imin,-1 do
+ local a = args[i]
+ if a:match '%s' then
+ a = '"'..a..'"'
+ end
+ append(cmd,a)
+ end
+ return table.concat(cmd,' '),args[imin]
+end
+
+--- parse command-line arguments into flags and parameters.
+-- Understands GNU-style command-line flags; short (-f) and long (--flag).
+-- These may be given a value with either '=' or ':' (-k:2,--alpha=3.2,-n2);
+-- note that a number value can be given without a space.
+-- Multiple short args can be combined like so: (-abcd).
+-- @param args an array of strings (default is the global 'arg')
+-- @param flags_with_values any flags that take values, e.g. <code>{out=true}</code>
+-- @return a table of flags (flag=value pairs)
+-- @return an array of parameters
+-- @raise if args is nil, then the global `args` must be available!
+function app.parse_args (args,flags_with_values)
+ if not args then
+ args = _G.arg
+ if not args then error "Not in a main program: 'arg' not found" end
+ end
+ flags_with_values = flags_with_values or {}
+ local _args = {}
+ local flags = {}
+ local i = 1
+ while i <= #args do
+ local a = args[i]
+ local v = a:match('^-(.+)')
+ local is_long
+ if v then -- we have a flag
+ if v:find '^-' then
+ is_long = true
+ v = v:sub(2)
+ end
+ if flags_with_values[v] then
+ if i == #_args or args[i+1]:find '^-' then
+ return utils.raise ("no value for '"..v.."'")
+ end
+ flags[v] = args[i+1]
+ i = i + 1
+ else
+ -- a value can be indicated with = or :
+ local var,val = utils.splitv (v,'[=:]')
+ var = var or v
+ val = val or true
+ if not is_long then
+ if #var > 1 then
+ if var:find '.%d+' then -- short flag, number value
+ val = var:sub(2)
+ var = var:sub(1,1)
+ else -- multiple short flags
+ for i = 1,#var do
+ flags[var:sub(i,i)] = true
+ end
+ val = nil -- prevents use of var as a flag below
+ end
+ else -- single short flag (can have value, defaults to true)
+ val = val or true
+ end
+ end
+ if val then
+ flags[var] = val
+ end
+ end
+ else
+ _args[#_args+1] = a
+ end
+ i = i + 1
+ end
+ return flags,_args
+end
+
+return app
--- /dev/null
+--- Operations on two-dimensional arrays.
+-- See @{02-arrays.md.Operations_on_two_dimensional_tables|The Guide}
+--
+-- Dependencies: `pl.utils`, `pl.tablex`
+-- @module pl.array2d
+
+local require, type,tonumber,assert,tostring,io,ipairs,string,table =
+ _G.require, _G.type,_G.tonumber,_G.assert,_G.tostring,_G.io,_G.ipairs,_G.string,_G.table
+local setmetatable,getmetatable = setmetatable,getmetatable
+
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+
+local imap,tmap,reduce,keys,tmap2,tset,index_by = tablex.imap,tablex.map,tablex.reduce,tablex.keys,tablex.map2,tablex.set,tablex.index_by
+local remove = table.remove
+local splitv,fprintf,assert_arg = utils.splitv,utils.fprintf,utils.assert_arg
+local byte = string.byte
+local stdout = io.stdout
+
+local array2d = {}
+
+local function obj (int,out)
+ local mt = getmetatable(int)
+ if mt then
+ setmetatable(out,mt)
+ end
+ return out
+end
+
+local function makelist (res)
+ return setmetatable(res,utils.stdmt.List)
+end
+
+
+local function index (t,k)
+ return t[k]
+end
+
+--- return the row and column size.
+-- @param t a 2d array
+-- @return number of rows
+-- @return number of cols
+function array2d.size (t)
+ assert_arg(1,t,'table')
+ return #t,#t[1]
+end
+
+--- extract a column from the 2D array.
+-- @param a 2d array
+-- @param key an index or key
+-- @return 1d array
+function array2d.column (a,key)
+ assert_arg(1,a,'table')
+ return makelist(imap(index,a,key))
+end
+local column = array2d.column
+
+--- map a function over a 2D array
+-- @param f a function of at least one argument
+-- @param a 2d array
+-- @param arg an optional extra argument to be passed to the function.
+-- @return 2d array
+function array2d.map (f,a,arg)
+ assert_arg(1,a,'table')
+ f = utils.function_arg(1,f)
+ return obj(a,imap(function(row) return imap(f,row,arg) end, a))
+end
+
+--- reduce the rows using a function.
+-- @param f a binary function
+-- @param a 2d array
+-- @return 1d array
+-- @see pl.tablex.reduce
+function array2d.reduce_rows (f,a)
+ assert_arg(1,a,'table')
+ return tmap(function(row) return reduce(f,row) end, a)
+end
+
+--- reduce the columns using a function.
+-- @param f a binary function
+-- @param a 2d array
+-- @return 1d array
+-- @see pl.tablex.reduce
+function array2d.reduce_cols (f,a)
+ assert_arg(1,a,'table')
+ return tmap(function(c) return reduce(f,column(a,c)) end, keys(a[1]))
+end
+
+--- reduce a 2D array into a scalar, using two operations.
+-- @param opc operation to reduce the final result
+-- @param opr operation to reduce the rows
+-- @param a 2D array
+function array2d.reduce2 (opc,opr,a)
+ assert_arg(3,a,'table')
+ local tmp = array2d.reduce_rows(opr,a)
+ return reduce(opc,tmp)
+end
+
+local function dimension (t)
+ return type(t[1])=='table' and 2 or 1
+end
+
+--- map a function over two arrays.
+-- They can be both or either 2D arrays
+-- @param f function of at least two arguments
+-- @param ad order of first array
+-- @param bd order of second array
+-- @param a 1d or 2d array
+-- @param b 1d or 2d array
+-- @param arg optional extra argument to pass to function
+-- @return 2D array, unless both arrays are 1D
+function array2d.map2 (f,ad,bd,a,b,arg)
+ assert_arg(1,a,'table')
+ assert_arg(2,b,'table')
+ f = utils.function_arg(1,f)
+ --local ad,bd = dimension(a),dimension(b)
+ if ad == 1 and bd == 2 then
+ return imap(function(row)
+ return tmap2(f,a,row,arg)
+ end, b)
+ elseif ad == 2 and bd == 1 then
+ return imap(function(row)
+ return tmap2(f,row,b,arg)
+ end, a)
+ elseif ad == 1 and bd == 1 then
+ return tmap2(f,a,b)
+ elseif ad == 2 and bd == 2 then
+ return tmap2(function(rowa,rowb)
+ return tmap2(f,rowa,rowb,arg)
+ end, a,b)
+ end
+end
+
+--- cartesian product of two 1d arrays.
+-- @param f a function of 2 arguments
+-- @param t1 a 1d table
+-- @param t2 a 1d table
+-- @return 2d table
+-- @usage product('..',{1,2},{'a','b'}) == {{'1a','2a'},{'1b','2b'}}
+function array2d.product (f,t1,t2)
+ f = utils.function_arg(1,f)
+ assert_arg(2,t1,'table')
+ assert_arg(3,t2,'table')
+ local res, map = {}, tablex.map
+ for i,v in ipairs(t2) do
+ res[i] = map(f,t1,v)
+ end
+ return res
+end
+
+--- flatten a 2D array.
+-- (this goes over columns first.)
+-- @param t 2d table
+-- @return a 1d table
+-- @usage flatten {{1,2},{3,4},{5,6}} == {1,2,3,4,5,6}
+function array2d.flatten (t)
+ local res = {}
+ local k = 1
+ for _,a in ipairs(t) do -- for all rows
+ for i = 1,#a do
+ res[k] = a[i]
+ k = k + 1
+ end
+ end
+ return makelist(res)
+end
+
+--- reshape a 2D array.
+-- @param t 2d array
+-- @param nrows new number of rows
+-- @param co column-order (Fortran-style) (default false)
+-- @return a new 2d array
+function array2d.reshape (t,nrows,co)
+ local nr,nc = array2d.size(t)
+ local ncols = nr*nc / nrows
+ local res = {}
+ local ir,ic = 1,1
+ for i = 1,nrows do
+ local row = {}
+ for j = 1,ncols do
+ row[j] = t[ir][ic]
+ if not co then
+ ic = ic + 1
+ if ic > nc then
+ ir = ir + 1
+ ic = 1
+ end
+ else
+ ir = ir + 1
+ if ir > nr then
+ ic = ic + 1
+ ir = 1
+ end
+ end
+ end
+ res[i] = row
+ end
+ return obj(t,res)
+end
+
+--- swap two rows of an array.
+-- @param t a 2d array
+-- @param i1 a row index
+-- @param i2 a row index
+function array2d.swap_rows (t,i1,i2)
+ assert_arg(1,t,'table')
+ t[i1],t[i2] = t[i2],t[i1]
+end
+
+--- swap two columns of an array.
+-- @param t a 2d array
+-- @param j1 a column index
+-- @param j2 a column index
+function array2d.swap_cols (t,j1,j2)
+ assert_arg(1,t,'table')
+ for i = 1,#t do
+ local row = t[i]
+ row[j1],row[j2] = row[j2],row[j1]
+ end
+end
+
+--- extract the specified rows.
+-- @param t 2d array
+-- @param ridx a table of row indices
+function array2d.extract_rows (t,ridx)
+ return obj(t,index_by(t,ridx))
+end
+
+--- extract the specified columns.
+-- @param t 2d array
+-- @param cidx a table of column indices
+function array2d.extract_cols (t,cidx)
+ assert_arg(1,t,'table')
+ local res = {}
+ for i = 1,#t do
+ res[i] = index_by(t[i],cidx)
+ end
+ return obj(t,res)
+end
+
+--- remove a row from an array.
+-- @class function
+-- @name array2d.remove_row
+-- @param t a 2d array
+-- @param i a row index
+array2d.remove_row = remove
+
+--- remove a column from an array.
+-- @param t a 2d array
+-- @param j a column index
+function array2d.remove_col (t,j)
+ assert_arg(1,t,'table')
+ for i = 1,#t do
+ remove(t[i],j)
+ end
+end
+
+local Ai = byte 'A'
+
+local function _parse (s)
+ local c,r
+ if s:sub(1,1) == 'R' then
+ r,c = s:match 'R(%d+)C(%d+)'
+ r,c = tonumber(r),tonumber(c)
+ else
+ c,r = s:match '(.)(.)'
+ c = byte(c) - byte 'A' + 1
+ r = tonumber(r)
+ end
+ assert(c ~= nil and r ~= nil,'bad cell specifier: '..s)
+ return r,c
+end
+
+--- parse a spreadsheet range.
+-- The range can be specified either as 'A1:B2' or 'R1C1:R2C2';
+-- a special case is a single element (e.g 'A1' or 'R1C1')
+-- @param s a range.
+-- @return start col
+-- @return start row
+-- @return end col
+-- @return end row
+function array2d.parse_range (s)
+ if s:find ':' then
+ local start,finish = splitv(s,':')
+ local i1,j1 = _parse(start)
+ local i2,j2 = _parse(finish)
+ return i1,j1,i2,j2
+ else -- single value
+ local i,j = _parse(s)
+ return i,j
+ end
+end
+
+--- get a slice of a 2D array using spreadsheet range notation. @see parse_range
+-- @param t a 2D array
+-- @param rstr range expression
+-- @return a slice
+-- @see array2d.parse_range
+-- @see array2d.slice
+function array2d.range (t,rstr)
+ assert_arg(1,t,'table')
+ local i1,j1,i2,j2 = array2d.parse_range(rstr)
+ if i2 then
+ return array2d.slice(t,i1,j1,i2,j2)
+ else -- single value
+ return t[i1][j1]
+ end
+end
+
+local function default_range (t,i1,j1,i2,j2)
+ local nr, nc = array2d.size(t)
+ i1,j1 = i1 or 1, j1 or 1
+ i2,j2 = i2 or nr, j2 or nc
+ if i2 < 0 then i2 = nr + i2 + 1 end
+ if j2 < 0 then j2 = nc + j2 + 1 end
+ return i1,j1,i2,j2
+end
+
+--- get a slice of a 2D array. Note that if the specified range has
+-- a 1D result, the rank of the result will be 1.
+-- @param t a 2D array
+-- @param i1 start row (default 1)
+-- @param j1 start col (default 1)
+-- @param i2 end row (default N)
+-- @param j2 end col (default M)
+-- @return an array, 2D in general but 1D in special cases.
+function array2d.slice (t,i1,j1,i2,j2)
+ assert_arg(1,t,'table')
+ i1,j1,i2,j2 = default_range(t,i1,j1,i2,j2)
+ local res = {}
+ for i = i1,i2 do
+ local val
+ local row = t[i]
+ if j1 == j2 then
+ val = row[j1]
+ else
+ val = {}
+ for j = j1,j2 do
+ val[#val+1] = row[j]
+ end
+ end
+ res[#res+1] = val
+ end
+ if i1 == i2 then res = res[1] end
+ return obj(t,res)
+end
+
+--- set a specified range of an array to a value.
+-- @param t a 2D array
+-- @param value the value (may be a function)
+-- @param i1 start row (default 1)
+-- @param j1 start col (default 1)
+-- @param i2 end row (default N)
+-- @param j2 end col (default M)
+-- @see tablex.set
+function array2d.set (t,value,i1,j1,i2,j2)
+ i1,j1,i2,j2 = default_range(t,i1,j1,i2,j2)
+ for i = i1,i2 do
+ tset(t[i],value,j1,j2)
+ end
+end
+
+--- write a 2D array to a file.
+-- @param t a 2D array
+-- @param f a file object (default stdout)
+-- @param fmt a format string (default is just to use tostring)
+-- @param i1 start row (default 1)
+-- @param j1 start col (default 1)
+-- @param i2 end row (default N)
+-- @param j2 end col (default M)
+function array2d.write (t,f,fmt,i1,j1,i2,j2)
+ assert_arg(1,t,'table')
+ f = f or stdout
+ local rowop
+ if fmt then
+ rowop = function(row,j) fprintf(f,fmt,row[j]) end
+ else
+ rowop = function(row,j) f:write(tostring(row[j]),' ') end
+ end
+ local function newline()
+ f:write '\n'
+ end
+ array2d.forall(t,rowop,newline,i1,j1,i2,j2)
+end
+
+--- perform an operation for all values in a 2D array.
+-- @param t 2D array
+-- @param row_op function to call on each value
+-- @param end_row_op function to call at end of each row
+-- @param i1 start row (default 1)
+-- @param j1 start col (default 1)
+-- @param i2 end row (default N)
+-- @param j2 end col (default M)
+function array2d.forall (t,row_op,end_row_op,i1,j1,i2,j2)
+ assert_arg(1,t,'table')
+ i1,j1,i2,j2 = default_range(t,i1,j1,i2,j2)
+ for i = i1,i2 do
+ local row = t[i]
+ for j = j1,j2 do
+ row_op(row,j)
+ end
+ if end_row_op then end_row_op(i) end
+ end
+end
+
+local min, max = math.min, math.max
+
+---- move a block from the destination to the source.
+-- @param dest a 2D array
+-- @param di start row in dest
+-- @param dj start col in dest
+-- @param src a 2D array
+-- @param i1 start row (default 1)
+-- @param j1 start col (default 1)
+-- @param i2 end row (default N)
+-- @param j2 end col (default M)
+function array2d.move (dest,di,dj,src,i1,j1,i2,j2)
+ assert_arg(1,dest,'table')
+ assert_arg(4,src,'table')
+ i1,j1,i2,j2 = default_range(src,i1,j1,i2,j2)
+ local nr,nc = array2d.size(dest)
+ i2, j2 = min(nr,i2), min(nc,j2)
+ --i1, j1 = max(1,i1), max(1,j1)
+ dj = dj - 1
+ for i = i1,i2 do
+ local drow, srow = dest[i+di-1], src[i]
+ for j = j1,j2 do
+ drow[j+dj] = srow[j]
+ end
+ end
+end
+
+--- iterate over all elements in a 2D array, with optional indices.
+-- @param a 2D array
+-- @param indices with indices (default false)
+-- @param i1 start row (default 1)
+-- @param j1 start col (default 1)
+-- @param i2 end row (default N)
+-- @param j2 end col (default M)
+-- @return either value or i,j,value depending on indices
+function array2d.iter (a,indices,i1,j1,i2,j2)
+ assert_arg(1,a,'table')
+ local norowset = not (i2 and j2)
+ i1,j1,i2,j2 = default_range(a,i1,j1,i2,j2)
+ local n,i,j = i2-i1+1,i1-1,j1-1
+ local row,nr = nil,0
+ local onr = j2 - j1 + 1
+ return function()
+ j = j + 1
+ if j > nr then
+ j = j1
+ i = i + 1
+ if i > i2 then return nil end
+ row = a[i]
+ nr = norowset and #row or onr
+ end
+ if indices then
+ return i,j,row[j]
+ else
+ return row[j]
+ end
+ end
+end
+
+--- iterate over all columns.
+-- @param a a 2D array
+-- @return each column in turn
+function array2d.columns (a)
+ assert_arg(1,a,'table')
+ local n = a[1][1]
+ local i = 0
+ return function()
+ i = i + 1
+ if i > n then return nil end
+ return column(a,i)
+ end
+end
+
+--- new array of specified dimensions
+-- @param rows number of rows
+-- @param cols number of cols
+-- @param val initial value; if it's a function then use `val(i,j)`
+-- @return new 2d array
+function array2d.new(rows,cols,val)
+ local res = {}
+ local fun = utils.is_callable(val)
+ for i = 1,rows do
+ local row = {}
+ if fun then
+ for j = 1,cols do row[j] = val(i,j) end
+ else
+ for j = 1,cols do row[j] = val end
+ end
+ res[i] = row
+ end
+ return res
+end
+
+return array2d
+
+
--- /dev/null
+--- Provides a reuseable and convenient framework for creating classes in Lua.
+-- Two possible notations:
+--
+-- B = class(A)
+-- class.B(A)
+--
+-- The latter form creates a named class.
+--
+-- See the Guide for further @{01-introduction.md.Simplifying_Object_Oriented_Programming_in_Lua|discussion}
+-- @module pl.class
+
+local error, getmetatable, io, pairs, rawget, rawset, setmetatable, tostring, type =
+ _G.error, _G.getmetatable, _G.io, _G.pairs, _G.rawget, _G.rawset, _G.setmetatable, _G.tostring, _G.type
+-- this trickery is necessary to prevent the inheritance of 'super' and
+-- the resulting recursive call problems.
+local function call_ctor (c,obj,...)
+ -- nice alias for the base class ctor
+ local base = rawget(c,'_base')
+ if base then
+ local parent_ctor = rawget(base,'_init')
+ if parent_ctor then
+ obj.super = function(obj,...)
+ call_ctor(base,obj,...)
+ end
+ end
+ end
+ local res = c._init(obj,...)
+ obj.super = nil
+ return res
+end
+
+local function is_a(self,klass)
+ local m = getmetatable(self)
+ if not m then return false end --*can't be an object!
+ while m do
+ if m == klass then return true end
+ m = rawget(m,'_base')
+ end
+ return false
+end
+
+local function class_of(klass,obj)
+ if type(klass) ~= 'table' or not rawget(klass,'is_a') then return false end
+ return klass.is_a(obj,klass)
+end
+
+local function _class_tostring (obj)
+ local mt = obj._class
+ local name = rawget(mt,'_name')
+ setmetatable(obj,nil)
+ local str = tostring(obj)
+ setmetatable(obj,mt)
+ if name then str = name ..str:gsub('table','') end
+ return str
+end
+
+local function tupdate(td,ts)
+ for k,v in pairs(ts) do
+ td[k] = v
+ end
+end
+
+local function _class(base,c_arg,c)
+ c = c or {} -- a new class instance, which is the metatable for all objects of this type
+ -- the class will be the metatable for all its objects,
+ -- and they will look up their methods in it.
+ local mt = {} -- a metatable for the class instance
+
+ if type(base) == 'table' then
+ -- our new class is a shallow copy of the base class!
+ tupdate(c,base)
+ c._base = base
+ -- inherit the 'not found' handler, if present
+ if rawget(c,'_handler') then mt.__index = c._handler end
+ elseif base ~= nil then
+ error("must derive from a table type",3)
+ end
+
+ c.__index = c
+ setmetatable(c,mt)
+ c._init = nil
+
+ if base and rawget(base,'_class_init') then
+ base._class_init(c,c_arg)
+ end
+
+ -- expose a ctor which can be called by <classname>(<args>)
+ mt.__call = function(class_tbl,...)
+ local obj = {}
+ setmetatable(obj,c)
+
+ if rawget(c,'_init') then -- explicit constructor
+ local res = call_ctor(c,obj,...)
+ if res then -- _if_ a ctor returns a value, it becomes the object...
+ obj = res
+ setmetatable(obj,c)
+ end
+ elseif base and rawget(base,'_init') then -- default constructor
+ -- make sure that any stuff from the base class is initialized!
+ call_ctor(base,obj,...)
+ end
+
+ if base and rawget(base,'_post_init') then
+ base._post_init(obj)
+ end
+
+ if not rawget(c,'__tostring') then
+ c.__tostring = _class_tostring
+ end
+ return obj
+ end
+ -- Call Class.catch to set a handler for methods/properties not found in the class!
+ c.catch = function(handler)
+ c._handler = handler
+ mt.__index = handler
+ end
+ c.is_a = is_a
+ c.class_of = class_of
+ c._class = c
+
+ return c
+end
+
+--- create a new class, derived from a given base class.
+-- Supporting two class creation syntaxes:
+-- either `Name = class(base)` or `class.Name(base)`
+-- @function class
+-- @param base optional base class
+-- @param c_arg optional parameter to class ctor
+-- @param c optional table to be used as class
+local class
+class = setmetatable({},{
+ __call = function(fun,...)
+ return _class(...)
+ end,
+ __index = function(tbl,key)
+ if key == 'class' then
+ io.stderr:write('require("pl.class").class is deprecated. Use require("pl.class")\n')
+ return class
+ end
+ local env = _G
+ return function(...)
+ local c = _class(...)
+ c._name = key
+ rawset(env,key,c)
+ return c
+ end
+ end
+})
+
+class.properties = class()
+
+function class.properties._class_init(klass)
+ klass.__index = function(t,key)
+ -- normal class lookup!
+ local v = klass[key]
+ if v then return v end
+ -- is it a getter?
+ v = rawget(klass,'get_'..key)
+ if v then
+ return v(t)
+ end
+ -- is it a field?
+ return rawget(t,'_'..key)
+ end
+ klass.__newindex = function (t,key,value)
+ -- if there's a setter, use that, otherwise directly set table
+ local p = 'set_'..key
+ local setter = klass[p]
+ if setter then
+ setter(t,value)
+ else
+ rawset(t,key,value)
+ end
+ end
+end
+
+
+return class
+
--- /dev/null
+--- List comprehensions implemented in Lua.
+--
+-- See the [wiki page](http://lua-users.org/wiki/ListComprehensions)
+--
+-- local C= require 'pl.comprehension' . new()
+--
+-- C ('x for x=1,10') ()
+-- ==> {1,2,3,4,5,6,7,8,9,10}
+-- C 'x^2 for x=1,4' ()
+-- ==> {1,4,9,16}
+-- C '{x,x^2} for x=1,4' ()
+-- ==> {{1,1},{2,4},{3,9},{4,16}}
+-- C '2*x for x' {1,2,3}
+-- ==> {2,4,6}
+-- dbl = C '2*x for x'
+-- dbl {10,20,30}
+-- ==> {20,40,60}
+-- C 'x for x if x % 2 == 0' {1,2,3,4,5}
+-- ==> {2,4}
+-- C '{x,y} for x = 1,2 for y = 1,2' ()
+-- ==> {{1,1},{1,2},{2,1},{2,2}}
+-- C '{x,y} for x for y' ({1,2},{10,20})
+-- ==> {{1,10},{1,20},{2,10},{2,20}}
+-- assert(C 'sum(x^2 for x)' {2,3,4} == 2^2+3^2+4^2)
+--
+-- (c) 2008 David Manura. Licensed under the same terms as Lua (MIT license).
+--
+-- Dependencies: `pl.utils`, `pl.luabalanced`
+--
+-- See @{07-functional.md.List_Comprehensions|the Guide}
+-- @module pl.comprehension
+
+local utils = require 'pl.utils'
+
+local status,lb = pcall(require, "pl.luabalanced")
+if not status then
+ lb = require 'luabalanced'
+end
+
+local math_max = math.max
+local table_concat = table.concat
+
+-- fold operations
+-- http://en.wikipedia.org/wiki/Fold_(higher-order_function)
+local ops = {
+ list = {init=' {} ', accum=' __result[#__result+1] = (%s) '},
+ table = {init=' {} ', accum=' local __k, __v = %s __result[__k] = __v '},
+ sum = {init=' 0 ', accum=' __result = __result + (%s) '},
+ min = {init=' nil ', accum=' local __tmp = %s ' ..
+ ' if __result then if __tmp < __result then ' ..
+ '__result = __tmp end else __result = __tmp end '},
+ max = {init=' nil ', accum=' local __tmp = %s ' ..
+ ' if __result then if __tmp > __result then ' ..
+ '__result = __tmp end else __result = __tmp end '},
+}
+
+
+-- Parses comprehension string expr.
+-- Returns output expression list <out> string, array of for types
+-- ('=', 'in' or nil) <fortypes>, array of input variable name
+-- strings <invarlists>, array of input variable value strings
+-- <invallists>, array of predicate expression strings <preds>,
+-- operation name string <opname>, and number of placeholder
+-- parameters <max_param>.
+--
+-- The is equivalent to the mathematical set-builder notation:
+--
+-- <opname> { <out> | <invarlist> in <invallist> , <preds> }
+--
+-- @usage "x^2 for x" -- array values
+-- @usage "x^2 for x=1,10,2" -- numeric for
+-- @usage "k^v for k,v in pairs(_1)" -- iterator for
+-- @usage "(x+y)^2 for x for y if x > y" -- nested
+--
+local function parse_comprehension(expr)
+ local t = {}
+ local pos = 1
+
+ -- extract opname (if exists)
+ local opname
+ local tok, post = expr:match('^%s*([%a_][%w_]*)%s*%(()', pos)
+ local pose = #expr + 1
+ if tok then
+ local tok2, posb = lb.match_bracketed(expr, post-1)
+ assert(tok2, 'syntax error')
+ if expr:match('^%s*$', posb) then
+ opname = tok
+ pose = posb - 1
+ pos = post
+ end
+ end
+ opname = opname or "list"
+
+ -- extract out expression list
+ local out; out, pos = lb.match_explist(expr, pos)
+ assert(out, "syntax error: missing expression list")
+ out = table_concat(out, ', ')
+
+ -- extract "for" clauses
+ local fortypes = {}
+ local invarlists = {}
+ local invallists = {}
+ while 1 do
+ local post = expr:match('^%s*for%s+()', pos)
+ if not post then break end
+ pos = post
+
+ -- extract input vars
+ local iv; iv, pos = lb.match_namelist(expr, pos)
+ assert(#iv > 0, 'syntax error: zero variables')
+ for _,ident in ipairs(iv) do
+ assert(not ident:match'^__',
+ "identifier " .. ident .. " may not contain __ prefix")
+ end
+ invarlists[#invarlists+1] = iv
+
+ -- extract '=' or 'in' (optional)
+ local fortype, post = expr:match('^(=)%s*()', pos)
+ if not fortype then fortype, post = expr:match('^(in)%s+()', pos) end
+ if fortype then
+ pos = post
+ -- extract input value range
+ local il; il, pos = lb.match_explist(expr, pos)
+ assert(#il > 0, 'syntax error: zero expressions')
+ assert(fortype ~= '=' or #il == 2 or #il == 3,
+ 'syntax error: numeric for requires 2 or three expressions')
+ fortypes[#invarlists] = fortype
+ invallists[#invarlists] = il
+ else
+ fortypes[#invarlists] = false
+ invallists[#invarlists] = false
+ end
+ end
+ assert(#invarlists > 0, 'syntax error: missing "for" clause')
+
+ -- extract "if" clauses
+ local preds = {}
+ while 1 do
+ local post = expr:match('^%s*if%s+()', pos)
+ if not post then break end
+ pos = post
+ local pred; pred, pos = lb.match_expression(expr, pos)
+ assert(pred, 'syntax error: predicated expression not found')
+ preds[#preds+1] = pred
+ end
+
+ -- extract number of parameter variables (name matching "_%d+")
+ local stmp = ''; lb.gsub(expr, function(u, sin) -- strip comments/strings
+ if u == 'e' then stmp = stmp .. ' ' .. sin .. ' ' end
+ end)
+ local max_param = 0; stmp:gsub('[%a_][%w_]*', function(s)
+ local s = s:match('^_(%d+)$')
+ if s then max_param = math_max(max_param, tonumber(s)) end
+ end)
+
+ if pos ~= pose then
+ assert(false, "syntax error: unrecognized " .. expr:sub(pos))
+ end
+
+ --DEBUG:
+ --print('----\n', string.format("%q", expr), string.format("%q", out), opname)
+ --for k,v in ipairs(invarlists) do print(k,v, invallists[k]) end
+ --for k,v in ipairs(preds) do print(k,v) end
+
+ return out, fortypes, invarlists, invallists, preds, opname, max_param
+end
+
+
+-- Create Lua code string representing comprehension.
+-- Arguments are in the form returned by parse_comprehension.
+local function code_comprehension(
+ out, fortypes, invarlists, invallists, preds, opname, max_param
+)
+ local op = assert(ops[opname])
+ local code = op.accum:gsub('%%s', out)
+
+ for i=#preds,1,-1 do local pred = preds[i]
+ code = ' if ' .. pred .. ' then ' .. code .. ' end '
+ end
+ for i=#invarlists,1,-1 do
+ if not fortypes[i] then
+ local arrayname = '__in' .. i
+ local idx = '__idx' .. i
+ code =
+ ' for ' .. idx .. ' = 1, #' .. arrayname .. ' do ' ..
+ ' local ' .. invarlists[i][1] .. ' = ' .. arrayname .. '['..idx..'] ' ..
+ code .. ' end '
+ else
+ code =
+ ' for ' ..
+ table_concat(invarlists[i], ', ') ..
+ ' ' .. fortypes[i] .. ' ' ..
+ table_concat(invallists[i], ', ') ..
+ ' do ' .. code .. ' end '
+ end
+ end
+ code = ' local __result = ( ' .. op.init .. ' ) ' .. code
+ return code
+end
+
+
+-- Convert code string represented by code_comprehension
+-- into Lua function. Also must pass ninputs = #invarlists,
+-- max_param, and invallists (from parse_comprehension).
+-- Uses environment env.
+local function wrap_comprehension(code, ninputs, max_param, invallists, env)
+ assert(ninputs > 0)
+ local ts = {}
+ for i=1,max_param do
+ ts[#ts+1] = '_' .. i
+ end
+ for i=1,ninputs do
+ if not invallists[i] then
+ local name = '__in' .. i
+ ts[#ts+1] = name
+ end
+ end
+ if #ts > 0 then
+ code = ' local ' .. table_concat(ts, ', ') .. ' = ... ' .. code
+ end
+ code = code .. ' return __result '
+ --print('DEBUG:', code)
+ local f, err = utils.load(code,'tmp','t',env)
+ if not f then assert(false, err .. ' with generated code ' .. code) end
+ return f
+end
+
+
+-- Build Lua function from comprehension string.
+-- Uses environment env.
+local function build_comprehension(expr, env)
+ local out, fortypes, invarlists, invallists, preds, opname, max_param
+ = parse_comprehension(expr)
+ local code = code_comprehension(
+ out, fortypes, invarlists, invallists, preds, opname, max_param)
+ local f = wrap_comprehension(code, #invarlists, max_param, invallists, env)
+ return f
+end
+
+
+-- Creates new comprehension cache.
+-- Any list comprehension function created are set to the environment
+-- env (defaults to caller of new).
+local function new(env)
+ -- Note: using a single global comprehension cache would have had
+ -- security implications (e.g. retrieving cached functions created
+ -- in other environments).
+ -- The cache lookup function could have instead been written to retrieve
+ -- the caller's environment, lookup up the cache private to that
+ -- environment, and then looked up the function in that cache.
+ -- That would avoid the need for this <new> call to
+ -- explicitly manage caches; however, that might also have an undue
+ -- performance penalty.
+
+ if not env then
+ env = getfenv(2)
+ end
+
+ local mt = {}
+ local cache = setmetatable({}, mt)
+
+ -- Index operator builds, caches, and returns Lua function
+ -- corresponding to comprehension expression string.
+ --
+ -- Example: f = comprehension['x^2 for x']
+ --
+ function mt:__index(expr)
+ local f = build_comprehension(expr, env)
+ self[expr] = f -- cache
+ return f
+ end
+
+ -- Convenience syntax.
+ -- Allows comprehension 'x^2 for x' instead of comprehension['x^2 for x'].
+ mt.__call = mt.__index
+
+ cache.new = new
+
+ return cache
+end
+
+
+local comprehension = {}
+comprehension.new = new
+
+return comprehension
--- /dev/null
+--- Reads configuration files into a Lua table.
+-- Understands INI files, classic Unix config files, and simple
+-- delimited columns of values.
+--
+-- # test.config
+-- # Read timeout in seconds
+-- read.timeout=10
+-- # Write timeout in seconds
+-- write.timeout=5
+-- #acceptable ports
+-- ports = 1002,1003,1004
+--
+-- -- readconfig.lua
+-- require 'pl'
+-- local t = config.read 'test.config'
+-- print(pretty.write(t))
+--
+-- ### output #####
+-- {
+-- ports = {
+-- 1002,
+-- 1003,
+-- 1004
+-- },
+-- write_timeout = 5,
+-- read_timeout = 10
+-- }
+--
+-- See the Guide for further @{06-data.md.Reading_Configuration_Files|discussion}
+--
+-- Dependencies: none
+-- @module pl.config
+
+local type,tonumber,ipairs,io, table = _G.type,_G.tonumber,_G.ipairs,_G.io,_G.table
+
+local function split(s,re)
+ local res = {}
+ local t_insert = table.insert
+ re = '[^'..re..']+'
+ for k in s:gmatch(re) do t_insert(res,k) end
+ return res
+end
+
+local function strip(s)
+ return s:gsub('^%s+',''):gsub('%s+$','')
+end
+
+local function strip_quotes (s)
+ return s:gsub("['\"](.*)['\"]",'%1')
+end
+
+local config = {}
+
+--- like io.lines(), but allows for lines to be continued with '\'.
+-- @param file a file-like object (anything where read() returns the next line) or a filename.
+-- Defaults to stardard input.
+-- @return an iterator over the lines, or nil
+-- @return error 'not a file-like object' or 'file is nil'
+function config.lines(file)
+ local f,openf,err
+ local line = ''
+ if type(file) == 'string' then
+ f,err = io.open(file,'r')
+ if not f then return nil,err end
+ openf = true
+ else
+ f = file or io.stdin
+ if not file.read then return nil, 'not a file-like object' end
+ end
+ if not f then return nil, 'file is nil' end
+ return function()
+ local l = f:read()
+ while l do
+ -- does the line end with '\'?
+ local i = l:find '\\%s*$'
+ if i then -- if so,
+ line = line..l:sub(1,i-1)
+ elseif line == '' then
+ return l
+ else
+ l = line..l
+ line = ''
+ return l
+ end
+ l = f:read()
+ end
+ if openf then f:close() end
+ end
+end
+
+--- read a configuration file into a table
+-- @param file either a file-like object or a string, which must be a filename
+-- @param cnfg a configuration table that may contain these fields:
+-- <ul>
+-- <li> variablilize make names into valid Lua identifiers (default true)</li>
+-- <li> convert_numbers try to convert values into numbers (default true)</li>
+-- <li> trim_space ensure that there is no starting or trailing whitespace with values (default true)</li>
+-- <li> trim_quotes remove quotes from strings (default false)</li>
+-- <li> list_delim delimiter to use when separating columns (default ',')</li>
+-- </ul>
+-- @return a table containing items, or nil
+-- @return error message (same as @{config.lines}
+function config.read(file,cnfg)
+ local f,openf,err
+ cnfg = cnfg or {}
+ local function check_cnfg (var,def)
+ local val = cnfg[var]
+ if val == nil then return def else return val end
+ end
+ local t = {}
+ local top_t = t
+ local variablilize = check_cnfg ('variabilize',true)
+ local list_delim = check_cnfg('list_delim',',')
+ local convert_numbers = check_cnfg('convert_numbers',true)
+ local trim_space = check_cnfg('trim_space',true)
+ local trim_quotes = check_cnfg('trim_quotes',false)
+ local ignore_assign = check_cnfg('ignore_assign',false)
+
+ local function process_name(key)
+ if variablilize then
+ key = key:gsub('[^%w]','_')
+ end
+ return key
+ end
+
+ local function process_value(value)
+ if list_delim and value:find(list_delim) then
+ value = split(value,list_delim)
+ for i,v in ipairs(value) do
+ value[i] = process_value(v)
+ end
+ elseif convert_numbers and value:find('^[%d%+%-]') then
+ local val = tonumber(value)
+ if val then value = val end
+ end
+ if type(value) == 'string' then
+ if trim_space then value = strip(value) end
+ if trim_quotes then value = strip_quotes(value) end
+ end
+ return value
+ end
+
+ local iter,err = config.lines(file)
+ if not iter then return nil,err end
+ for line in iter do
+ -- strips comments
+ local ci = line:find('%s*[#;]')
+ if ci then line = line:sub(1,ci-1) end
+ -- and ignore blank lines
+ if line:find('^%s*$') then
+ elseif line:find('^%[') then -- section!
+ local section = process_name(line:match('%[([^%]]+)%]'))
+ t = top_t
+ t[section] = {}
+ t = t[section]
+ else
+ local i1,i2 = line:find('%s*=%s*')
+ if i1 and not ignore_assign then -- key,value assignment
+ local key = process_name(line:sub(1,i1-1))
+ local value = process_value(line:sub(i2+1))
+ t[key] = value
+ else -- a plain list of values...
+ t[#t+1] = process_value(line)
+ end
+ end
+ end
+ return top_t
+end
+
+return config
--- /dev/null
+--- Reading and querying simple tabular data.
+--
+-- data.read 'test.txt'
+-- ==> {{10,20},{2,5},{40,50},fieldnames={'x','y'},delim=','}
+--
+-- Provides a way of creating basic SQL-like queries.
+--
+-- require 'pl'
+-- local d = data.read('xyz.txt')
+-- local q = d:select('x,y,z where x > 3 and z < 2 sort by y')
+-- for x,y,z in q do
+-- print(x,y,z)
+-- end
+--
+-- See @{06-data.md.Reading_Columnar_Data|the Guide}
+--
+-- Dependencies: `pl.utils`, `pl.array2d` (fallback methods)
+-- @module pl.data
+
+local utils = require 'pl.utils'
+local _DEBUG = rawget(_G,'_DEBUG')
+
+local patterns,function_arg,usplit = utils.patterns,utils.function_arg,utils.split
+local append,concat = table.insert,table.concat
+local gsub = string.gsub
+local io = io
+local _G,print,type,tonumber,ipairs,setmetatable,pcall,error,setfenv = _G,print,type,tonumber,ipairs,setmetatable,pcall,error,setfenv
+
+
+local data = {}
+
+local parse_select
+
+local function count(s,chr)
+ chr = utils.escape(chr)
+ local _,cnt = s:gsub(chr,' ')
+ return cnt
+end
+
+local function rstrip(s)
+ return s:gsub('%s+$','')
+end
+
+local function make_list(l)
+ return setmetatable(l,utils.stdmt.List)
+end
+
+local function split(s,delim)
+ return make_list(usplit(s,delim))
+end
+
+local function map(fun,t)
+ local res = {}
+ for i = 1,#t do
+ append(res,fun(t[i]))
+ end
+ return res
+end
+
+local function find(t,v)
+ for i = 1,#t do
+ if v == t[i] then return i end
+ end
+end
+
+local DataMT = {
+ column_by_name = function(self,name)
+ if type(name) == 'number' then
+ name = '$'..name
+ end
+ local arr = {}
+ for res in data.query(self,name) do
+ append(arr,res)
+ end
+ return make_list(arr)
+ end,
+
+ copy_select = function(self,condn)
+ condn = parse_select(condn,self)
+ local iter = data.query(self,condn)
+ local res = {}
+ local row = make_list{iter()}
+ while #row > 0 do
+ append(res,row)
+ row = make_list{iter()}
+ end
+ res.delim = self.delim
+ return data.new(res,split(condn.fields,','))
+ end,
+
+ column_names = function(self)
+ return self.fieldnames
+ end,
+}
+
+local array2d
+
+DataMT.__index = function(self,name)
+ local f = DataMT[name]
+ if f then return f end
+ if not array2d then
+ array2d = require 'pl.array2d'
+ end
+ return array2d[name]
+end
+
+--- return a particular column as a list of values (method).
+-- @param name either name of column, or numerical index.
+-- @function Data.column_by_name
+
+--- return a query iterator on this data (method).
+-- @param condn the query expression
+-- @function Data.select
+-- @see data.query
+
+--- return a row iterator on this data (method).
+-- @param condn the query expression
+-- @function Data.select_row
+
+--- return a new data object based on this query (method).
+-- @param condn the query expression
+-- @function Data.copy_select
+
+--- return the field names of this data object (method).
+-- @function Data.column_names
+
+--- write out a row (method).
+-- @param f file-like object
+-- @function Data.write_row
+
+--- write data out to file (method).
+-- @param f file-like object
+-- @function Data.write
+
+
+-- [guessing delimiter] We check for comma, tab and spaces in that order.
+-- [issue] any other delimiters to be checked?
+local delims = {',','\t',' ',';'}
+
+local function guess_delim (line)
+ if line=='' then return ' ' end
+ for _,delim in ipairs(delims) do
+ if count(line,delim) > 0 then
+ return delim == ' ' and '%s+' or delim
+ end
+ end
+ return ' '
+end
+
+-- [file parameter] If it's a string, we try open as a filename. If nil, then
+-- either stdin or stdout depending on the mode. Otherwise, check if this is
+-- a file-like object (implements read or write depending)
+local function open_file (f,mode)
+ local opened, err
+ local reading = mode == 'r'
+ if type(f) == 'string' then
+ if f == 'stdin' then
+ f = io.stdin
+ elseif f == 'stdout' then
+ f = io.stdout
+ else
+ f,err = io.open(f,mode)
+ if not f then return nil,err end
+ opened = true
+ end
+ end
+ if f and ((reading and not f.read) or (not reading and not f.write)) then
+ return nil, "not a file-like object"
+ end
+ return f,nil,opened
+end
+
+local function all_n ()
+
+end
+
+--- read a delimited file in a Lua table.
+-- By default, attempts to treat first line as separated list of fieldnames.
+-- @param file a filename or a file-like object (default stdin)
+-- @param cnfg options table: can override delim (a string pattern), fieldnames (a list),
+-- specify no_convert (default is to convert), numfields (indices of columns known
+-- to be numbers) and thousands_dot (thousands separator in Excel CSV is '.')
+function data.read(file,cnfg)
+ local convert,err,opened
+ local D = {}
+ if not cnfg then cnfg = {} end
+ local f,err,opened = open_file(file,'r')
+ if not f then return nil, err end
+ local thousands_dot = cnfg.thousands_dot
+
+ local function try_tonumber(x)
+ if thousands_dot then x = x:gsub('%.(...)','%1') end
+ return tonumber(x)
+ end
+
+ local line = f:read()
+ if not line then return nil, "empty file" end
+ -- first question: what is the delimiter?
+ D.delim = cnfg.delim and cnfg.delim or guess_delim(line)
+ local delim = D.delim
+ local collect_end = cnfg.last_field_collect
+ local numfields = cnfg.numfields
+ -- some space-delimited data starts with a space. This should not be a column,
+ -- although it certainly would be for comma-separated, etc.
+ local strip
+ if delim == '%s+' and line:find(delim) == 1 then
+ strip = function(s) return s:gsub('^%s+','') end
+ line = strip(line)
+ end
+ -- first line will usually be field names. Unless fieldnames are specified,
+ -- we check if it contains purely numerical values for the case of reading
+ -- plain data files.
+ if not cnfg.fieldnames then
+ local fields = split(line,delim)
+ local nums = map(tonumber,fields)
+ if #nums == #fields then
+ convert = tonumber
+ append(D,nums)
+ numfields = {}
+ for i = 1,#nums do numfields[i] = i end
+ else
+ cnfg.fieldnames = fields
+ end
+ line = f:read()
+ if strip then line = strip(line) end
+ elseif type(cnfg.fieldnames) == 'string' then
+ cnfg.fieldnames = split(cnfg.fieldnames,delim)
+ end
+ -- at this point, the column headers have been read in. If the first
+ -- row consisted of numbers, it has already been added to the dataset.
+ if cnfg.fieldnames then
+ D.fieldnames = cnfg.fieldnames
+ -- [conversion] unless @no_convert, we need the numerical field indices
+ -- of the first data row. Can also be specified by @numfields.
+ if not cnfg.no_convert then
+ if not numfields then
+ numfields = {}
+ local fields = split(line,D.delim)
+ for i = 1,#fields do
+ if tonumber(fields[i]) then
+ append(numfields,i)
+ end
+ end
+ end
+ if #numfields > 0 then -- there are numerical fields
+ -- note that using dot as the thousands separator (@thousands_dot)
+ -- requires a special conversion function!
+ convert = thousands_dot and try_tonumber or tonumber
+ end
+ end
+ end
+ -- keep going until finished
+ while line do
+ if not line:find ('^%s*$') then
+ if strip then line = strip(line) end
+ local fields = split(line,delim)
+ if convert then
+ for k = 1,#numfields do
+ local i = numfields[k]
+ local val = convert(fields[i])
+ if val == nil then
+ return nil, "not a number: "..fields[i]
+ else
+ fields[i] = val
+ end
+ end
+ end
+ -- [collecting end field] If @last_field_collect then we will collect
+ -- all extra space-delimited fields into a single last field.
+ if collect_end and #fields > #D.fieldnames then
+ local ends,N = {},#D.fieldnames
+ for i = N+1,#fields do
+ append(ends,fields[i])
+ end
+ ends = concat(ends,' ')
+ local cfields = {}
+ for i = 1,N do cfields[i] = fields[i] end
+ cfields[N] = cfields[N]..' '..ends
+ fields = cfields
+ end
+ append(D,fields)
+ end
+ line = f:read()
+ end
+ if opened then f:close() end
+ if delim == '%s+' then D.delim = ' ' end
+ if not D.fieldnames then D.fieldnames = {} end
+ return data.new(D)
+end
+
+local function write_row (data,f,row,delim)
+ f:write(concat(row,delim),'\n')
+end
+
+function DataMT:write_row(f,row)
+ write_row(self,f,row,self.delim)
+end
+
+--- write 2D data to a file.
+-- Does not assume that the data has actually been
+-- generated with `new` or `read`.
+-- @param data 2D array
+-- @param file filename or file-like object
+-- @param fieldnames list of fields (optional)
+-- @param delim delimiter (default tab)
+function data.write (data,file,fieldnames,delim)
+ local f,err,opened = open_file(file,'w')
+ if not f then return nil, err end
+ if fieldnames and #fieldnames > 0 then
+ f:write(concat(data.fieldnames,delim),'\n')
+ end
+ delim = delim or '\t'
+ for i = 1,#data do
+ write_row(data,f,data[i],delim)
+ end
+ if opened then f:close() end
+end
+
+
+function DataMT:write(file)
+ data.write(self,file,self.fieldnames,self.delim)
+end
+
+local function massage_fieldnames (fields)
+ -- [fieldnames must be valid Lua identifiers] fix 0.8 was %A
+ for i = 1,#fields do
+ fields[i] = fields[i]:gsub('%W','_')
+ end
+end
+
+--- create a new dataset from a table of rows. <br>
+-- Can specify the fieldnames, else the table must have a field called
+-- 'fieldnames', which is either a string of delimiter-separated names,
+-- or a table of names. <br>
+-- If the table does not have a field called 'delim', then an attempt will be
+-- made to guess it from the fieldnames string, defaults otherwise to tab.
+-- @param d the table.
+-- @param fieldnames optional fieldnames
+-- @return the table.
+function data.new (d,fieldnames)
+ d.fieldnames = d.fieldnames or fieldnames or ''
+ if not d.delim and type(d.fieldnames) == 'string' then
+ d.delim = guess_delim(d.fieldnames)
+ d.fieldnames = split(d.fieldnames,d.delim)
+ end
+ d.fieldnames = make_list(d.fieldnames)
+ massage_fieldnames(d.fieldnames)
+ setmetatable(d,DataMT)
+ -- a query with just the fieldname will return a sequence
+ -- of values, which seq.copy turns into a table.
+ return d
+end
+
+local sorted_query = [[
+return function (t)
+ local i = 0
+ local v
+ local ls = {}
+ for i,v in ipairs(t) do
+ if CONDITION then
+ ls[#ls+1] = v
+ end
+ end
+ table.sort(ls,function(v1,v2)
+ return SORT_EXPR
+ end)
+ local n = #ls
+ return function()
+ i = i + 1
+ v = ls[i]
+ if i > n then return end
+ return FIELDLIST
+ end
+end
+]]
+
+-- question: is this optimized case actually worth the extra code?
+local simple_query = [[
+return function (t)
+ local n = #t
+ local i = 0
+ local v
+ return function()
+ repeat
+ i = i + 1
+ v = t[i]
+ until i > n or CONDITION
+ if i > n then return end
+ return FIELDLIST
+ end
+end
+]]
+
+local function is_string (s)
+ return type(s) == 'string'
+end
+
+local field_error
+
+local function fieldnames_as_string (data)
+ return concat(data.fieldnames,',')
+end
+
+local function massage_fields(data,f)
+ local idx
+ if f:find '^%d+$' then
+ idx = tonumber(f)
+ else
+ idx = find(data.fieldnames,f)
+ end
+ if idx then
+ return 'v['..idx..']'
+ else
+ field_error = f..' not found in '..fieldnames_as_string(data)
+ return f
+ end
+end
+
+
+local function process_select (data,parms)
+ --- preparing fields ----
+ local res,ret
+ field_error = nil
+ local fields = parms.fields
+ local numfields = fields:find '%$' or #data.fieldnames == 0
+ if fields:find '^%s*%*%s*' then
+ if not numfields then
+ fields = fieldnames_as_string(data)
+ else
+ local ncol = #data[1]
+ fields = {}
+ for i = 1,ncol do append(fields,'$'..i) end
+ fields = concat(fields,',')
+ end
+ end
+ local idpat = patterns.IDEN
+ if numfields then
+ idpat = '%$(%d+)'
+ else
+ -- massage field names to replace non-identifier chars
+ fields = rstrip(fields):gsub('[^,%w]','_')
+ end
+ local massage_fields = utils.bind1(massage_fields,data)
+ ret = gsub(fields,idpat,massage_fields)
+ if field_error then return nil,field_error end
+ parms.fields = fields
+ parms.proc_fields = ret
+ parms.where = parms.where or 'true'
+ if is_string(parms.where) then
+ parms.where = gsub(parms.where,idpat,massage_fields)
+ field_error = nil
+ end
+ return true
+end
+
+
+parse_select = function(s,data)
+ local endp
+ local parms = {}
+ local w1,w2 = s:find('where ')
+ local s1,s2 = s:find('sort by ')
+ if w1 then -- where clause!
+ endp = (s1 or 0)-1
+ parms.where = s:sub(w2+1,endp)
+ end
+ if s1 then -- sort by clause (must be last!)
+ parms.sort_by = s:sub(s2+1)
+ end
+ endp = (w1 or s1 or 0)-1
+ parms.fields = s:sub(1,endp)
+ local status,err = process_select(data,parms)
+ if not status then return nil,err
+ else return parms end
+end
+
+--- create a query iterator from a select string.
+-- Select string has this format: <br>
+-- FIELDLIST [ where LUA-CONDN [ sort by FIELD] ]<br>
+-- FIELDLIST is a comma-separated list of valid fields, or '*'. <br> <br>
+-- The condition can also be a table, with fields 'fields' (comma-sep string or
+-- table), 'sort_by' (string) and 'where' (Lua expression string or function)
+-- @param data table produced by read
+-- @param condn select string or table
+-- @param context a list of tables to be searched when resolving functions
+-- @param return_row if true, wrap the results in a row table
+-- @return an iterator over the specified fields, or nil
+-- @return an error message
+function data.query(data,condn,context,return_row)
+ local err
+ if is_string(condn) then
+ condn,err = parse_select(condn,data)
+ if not condn then return nil,err end
+ elseif type(condn) == 'table' then
+ if type(condn.fields) == 'table' then
+ condn.fields = concat(condn.fields,',')
+ end
+ if not condn.proc_fields then
+ local status,err = process_select(data,condn)
+ if not status then return nil,err end
+ end
+ else
+ return nil, "condition must be a string or a table"
+ end
+ local query, k
+ if condn.sort_by then -- use sorted_query
+ query = sorted_query
+ else
+ query = simple_query
+ end
+ local fields = condn.proc_fields or condn.fields
+ if return_row then
+ fields = '{'..fields..'}'
+ end
+ query,k = query:gsub('FIELDLIST',fields)
+ if is_string(condn.where) then
+ query = query:gsub('CONDITION',condn.where)
+ condn.where = nil
+ else
+ query = query:gsub('CONDITION','_condn(v)')
+ condn.where = function_arg(0,condn.where,'condition.where must be callable')
+ end
+ if condn.sort_by then
+ local expr,sort_var,sort_dir
+ local sort_by = condn.sort_by
+ local i1,i2 = sort_by:find('%s+')
+ if i1 then
+ sort_var,sort_dir = sort_by:sub(1,i1-1),sort_by:sub(i2+1)
+ else
+ sort_var = sort_by
+ sort_dir = 'asc'
+ end
+ if sort_var:match '^%$' then sort_var = sort_var:sub(2) end
+ sort_var = massage_fields(data,sort_var)
+ if field_error then return nil,field_error end
+ if sort_dir == 'asc' then
+ sort_dir = '<'
+ else
+ sort_dir = '>'
+ end
+ expr = ('%s %s %s'):format(sort_var:gsub('v','v1'),sort_dir,sort_var:gsub('v','v2'))
+ query = query:gsub('SORT_EXPR',expr)
+ end
+ if condn.where then
+ query = 'return function(_condn) '..query..' end'
+ end
+ if _DEBUG then print(query) end
+
+ local fn,err = loadstring(query,'tmp')
+ if not fn then return nil,err end
+ fn = fn() -- get the function
+ if condn.where then
+ fn = fn(condn.where)
+ end
+ local qfun = fn(data)
+ if context then
+ -- [specifying context for condition] @context is a list of tables which are
+ -- 'injected'into the condition's custom context
+ append(context,_G)
+ local lookup = {}
+ setfenv(qfun,lookup)
+ setmetatable(lookup,{
+ __index = function(tbl,key)
+ -- _G.print(tbl,key)
+ for k,t in ipairs(context) do
+ if t[key] then return t[key] end
+ end
+ end
+ })
+ end
+ return qfun
+end
+
+
+DataMT.select = data.query
+DataMT.select_row = function(d,condn,context)
+ return data.query(d,condn,context,true)
+end
+
+--- Filter input using a query.
+-- @param Q a query string
+-- @param infile filename or file-like object
+-- @param outfile filename or file-like object
+-- @param dont_fail true if you want to return an error, not just fail
+function data.filter (Q,infile,outfile,dont_fail)
+ local err
+ local d = data.read(infile or 'stdin')
+ local out = open_file(outfile or 'stdout')
+ local iter,err = d:select(Q)
+ local delim = d.delim
+ if not iter then
+ err = 'error: '..err
+ if dont_fail then
+ return nil,err
+ else
+ utils.quit(1,err)
+ end
+ end
+ while true do
+ local res = {iter()}
+ if #res == 0 then break end
+ out:write(concat(res,delim),'\n')
+ end
+end
+
+return data
+
--- /dev/null
+--- Useful functions for getting directory contents and matching them against wildcards.
+--
+-- Dependencies: `pl.utils`, `pl.path`, `pl.tablex`
+--
+-- Soft Dependencies: `alien`, `ffi` (either are used on Windows for copying/moving files)
+-- @module pl.dir
+
+local utils = require 'pl.utils'
+local path = require 'pl.path'
+local is_windows = path.is_windows
+local tablex = require 'pl.tablex'
+local ldir = path.dir
+local chdir = path.chdir
+local mkdir = path.mkdir
+local rmdir = path.rmdir
+local sub = string.sub
+local os,pcall,ipairs,pairs,require,setmetatable,_G = os,pcall,ipairs,pairs,require,setmetatable,_G
+local remove = os.remove
+local append = table.insert
+local wrap = coroutine.wrap
+local yield = coroutine.yield
+local assert_arg,assert_string,raise = utils.assert_arg,utils.assert_string,utils.raise
+local List = utils.stdmt.List
+
+local dir = {}
+
+local function assert_dir (n,val)
+ assert_arg(n,val,'string',path.isdir,'not a directory')
+end
+
+local function assert_file (n,val)
+ assert_arg(n,val,'string',path.isfile,'not a file')
+end
+
+local function filemask(mask)
+ mask = utils.escape(mask)
+ return mask:gsub('%%%*','.+'):gsub('%%%?','.')..'$'
+end
+
+--- does the filename match the shell pattern?.
+-- (cf. fnmatch.fnmatch in Python, 11.8)
+-- @param file A file name
+-- @param pattern A shell pattern
+-- @return true or false
+-- @raise file and pattern must be strings
+function dir.fnmatch(file,pattern)
+ assert_string(1,file)
+ assert_string(2,pattern)
+ return path.normcase(file):find(filemask(pattern)) ~= nil
+end
+
+--- return a list of all files which match the pattern.
+-- (cf. fnmatch.filter in Python, 11.8)
+-- @param files A table containing file names
+-- @param pattern A shell pattern.
+-- @return list of files
+-- @raise file and pattern must be strings
+function dir.filter(files,pattern)
+ assert_arg(1,files,'table')
+ assert_string(2,pattern)
+ local res = {}
+ local mask = filemask(pattern)
+ for i,f in ipairs(files) do
+ if f:find(mask) then append(res,f) end
+ end
+ return setmetatable(res,List)
+end
+
+local function _listfiles(dir,filemode,match)
+ local res = {}
+ local check = utils.choose(filemode,path.isfile,path.isdir)
+ if not dir then dir = '.' end
+ for f in ldir(dir) do
+ if f ~= '.' and f ~= '..' then
+ local p = path.join(dir,f)
+ if check(p) and (not match or match(p)) then
+ append(res,p)
+ end
+ end
+ end
+ return setmetatable(res,List)
+end
+
+--- return a list of all files in a directory which match the a shell pattern.
+-- @param dir A directory. If not given, all files in current directory are returned.
+-- @param mask A shell pattern. If not given, all files are returned.
+-- @return lsit of files
+-- @raise dir and mask must be strings
+function dir.getfiles(dir,mask)
+ assert_dir(1,dir)
+ assert_string(2,mask)
+ local match
+ if mask then
+ mask = filemask(mask)
+ match = function(f)
+ return f:find(mask)
+ end
+ end
+ return _listfiles(dir,true,match)
+end
+
+--- return a list of all subdirectories of the directory.
+-- @param dir A directory
+-- @return a list of directories
+-- @raise dir must be a string
+function dir.getdirectories(dir)
+ assert_dir(1,dir)
+ return _listfiles(dir,false)
+end
+
+local function quote_argument (f)
+ f = path.normcase(f)
+ if f:find '%s' then
+ return '"'..f..'"'
+ else
+ return f
+ end
+end
+
+
+local alien,ffi,ffi_checked,CopyFile,MoveFile,GetLastError,win32_errors,cmd_tmpfile
+
+local function execute_command(cmd,parms)
+ if not cmd_tmpfile then cmd_tmpfile = path.tmpname () end
+ local err = path.is_windows and ' > ' or ' 2> '
+ cmd = cmd..' '..parms..err..cmd_tmpfile
+ local ret = utils.execute(cmd)
+ if not ret then
+ return false,(utils.readfile(cmd_tmpfile):gsub('\n(.*)',''))
+ else
+ return true
+ end
+end
+
+local function find_ffi_copyfile ()
+ if not ffi_checked then
+ ffi_checked = true
+ local res
+ res,alien = pcall(require,'alien')
+ if not res then
+ alien = nil
+ res, ffi = pcall(require,'ffi')
+ end
+ if not res then
+ ffi = nil
+ return
+ end
+ else
+ return
+ end
+ if alien then
+ -- register the Win32 CopyFile and MoveFile functions
+ local kernel = alien.load('kernel32.dll')
+ CopyFile = kernel.CopyFileA
+ CopyFile:types{'string','string','int',ret='int',abi='stdcall'}
+ MoveFile = kernel.MoveFileA
+ MoveFile:types{'string','string',ret='int',abi='stdcall'}
+ GetLastError = kernel.GetLastError
+ GetLastError:types{ret ='int', abi='stdcall'}
+ elseif ffi then
+ ffi.cdef [[
+ int CopyFileA(const char *src, const char *dest, int iovr);
+ int MoveFileA(const char *src, const char *dest);
+ int GetLastError();
+ ]]
+ CopyFile = ffi.C.CopyFileA
+ MoveFile = ffi.C.MoveFileA
+ GetLastError = ffi.C.GetLastError
+ end
+ win32_errors = {
+ ERROR_FILE_NOT_FOUND = 2,
+ ERROR_PATH_NOT_FOUND = 3,
+ ERROR_ACCESS_DENIED = 5,
+ ERROR_WRITE_PROTECT = 19,
+ ERROR_BAD_UNIT = 20,
+ ERROR_NOT_READY = 21,
+ ERROR_WRITE_FAULT = 29,
+ ERROR_READ_FAULT = 30,
+ ERROR_SHARING_VIOLATION = 32,
+ ERROR_LOCK_VIOLATION = 33,
+ ERROR_HANDLE_DISK_FULL = 39,
+ ERROR_BAD_NETPATH = 53,
+ ERROR_NETWORK_BUSY = 54,
+ ERROR_DEV_NOT_EXIST = 55,
+ ERROR_FILE_EXISTS = 80,
+ ERROR_OPEN_FAILED = 110,
+ ERROR_INVALID_NAME = 123,
+ ERROR_BAD_PATHNAME = 161,
+ ERROR_ALREADY_EXISTS = 183,
+ }
+end
+
+local function two_arguments (f1,f2)
+ return quote_argument(f1)..' '..quote_argument(f2)
+end
+
+local function file_op (is_copy,src,dest,flag)
+ if flag == 1 and path.exists(dest) then
+ return false,"cannot overwrite destination"
+ end
+ if is_windows then
+ -- if we haven't tried to load Alien/LuaJIT FFI before, then do so
+ find_ffi_copyfile()
+ -- fallback if there's no Alien, just use DOS commands *shudder*
+ -- 'rename' involves a copy and then deleting the source.
+ if not CopyFile then
+ src = path.normcase(src)
+ dest = path.normcase(dest)
+ local cmd = is_copy and 'copy' or 'rename'
+ local res, err = execute_command('copy',two_arguments(src,dest))
+ if not res then return nil,err end
+ if not is_copy then
+ return execute_command('del',quote_argument(src))
+ end
+ else
+ if path.isdir(dest) then
+ dest = path.join(dest,path.basename(src))
+ end
+ local ret
+ if is_copy then ret = CopyFile(src,dest,flag)
+ else ret = MoveFile(src,dest) end
+ if ret == 0 then
+ local err = GetLastError()
+ for name,value in pairs(win32_errors) do
+ if value == err then return false,name end
+ end
+ return false,"Error #"..err
+ else return true
+ end
+ end
+ else -- for Unix, just use cp for now
+ return execute_command(is_copy and 'cp' or 'mv',
+ two_arguments(src,dest))
+ end
+end
+
+--- copy a file.
+-- @param src source file
+-- @param dest destination file or directory
+-- @param flag true if you want to force the copy (default)
+-- @return true if operation succeeded
+-- @raise src and dest must be strings
+function dir.copyfile (src,dest,flag)
+ assert_string(1,src)
+ assert_string(2,dest)
+ flag = flag==nil or flag
+ return file_op(true,src,dest,flag and 0 or 1)
+end
+
+--- move a file.
+-- @param src source file
+-- @param dest destination file or directory
+-- @return true if operation succeeded
+-- @raise src and dest must be strings
+function dir.movefile (src,dest)
+ assert_string(1,src)
+ assert_string(2,dest)
+ return file_op(false,src,dest,0)
+end
+
+local function _dirfiles(dir,attrib)
+ local dirs = {}
+ local files = {}
+ for f in ldir(dir) do
+ if f ~= '.' and f ~= '..' then
+ local p = path.join(dir,f)
+ local mode = attrib(p,'mode')
+ if mode=='directory' then
+ append(dirs,f)
+ else
+ append(files,f)
+ end
+ end
+ end
+ return setmetatable(dirs,List),setmetatable(files,List)
+end
+
+
+local function _walker(root,bottom_up,attrib)
+ local dirs,files = _dirfiles(root,attrib)
+ if not bottom_up then yield(root,dirs,files) end
+ for i,d in ipairs(dirs) do
+ _walker(root..path.sep..d,bottom_up,attrib)
+ end
+ if bottom_up then yield(root,dirs,files) end
+end
+
+--- return an iterator which walks through a directory tree starting at root.
+-- The iterator returns (root,dirs,files)
+-- Note that dirs and files are lists of names (i.e. you must say path.join(root,d)
+-- to get the actual full path)
+-- If bottom_up is false (or not present), then the entries at the current level are returned
+-- before we go deeper. This means that you can modify the returned list of directories before
+-- continuing.
+-- This is a clone of os.walk from the Python libraries.
+-- @param root A starting directory
+-- @param bottom_up False if we start listing entries immediately.
+-- @param follow_links follow symbolic links
+-- @return an iterator returning root,dirs,files
+-- @raise root must be a string
+function dir.walk(root,bottom_up,follow_links)
+ assert_string(1,root)
+ if not path.isdir(root) then return raise 'not a directory' end
+ local attrib
+ if path.is_windows or not follow_links then
+ attrib = path.attrib
+ else
+ attrib = path.link_attrib
+ end
+ return wrap(function () _walker(root,bottom_up,attrib) end)
+end
+
+--- remove a whole directory tree.
+-- @param fullpath A directory path
+-- @return true or nil
+-- @return error if failed
+-- @raise fullpath must be a string
+function dir.rmtree(fullpath)
+ assert_string(1,fullpath)
+ if not path.isdir(fullpath) then return raise 'not a directory' end
+ if path.islink(fullpath) then return false,'will not follow symlink' end
+ for root,dirs,files in dir.walk(fullpath,true) do
+ for i,f in ipairs(files) do
+ remove(path.join(root,f))
+ end
+ rmdir(root)
+ end
+ return true
+end
+
+local dirpat
+if path.is_windows then
+ dirpat = '(.+)\\[^\\]+$'
+else
+ dirpat = '(.+)/[^/]+$'
+end
+
+local _makepath
+function _makepath(p)
+ -- windows root drive case
+ if p:find '^%a:[\\]*$' then
+ return true
+ end
+ if not path.isdir(p) then
+ local subp = p:match(dirpat)
+ if not _makepath(subp) then return raise ('cannot create '..subp) end
+ return mkdir(p)
+ else
+ return true
+ end
+end
+
+--- create a directory path.
+-- This will create subdirectories as necessary!
+-- @param p A directory path
+-- @return a valid created path
+-- @raise p must be a string
+function dir.makepath (p)
+ assert_string(1,p)
+ return _makepath(path.normcase(path.abspath(p)))
+end
+
+
+--- clone a directory tree. Will always try to create a new directory structure
+-- if necessary.
+-- @param path1 the base path of the source tree
+-- @param path2 the new base path for the destination
+-- @param file_fun an optional function to apply on all files
+-- @param verbose an optional boolean to control the verbosity of the output.
+-- It can also be a logging function that behaves like print()
+-- @return true, or nil
+-- @return error message, or list of failed directory creations
+-- @return list of failed file operations
+-- @raise path1 and path2 must be strings
+-- @usage clonetree('.','../backup',copyfile)
+function dir.clonetree (path1,path2,file_fun,verbose)
+ assert_string(1,path1)
+ assert_string(2,path2)
+ if verbose == true then verbose = print end
+ local abspath,normcase,isdir,join = path.abspath,path.normcase,path.isdir,path.join
+ local faildirs,failfiles = {},{}
+ if not isdir(path1) then return raise 'source is not a valid directory' end
+ path1 = abspath(normcase(path1))
+ path2 = abspath(normcase(path2))
+ if verbose then verbose('normalized:',path1,path2) end
+ -- particularly NB that the new path isn't fully contained in the old path
+ if path1 == path2 then return raise "paths are the same" end
+ local i1,i2 = path2:find(path1,1,true)
+ if i2 == #path1 and path2:sub(i2+1,i2+1) == path.sep then
+ return raise 'destination is a subdirectory of the source'
+ end
+ local cp = path.common_prefix (path1,path2)
+ local idx = #cp
+ if idx == 0 then -- no common path, but watch out for Windows paths!
+ if path1:sub(2,2) == ':' then idx = 3 end
+ end
+ for root,dirs,files in dir.walk(path1) do
+ local opath = path2..root:sub(idx)
+ if verbose then verbose('paths:',opath,root) end
+ if not isdir(opath) then
+ local ret = dir.makepath(opath)
+ if not ret then append(faildirs,opath) end
+ if verbose then verbose('creating:',opath,ret) end
+ end
+ if file_fun then
+ for i,f in ipairs(files) do
+ local p1 = join(root,f)
+ local p2 = join(opath,f)
+ local ret = file_fun(p1,p2)
+ if not ret then append(failfiles,p2) end
+ if verbose then
+ verbose('files:',p1,p2,ret)
+ end
+ end
+ end
+ end
+ return true,faildirs,failfiles
+end
+
+--- return an iterator over all entries in a directory tree
+-- @param d a directory
+-- @return an iterator giving pathname and mode (true for dir, false otherwise)
+-- @raise d must be a non-empty string
+function dir.dirtree( d )
+ assert( d and d ~= "", "directory parameter is missing or empty" )
+ local exists, isdir = path.exists, path.isdir
+ local sep = path.sep
+
+ local last = sub ( d, -1 )
+ if last == sep or last == '/' then
+ d = sub( d, 1, -2 )
+ end
+
+ local function yieldtree( dir )
+ for entry in ldir( dir ) do
+ if entry ~= "." and entry ~= ".." then
+ entry = dir .. sep .. entry
+ if exists(entry) then -- Just in case a symlink is broken.
+ local is_dir = isdir(entry)
+ yield( entry, is_dir )
+ if is_dir then
+ yieldtree( entry )
+ end
+ end
+ end
+ end
+ end
+
+ return wrap( function() yieldtree( d ) end )
+end
+
+
+--- Recursively returns all the file starting at <i>path</i>. It can optionally take a shell pattern and
+-- only returns files that match <i>pattern</i>. If a pattern is given it will do a case insensitive search.
+-- @param start_path {string} A directory. If not given, all files in current directory are returned.
+-- @param pattern {string} A shell pattern. If not given, all files are returned.
+-- @return Table containing all the files found recursively starting at <i>path</i> and filtered by <i>pattern</i>.
+-- @raise start_path must be a string
+function dir.getallfiles( start_path, pattern )
+ assert( type( start_path ) == "string", "bad argument #1 to 'GetAllFiles' (Expected string but recieved " .. type( start_path ) .. ")" )
+ pattern = pattern or ""
+
+ local files = {}
+ local normcase = path.normcase
+ for filename, mode in dir.dirtree( start_path ) do
+ if not mode then
+ local mask = filemask( pattern )
+ if normcase(filename):find( mask ) then
+ files[#files + 1] = filename
+ end
+ end
+ end
+
+ return files
+end
+
+return dir
--- /dev/null
+--- File manipulation functions: reading, writing, moving and copying.
+--
+-- Dependencies: `pl.utils`, `pl.dir`, `pl.path`
+-- @module pl.file
+local os = os
+local utils = require 'pl.utils'
+local dir = require 'pl.dir'
+local path = require 'pl.path'
+
+--[[
+module ('pl.file',utils._module)
+]]
+local file = {}
+
+--- return the contents of a file as a string
+-- @class function
+-- @name file.read
+-- @param filename The file path
+-- @return file contents
+file.read = utils.readfile
+
+--- write a string to a file
+-- @class function
+-- @name file.write
+-- @param filename The file path
+-- @param str The string
+file.write = utils.writefile
+
+--- copy a file.
+-- @class function
+-- @name file.copy
+-- @param src source file
+-- @param dest destination file
+-- @param flag true if you want to force the copy (default)
+-- @return true if operation succeeded
+file.copy = dir.copyfile
+
+--- move a file.
+-- @class function
+-- @name file.move
+-- @param src source file
+-- @param dest destination file
+-- @return true if operation succeeded, else false and the reason for the error.
+file.move = dir.movefile
+
+--- Return the time of last access as the number of seconds since the epoch.
+-- @class function
+-- @name file.access_time
+-- @param path A file path
+file.access_time = path.getatime
+
+---Return when the file was created.
+-- @class function
+-- @name file.creation_time
+-- @param path A file path
+file.creation_time = path.getctime
+
+--- Return the time of last modification
+-- @class function
+-- @name file.modified_time
+-- @param path A file path
+file.modified_time = path.getmtime
+
+--- Delete a file
+-- @class function
+-- @name file.delete
+-- @param path A file path
+file.delete = os.remove
+
+return file
--- /dev/null
+--- Functional helpers like composition, binding and placeholder expressions.
+-- Placeholder expressions are useful for short anonymous functions, and were
+-- inspired by the Boost Lambda library.
+--
+-- > utils.import 'pl.func'
+-- > ls = List{10,20,30}
+-- > = ls:map(_1+1)
+-- {11,21,31}
+--
+-- They can also be used to _bind_ particular arguments of a function.
+--
+-- > p = bind(print,'start>',_0)
+-- > p(10,20,30)
+-- > start> 10 20 30
+--
+-- See @{07-functional.md.Creating_Functions_from_Functions|the Guide}
+--
+-- Dependencies: `pl.utils`, `pl.tablex`
+-- @module pl.func
+local type,select,setmetatable,getmetatable,rawset = type,select,setmetatable,getmetatable,rawset
+local concat,append = table.concat,table.insert
+local max = math.max
+local print,tostring = print,tostring
+local pairs,ipairs,loadstring,rawget,unpack = pairs,ipairs,loadstring,rawget,unpack
+local _G = _G
+local utils = require 'pl.utils'
+local tablex = require 'pl.tablex'
+local map = tablex.map
+local _DEBUG = rawget(_G,'_DEBUG')
+local assert_arg = utils.assert_arg
+
+local func = {}
+
+-- metatable for Placeholder Expressions (PE)
+local _PEMT = {}
+
+local function P (t)
+ setmetatable(t,_PEMT)
+ return t
+end
+
+func.PE = P
+
+local function isPE (obj)
+ return getmetatable(obj) == _PEMT
+end
+
+func.isPE = isPE
+
+-- construct a placeholder variable (e.g _1 and _2)
+local function PH (idx)
+ return P {op='X',repr='_'..idx, index=idx}
+end
+
+-- construct a constant placeholder variable (e.g _C1 and _C2)
+local function CPH (idx)
+ return P {op='X',repr='_C'..idx, index=idx}
+end
+
+func._1,func._2,func._3,func._4,func._5 = PH(1),PH(2),PH(3),PH(4),PH(5)
+func._0 = P{op='X',repr='...',index=0}
+
+function func.Var (name)
+ local ls = utils.split(name,'[%s,]+')
+ local res = {}
+ for _,n in ipairs(ls) do
+ append(res,P{op='X',repr=n,index=0})
+ end
+ return unpack(res)
+end
+
+function func._ (value)
+ return P{op='X',repr=value,index='wrap'}
+end
+
+local repr
+
+func.Nil = func.Var 'nil'
+
+function _PEMT.__index(obj,key)
+ return P{op='[]',obj,key}
+end
+
+function _PEMT.__call(fun,...)
+ return P{op='()',fun,...}
+end
+
+function _PEMT.__tostring (e)
+ return repr(e)
+end
+
+function _PEMT.__unm(arg)
+ return P{op='-',arg}
+end
+
+function func.Not (arg)
+ return P{op='not',arg}
+end
+
+function func.Len (arg)
+ return P{op='#',arg}
+end
+
+
+local function binreg(context,t)
+ for name,op in pairs(t) do
+ rawset(context,name,function(x,y)
+ return P{op=op,x,y}
+ end)
+ end
+end
+
+local function import_name (name,fun,context)
+ rawset(context,name,function(...)
+ return P{op='()',fun,...}
+ end)
+end
+
+local imported_functions = {}
+
+local function is_global_table (n)
+ return type(_G[n]) == 'table'
+end
+
+--- wrap a table of functions. This makes them available for use in
+-- placeholder expressions.
+-- @param tname a table name
+-- @param context context to put results, defaults to environment of caller
+function func.import(tname,context)
+ assert_arg(1,tname,'string',is_global_table,'arg# 1: not a name of a global table')
+ local t = _G[tname]
+ context = context or _G
+ for name,fun in pairs(t) do
+ import_name(name,fun,context)
+ imported_functions[fun] = name
+ end
+end
+
+--- register a function for use in placeholder expressions.
+-- @param fun a function
+-- @param name an optional name
+-- @return a placeholder functiond
+function func.register (fun,name)
+ assert_arg(1,fun,'function')
+ if name then
+ assert_arg(2,name,'string')
+ imported_functions[fun] = name
+ end
+ return function(...)
+ return P{op='()',fun,...}
+ end
+end
+
+function func.lookup_imported_name (fun)
+ return imported_functions[fun]
+end
+
+local function _arg(...) return ... end
+
+function func.Args (...)
+ return P{op='()',_arg,...}
+end
+
+-- binary and unary operators, with their precedences (see 2.5.6)
+local operators = {
+ ['or'] = 0,
+ ['and'] = 1,
+ ['=='] = 2, ['~='] = 2, ['<'] = 2, ['>'] = 2, ['<='] = 2, ['>='] = 2,
+ ['..'] = 3,
+ ['+'] = 4, ['-'] = 4,
+ ['*'] = 5, ['/'] = 5, ['%'] = 5,
+ ['not'] = 6, ['#'] = 6, ['-'] = 6,
+ ['^'] = 7
+}
+
+-- comparisons (as prefix functions)
+binreg (func,{And='and',Or='or',Eq='==',Lt='<',Gt='>',Le='<=',Ge='>='})
+
+-- standard binary operators (as metamethods)
+binreg (_PEMT,{__add='+',__sub='-',__mul='*',__div='/',__mod='%',__pow='^',__concat='..'})
+
+binreg (_PEMT,{__eq='=='})
+
+--- all elements of a table except the first.
+-- @param ls a list-like table.
+function func.tail (ls)
+ assert_arg(1,ls,'table')
+ local res = {}
+ for i = 2,#ls do
+ append(res,ls[i])
+ end
+ return res
+end
+
+--- create a string representation of a placeholder expression.
+-- @param e a placeholder expression
+-- @param lastpred not used
+function repr (e,lastpred)
+ local tail = func.tail
+ if isPE(e) then
+ local pred = operators[e.op]
+ local ls = map(repr,e,pred)
+ if pred then --unary or binary operator
+ if #ls ~= 1 then
+ local s = concat(ls,' '..e.op..' ')
+ if lastpred and lastpred > pred then
+ s = '('..s..')'
+ end
+ return s
+ else
+ return e.op..' '..ls[1]
+ end
+ else -- either postfix, or a placeholder
+ if e.op == '[]' then
+ return ls[1]..'['..ls[2]..']'
+ elseif e.op == '()' then
+ local fn
+ if ls[1] ~= nil then -- was _args, undeclared!
+ fn = ls[1]
+ else
+ fn = ''
+ end
+ return fn..'('..concat(tail(ls),',')..')'
+ else
+ return e.repr
+ end
+ end
+ elseif type(e) == 'string' then
+ return '"'..e..'"'
+ elseif type(e) == 'function' then
+ local name = func.lookup_imported_name(e)
+ if name then return name else return tostring(e) end
+ else
+ return tostring(e) --should not really get here!
+ end
+end
+func.repr = repr
+
+-- collect all the non-PE values in this PE into vlist, and replace each occurence
+-- with a constant PH (_C1, etc). Return the maximum placeholder index found.
+local collect_values
+function collect_values (e,vlist)
+ if isPE(e) then
+ if e.op ~= 'X' then
+ local m = 0
+ for i,subx in ipairs(e) do
+ local pe = isPE(subx)
+ if pe then
+ if subx.op == 'X' and subx.index == 'wrap' then
+ subx = subx.repr
+ pe = false
+ else
+ m = max(m,collect_values(subx,vlist))
+ end
+ end
+ if not pe then
+ append(vlist,subx)
+ e[i] = CPH(#vlist)
+ end
+ end
+ return m
+ else -- was a placeholder, it has an index...
+ return e.index
+ end
+ else -- plain value has no placeholder dependence
+ return 0
+ end
+end
+func.collect_values = collect_values
+
+--- instantiate a PE into an actual function. First we find the largest placeholder used,
+-- e.g. _2; from this a list of the formal parameters can be build. Then we collect and replace
+-- any non-PE values from the PE, and build up a constant binding list.
+-- Finally, the expression can be compiled, and e.__PE_function is set.
+-- @param e a placeholder expression
+-- @return a function
+function func.instantiate (e)
+ local consts,values,parms = {},{},{}
+ local rep, err, fun
+ local n = func.collect_values(e,values)
+ for i = 1,#values do
+ append(consts,'_C'..i)
+ if _DEBUG then print(i,values[i]) end
+ end
+ for i =1,n do
+ append(parms,'_'..i)
+ end
+ consts = concat(consts,',')
+ parms = concat(parms,',')
+ rep = repr(e)
+ local fstr = ('return function(%s) return function(%s) return %s end end'):format(consts,parms,rep)
+ if _DEBUG then print(fstr) end
+ fun,err = loadstring(fstr,'fun')
+ if not fun then return nil,err end
+ fun = fun() -- get wrapper
+ fun = fun(unpack(values)) -- call wrapper (values could be empty)
+ e.__PE_function = fun
+ return fun
+end
+
+--- instantiate a PE unless it has already been done.
+-- @param e a placeholder expression
+-- @return the function
+function func.I(e)
+ if rawget(e,'__PE_function') then
+ return e.__PE_function
+ else return func.instantiate(e)
+ end
+end
+
+utils.add_function_factory(_PEMT,func.I)
+
+--- bind the first parameter of the function to a value.
+-- @class function
+-- @name func.curry
+-- @param fn a function of one or more arguments
+-- @param p a value
+-- @return a function of one less argument
+-- @usage (curry(math.max,10))(20) == math.max(10,20)
+func.curry = utils.bind1
+
+--- create a function which chains two functions.
+-- @param f a function of at least one argument
+-- @param g a function of at least one argument
+-- @return a function
+-- @usage printf = compose(io.write,string.format)
+function func.compose (f,g)
+ return function(...) return f(g(...)) end
+end
+
+--- bind the arguments of a function to given values.
+-- bind(fn,v,_2) is equivalent to curry(fn,v).
+-- @param fn a function of at least one argument
+-- @param ... values or placeholder variables
+-- @return a function
+-- @usage (bind(f,_1,a))(b) == f(a,b)
+-- @usage (bind(f,_2,_1))(a,b) == f(b,a)
+function func.bind(fn,...)
+ local args = table.pack(...)
+ local holders,parms,bvalues,values = {},{},{'fn'},{}
+ local nv,maxplace,varargs = 1,0,false
+ for i = 1,args.n do
+ local a = args[i]
+ if isPE(a) and a.op == 'X' then
+ append(holders,a.repr)
+ maxplace = max(maxplace,a.index)
+ if a.index == 0 then varargs = true end
+ else
+ local v = '_v'..nv
+ append(bvalues,v)
+ append(holders,v)
+ append(values,a)
+ nv = nv + 1
+ end
+ end
+ for np = 1,maxplace do
+ append(parms,'_'..np)
+ end
+ if varargs then append(parms,'...') end
+ bvalues = concat(bvalues,',')
+ parms = concat(parms,',')
+ holders = concat(holders,',')
+ local fstr = ([[
+return function (%s)
+ return function(%s) return fn(%s) end
+end
+]]):format(bvalues,parms,holders)
+ if _DEBUG then print(fstr) end
+ local res,err = loadstring(fstr)
+ res = res()
+ return res(fn,unpack(values))
+end
+
+return func
+
+
--- /dev/null
+--------------
+-- Entry point for loading all PL libraries only on demand.
+-- Requiring 'pl' means that whenever a module is implicitly accesssed
+-- (e.g. `utils.split`)
+-- then that module is dynamically loaded. The submodules are all brought into
+-- the global space.
+-- @module pl
+
+local modules = {
+ utils = true,path=true,dir=true,tablex=true,stringio=true,sip=true,
+ input=true,seq=true,lexer=true,stringx=true,
+ config=true,pretty=true,data=true,func=true,text=true,
+ operator=true,lapp=true,array2d=true,
+ comprehension=true,xml=true,
+ test = true, app = true, file = true, class = true, List = true,
+ Map = true, Set = true, OrderedMap = true, MultiMap = true,
+ Date = true,
+ -- classes --
+}
+_G.utils = require 'pl.utils'
+
+for name,klass in pairs(_G.utils.stdmt) do
+ klass.__index = function(t,key)
+ return require ('pl.'..name)[key]
+ end;
+end
+
+-- ensure that we play nice with libraries that also attach a metatable
+-- to the global table; always forward to a custom __index if we don't
+-- match
+
+local _hook,_prev_index
+local gmt = {}
+local prev_gmt = getmetatable(_G)
+if prev_gmt then
+ _prev_index = prev_gmt.__index
+ if prev_gmt.__newindex then
+ gmt.__index = prev_gmt.__newindex
+ end
+end
+
+function gmt.hook(handler)
+ _hook = handler
+end
+
+function gmt.__index(t,name)
+ local found = modules[name]
+ -- either true, or the name of the module containing this class.
+ -- either way, we load the required module and make it globally available.
+ if found then
+ -- e..g pretty.dump causes pl.pretty to become available as 'pretty'
+ rawset(_G,name,require('pl.'..name))
+ return _G[name]
+ else
+ local res
+ if _hook then
+ res = _hook(t,name)
+ if res then return res end
+ end
+ if _prev_index then
+ return _prev_index(t,name)
+ end
+ end
+end
+
+setmetatable(_G,gmt)
+
+if rawget(_G,'PENLIGHT_STRICT') then require 'pl.strict' end
--- /dev/null
+--- Iterators for extracting words or numbers from an input source.
+--
+-- require 'pl'
+-- local total,n = seq.sum(input.numbers())
+-- print('average',total/n)
+--
+-- See @{06-data.md.Reading_Unstructured_Text_Data|here}
+--
+-- Dependencies: `pl.utils`
+-- @module pl.input
+local strfind = string.find
+local strsub = string.sub
+local strmatch = string.match
+local utils = require 'pl.utils'
+local pairs,type,unpack,tonumber = pairs,type,unpack or table.unpack,tonumber
+local patterns = utils.patterns
+local io = io
+local assert_arg = utils.assert_arg
+
+--[[
+module ('pl.input',utils._module)
+]]
+
+local input = {}
+
+--- create an iterator over all tokens.
+-- based on allwords from PiL, 7.1
+-- @param getter any function that returns a line of text
+-- @param pattern
+-- @param fn Optionally can pass a function to process each token as it/s found.
+-- @return an iterator
+function input.alltokens (getter,pattern,fn)
+ local line = getter() -- current line
+ local pos = 1 -- current position in the line
+ return function () -- iterator function
+ while line do -- repeat while there are lines
+ local s, e = strfind(line, pattern, pos)
+ if s then -- found a word?
+ pos = e + 1 -- next position is after this token
+ local res = strsub(line, s, e) -- return the token
+ if fn then res = fn(res) end
+ return res
+ else
+ line = getter() -- token not found; try next line
+ pos = 1 -- restart from first position
+ end
+ end
+ return nil -- no more lines: end of traversal
+ end
+end
+local alltokens = input.alltokens
+
+-- question: shd this _split_ a string containing line feeds?
+
+--- create a function which grabs the next value from a source. If the source is a string, then the getter
+-- will return the string and thereafter return nil. If not specified then the source is assumed to be stdin.
+-- @param f a string or a file-like object (i.e. has a read() method which returns the next line)
+-- @return a getter function
+function input.create_getter(f)
+ if f then
+ if type(f) == 'string' then
+ local ls = utils.split(f,'\n')
+ local i,n = 0,#ls
+ return function()
+ i = i + 1
+ if i > n then return nil end
+ return ls[i]
+ end
+ else
+ -- anything that supports the read() method!
+ if not f.read then error('not a file-like object') end
+ return function() return f:read() end
+ end
+ else
+ return io.read -- i.e. just read from stdin
+ end
+end
+
+--- generate a sequence of numbers from a source.
+-- @param f A source
+-- @return An iterator
+function input.numbers(f)
+ return alltokens(input.create_getter(f),
+ '('..patterns.FLOAT..')',tonumber)
+end
+
+--- generate a sequence of words from a source.
+-- @param f A source
+-- @return An iterator
+function input.words(f)
+ return alltokens(input.create_getter(f),"%w+")
+end
+
+local function apply_tonumber (no_fail,...)
+ local args = {...}
+ for i = 1,#args do
+ local n = tonumber(args[i])
+ if n == nil then
+ if not no_fail then return nil,args[i] end
+ else
+ args[i] = n
+ end
+ end
+ return args
+end
+
+--- parse an input source into fields.
+-- By default, will fail if it cannot convert a field to a number.
+-- @param ids a list of field indices, or a maximum field index
+-- @param delim delimiter to parse fields (default space)
+-- @param f a source @see create_getter
+-- @param opts option table, {no_fail=true}
+-- @return an iterator with the field values
+-- @usage for x,y in fields {2,3} do print(x,y) end -- 2nd and 3rd fields from stdin
+function input.fields (ids,delim,f,opts)
+ local sep
+ local s
+ local getter = input.create_getter(f)
+ local no_fail = opts and opts.no_fail
+ local no_convert = opts and opts.no_convert
+ if not delim or delim == ' ' then
+ delim = '%s'
+ sep = '%s+'
+ s = '%s*'
+ else
+ sep = delim
+ s = ''
+ end
+ local max_id = 0
+ if type(ids) == 'table' then
+ for i,id in pairs(ids) do
+ if id > max_id then max_id = id end
+ end
+ else
+ max_id = ids
+ ids = {}
+ for i = 1,max_id do ids[#ids+1] = i end
+ end
+ local pat = '[^'..delim..']*'
+ local k = 1
+ for i = 1,max_id do
+ if ids[k] == i then
+ k = k + 1
+ s = s..'('..pat..')'
+ else
+ s = s..pat
+ end
+ if i < max_id then
+ s = s..sep
+ end
+ end
+ local linecount = 1
+ return function()
+ local line,results,err
+ repeat
+ line = getter()
+ linecount = linecount + 1
+ if not line then return nil end
+ if no_convert then
+ results = {strmatch(line,s)}
+ else
+ results,err = apply_tonumber(no_fail,strmatch(line,s))
+ if not results then
+ utils.quit("line "..(linecount-1)..": cannot convert '"..err.."' to number")
+ end
+ end
+ until #results > 0
+ return unpack(results)
+ end
+end
+
+return input
+
--- /dev/null
+--- Simple command-line parsing using human-readable specification.
+-- Supports GNU-style parameters.
+--
+-- lapp = require 'pl.lapp'
+-- local args = lapp [[
+-- Does some calculations
+-- -o,--offset (default 0.0) Offset to add to scaled number
+-- -s,--scale (number) Scaling factor
+-- <number> (number ) Number to be scaled
+-- ]]
+--
+-- print(args.offset + args.scale * args.number)
+--
+-- Lines begining with '-' are flags; there may be a short and a long name;
+-- lines begining wih '<var>' are arguments. Anything in parens after
+-- the flag/argument is either a default, a type name or a range constraint.
+--
+-- >See @{08-additional.md.Command_line_Programs_with_Lapp|the Guide}
+--
+-- Dependencies: `pl.sip`
+-- @module pl.lapp
+
+local status,sip = pcall(require,'pl.sip')
+if not status then
+ sip = require 'sip'
+end
+local match = sip.match_at_start
+local append,tinsert = table.insert,table.insert
+
+sip.custom_pattern('X','(%a[%w_%-]*)')
+
+local function lines(s) return s:gmatch('([^\n]*)\n') end
+local function lstrip(str) return str:gsub('^%s+','') end
+local function strip(str) return lstrip(str):gsub('%s+$','') end
+local function at(s,k) return s:sub(k,k) end
+local function isdigit(s) return s:find('^%d+$') == 1 end
+
+local lapp = {}
+
+local open_files,parms,aliases,parmlist,usage,windows,script
+
+lapp.callback = false -- keep Strict happy
+
+local filetypes = {
+ stdin = {io.stdin,'file-in'}, stdout = {io.stdout,'file-out'},
+ stderr = {io.stderr,'file-out'}
+}
+
+--- controls whether to dump usage on error.
+-- Defaults to true
+lapp.show_usage_error = true
+
+--- quit this script immediately.
+-- @param msg optional message
+-- @param no_usage suppress 'usage' display
+function lapp.quit(msg,no_usage)
+ if no_usage == 'throw' then
+ error(msg)
+ end
+ if msg then
+ io.stderr:write(msg..'\n\n')
+ end
+ if not no_usage then
+ io.stderr:write(usage)
+ end
+ os.exit(1)
+end
+
+--- print an error to stderr and quit.
+-- @param msg a message
+-- @param no_usage suppress 'usage' display
+function lapp.error(msg,no_usage)
+ if not lapp.show_usage_error then
+ no_usage = true
+ elseif lapp.show_usage_error == 'throw' then
+ no_usage = 'throw'
+ end
+ lapp.quit(script..': '..msg,no_usage)
+end
+
+--- open a file.
+-- This will quit on error, and keep a list of file objects for later cleanup.
+-- @param file filename
+-- @param opt same as second parameter of <code>io.open</code>
+function lapp.open (file,opt)
+ local val,err = io.open(file,opt)
+ if not val then lapp.error(err,true) end
+ append(open_files,val)
+ return val
+end
+
+--- quit if the condition is false.
+-- @param condn a condition
+-- @param msg an optional message
+function lapp.assert(condn,msg)
+ if not condn then
+ lapp.error(msg)
+ end
+end
+
+local function range_check(x,min,max,parm)
+ lapp.assert(min <= x and max >= x,parm..' out of range')
+end
+
+local function xtonumber(s)
+ local val = tonumber(s)
+ if not val then lapp.error("unable to convert to number: "..s) end
+ return val
+end
+
+local types
+
+local builtin_types = {string=true,number=true,['file-in']='file',['file-out']='file',boolean=true}
+
+local function convert_parameter(ps,val)
+ if ps.converter then
+ val = ps.converter(val)
+ end
+ if ps.type == 'number' then
+ val = xtonumber(val)
+ elseif builtin_types[ps.type] == 'file' then
+ val = lapp.open(val,(ps.type == 'file-in' and 'r') or 'w' )
+ elseif ps.type == 'boolean' then
+ val = true
+ end
+ if ps.constraint then
+ ps.constraint(val)
+ end
+ return val
+end
+
+--- add a new type to Lapp. These appear in parens after the value like
+-- a range constraint, e.g. '<ival> (integer) Process PID'
+-- @param name name of type
+-- @param converter either a function to convert values, or a Lua type name.
+-- @param constraint optional function to verify values, should use lapp.error
+-- if failed.
+function lapp.add_type (name,converter,constraint)
+ types[name] = {converter=converter,constraint=constraint}
+end
+
+local function force_short(short)
+ lapp.assert(#short==1,short..": short parameters should be one character")
+end
+
+local function process_default (sval,vtype)
+ local val
+ if not vtype or vtype == 'number' then
+ val = tonumber(sval)
+ end
+ if val then -- we have a number!
+ return val,'number'
+ elseif filetypes[sval] then
+ local ft = filetypes[sval]
+ return ft[1],ft[2]
+ else
+ if sval:match '^["\']' then sval = sval:sub(2,-2) end
+ return sval,vtype or 'string'
+ end
+end
+
+--- process a Lapp options string.
+-- Usually called as lapp().
+-- @param str the options text
+-- @param args a table of arguments (default is `_G.arg`)
+-- @return a table with parameter-value pairs
+function lapp.process_options_string(str,args)
+ local results = {}
+ local opts = {at_start=true}
+ local varargs
+ local arg = args or _G.arg
+ open_files = {}
+ parms = {}
+ aliases = {}
+ parmlist = {}
+ types = {}
+
+ local function check_varargs(s)
+ local res,cnt = s:gsub('^%.%.%.%s*','')
+ return res, (cnt > 0)
+ end
+
+ local function set_result(ps,parm,val)
+ if not ps.varargs then
+ results[parm] = val
+ else
+ if not results[parm] then
+ results[parm] = { val }
+ else
+ append(results[parm],val)
+ end
+ end
+ end
+
+ usage = str
+
+ for line in lines(str) do
+ local res = {}
+ local optspec,optparm,i1,i2,defval,vtype,constraint,rest
+ line = lstrip(line)
+ local function check(str)
+ return match(str,line,res)
+ end
+
+ -- flags: either '-<short>', '-<short>,--<long>' or '--<long>'
+ if check '-$v{short}, --$v{long} $' or check '-$v{short} $' or check '--$X{long} $' then
+ if res.long then
+ optparm = res.long:gsub('%W','_') -- so foo-bar becomes foo_bar in Lua
+ if res.short then aliases[res.short] = optparm end
+ else
+ optparm = res.short
+ end
+ if res.short then force_short(res.short) end
+ res.rest, varargs = check_varargs(res.rest)
+ elseif check '$<{name} $' then -- is it <parameter_name>?
+ -- so <input file...> becomes input_file ...
+ optparm,rest = res.name:match '([^%.]+)(.*)'
+ optparm = optparm:gsub('%A','_')
+ varargs = rest == '...'
+ append(parmlist,optparm)
+ end
+ -- this is not a pure doc line and specifies the flag/parameter type
+ if res.rest then
+ line = res.rest
+ res = {}
+ -- do we have ([<type>] [default <val>])?
+ if match('$({def} $',line,res) or match('$({def}',line,res) then
+ local typespec = strip(res.def)
+ local ftype, rest = typespec:match('^(%S+)(.*)$')
+ rest = strip(rest)
+ local default
+ if ftype == 'default' then
+ default = true
+ if rest == '' then lapp.error("value must follow default") end
+ else -- a type specification
+ if match('$f{min}..$f{max}',ftype,res) then
+ -- a numerical range like 1..10
+ local min,max = res.min,res.max
+ vtype = 'number'
+ constraint = function(x)
+ range_check(x,min,max,optparm)
+ end
+ elseif not ftype:match '|' then -- plain type
+ vtype = ftype
+ else
+ -- 'enum' type is a string which must belong to
+ -- one of several distinct values
+ local enums = ftype
+ local enump = '|' .. enums .. '|'
+ vtype = 'string'
+ constraint = function(s)
+ lapp.assert(enump:match('|'..s..'|'),
+ "value '"..s.."' not in "..enums
+ )
+ end
+ end
+ end
+ res.rest = rest
+ typespec = res.rest
+ -- optional 'default value' clause. Type is inferred as
+ -- 'string' or 'number' if there's no explicit type
+ if default or match('default $r{rest}',typespec,res) then
+ defval,vtype = process_default(res.rest,vtype)
+ end
+ --print('val',optparm,defval,vtype)
+ else -- must be a plain flag, no extra parameter required
+ defval = false
+ vtype = 'boolean'
+ end
+ local ps = {
+ type = vtype,
+ defval = defval,
+ required = defval == nil,
+ comment = res.rest or optparm,
+ constraint = constraint,
+ varargs = varargs
+ }
+ varargs = nil
+ if types[vtype] then
+ local converter = types[vtype].converter
+ if type(converter) == 'string' then
+ ps.type = converter
+ else
+ ps.converter = converter
+ end
+ ps.constraint = types[vtype].constraint
+ elseif not builtin_types[vtype] then
+ lapp.error(vtype.." is unknown type")
+ end
+ parms[optparm] = ps
+ end
+ end
+ -- cool, we have our parms, let's parse the command line args
+ local iparm = 1
+ local iextra = 1
+ local i = 1
+ local parm,ps,val
+
+ local function check_parm (parm)
+ local eqi = parm:find '='
+ if eqi then
+ tinsert(arg,i+1,parm:sub(eqi+1))
+ parm = parm:sub(1,eqi-1)
+ end
+ return parm,eqi
+ end
+
+ while i <= #arg do
+ local theArg = arg[i]
+ local res = {}
+ -- look for a flag, -<short flags> or --<long flag>
+ if match('--$S{long}',theArg,res) or match('-$S{short}',theArg,res) then
+ if res.long then -- long option
+ parm = check_parm(res.long)
+ elseif #res.short == 1 then
+ parm = res.short
+ else
+ local parmstr,eq = check_parm(res.short)
+ if not eq then
+ parm = at(parmstr,1)
+ if isdigit(at(parmstr,2)) then
+ -- a short option followed by a digit is an exception (for AW;))
+ -- push ahead into the arg array
+ tinsert(arg,i+1,parmstr:sub(2))
+ else
+ -- push multiple flags into the arg array!
+ for k = 2,#parmstr do
+ tinsert(arg,i+k-1,'-'..at(parmstr,k))
+ end
+ end
+ else
+ parm = parmstr
+ end
+ end
+ if parm == 'h' or parm == 'help' then
+ lapp.quit()
+ end
+ if aliases[parm] then parm = aliases[parm] end
+ else -- a parameter
+ parm = parmlist[iparm]
+ if not parm then
+ -- extra unnamed parameters are indexed starting at 1
+ parm = iextra
+ ps = { type = 'string' }
+ parms[parm] = ps
+ iextra = iextra + 1
+ else
+ ps = parms[parm]
+ end
+ if not ps.varargs then
+ iparm = iparm + 1
+ end
+ val = theArg
+ end
+ ps = parms[parm]
+ if not ps then lapp.error("unrecognized parameter: "..parm) end
+ if ps.type ~= 'boolean' then -- we need a value! This should follow
+ if not val then
+ i = i + 1
+ val = arg[i]
+ end
+ lapp.assert(val,parm.." was expecting a value")
+ end
+ ps.used = true
+ val = convert_parameter(ps,val)
+ set_result(ps,parm,val)
+ if builtin_types[ps.type] == 'file' then
+ set_result(ps,parm..'_name',theArg)
+ end
+ if lapp.callback then
+ lapp.callback(parm,theArg,res)
+ end
+ i = i + 1
+ val = nil
+ end
+ -- check unused parms, set defaults and check if any required parameters were missed
+ for parm,ps in pairs(parms) do
+ if not ps.used then
+ if ps.required then lapp.error("missing required parameter: "..parm) end
+ set_result(ps,parm,ps.defval)
+ end
+ end
+ return results
+end
+
+if arg then
+ script = arg[0]:gsub('.+[\\/]',''):gsub('%.%a+$','')
+else
+ script = "inter"
+end
+
+
+setmetatable(lapp, {
+ __call = function(tbl,str,args) return lapp.process_options_string(str,args) end,
+})
+
+
+return lapp
+
+
--- /dev/null
+--- Lexical scanner for creating a sequence of tokens from text.
+-- `lexer.scan(s)` returns an iterator over all tokens found in the
+-- string `s`. This iterator returns two values, a token type string
+-- (such as 'string' for quoted string, 'iden' for identifier) and the value of the
+-- token.
+--
+-- Versions specialized for Lua and C are available; these also handle block comments
+-- and classify keywords as 'keyword' tokens. For example:
+--
+-- > s = 'for i=1,n do'
+-- > for t,v in lexer.lua(s) do print(t,v) end
+-- keyword for
+-- iden i
+-- = =
+-- number 1
+-- , ,
+-- iden n
+-- keyword do
+--
+-- See the Guide for further @{06-data.md.Lexical_Scanning|discussion}
+-- @module pl.lexer
+
+local yield,wrap = coroutine.yield,coroutine.wrap
+local strfind = string.find
+local strsub = string.sub
+local append = table.insert
+
+local function assert_arg(idx,val,tp)
+ if type(val) ~= tp then
+ error("argument "..idx.." must be "..tp, 2)
+ end
+end
+
+local lexer = {}
+
+local NUMBER1 = '^[%+%-]?%d+%.?%d*[eE][%+%-]?%d+'
+local NUMBER2 = '^[%+%-]?%d+%.?%d*'
+local NUMBER3 = '^0x[%da-fA-F]+'
+local NUMBER4 = '^%d+%.?%d*[eE][%+%-]?%d+'
+local NUMBER5 = '^%d+%.?%d*'
+local IDEN = '^[%a_][%w_]*'
+local WSPACE = '^%s+'
+local STRING0 = [[^(['\"]).-\\%1]]
+local STRING1 = [[^(['\"]).-[^\]%1]]
+local STRING3 = "^((['\"])%2)" -- empty string
+local PREPRO = '^#.-[^\\]\n'
+
+local plain_matches,lua_matches,cpp_matches,lua_keyword,cpp_keyword
+
+local function tdump(tok)
+ return yield(tok,tok)
+end
+
+local function ndump(tok,options)
+ if options and options.number then
+ tok = tonumber(tok)
+ end
+ return yield("number",tok)
+end
+
+-- regular strings, single or double quotes; usually we want them
+-- without the quotes
+local function sdump(tok,options)
+ if options and options.string then
+ tok = tok:sub(2,-2)
+ end
+ return yield("string",tok)
+end
+
+-- long Lua strings need extra work to get rid of the quotes
+local function sdump_l(tok,options)
+ if options and options.string then
+ tok = tok:sub(3,-3)
+ end
+ return yield("string",tok)
+end
+
+local function chdump(tok,options)
+ if options and options.string then
+ tok = tok:sub(2,-2)
+ end
+ return yield("char",tok)
+end
+
+local function cdump(tok)
+ return yield('comment',tok)
+end
+
+local function wsdump (tok)
+ return yield("space",tok)
+end
+
+local function pdump (tok)
+ return yield('prepro',tok)
+end
+
+local function plain_vdump(tok)
+ return yield("iden",tok)
+end
+
+local function lua_vdump(tok)
+ if lua_keyword[tok] then
+ return yield("keyword",tok)
+ else
+ return yield("iden",tok)
+ end
+end
+
+local function cpp_vdump(tok)
+ if cpp_keyword[tok] then
+ return yield("keyword",tok)
+ else
+ return yield("iden",tok)
+ end
+end
+
+--- create a plain token iterator from a string or file-like object.
+-- @param s the string
+-- @param matches an optional match table (set of pattern-action pairs)
+-- @param filter a table of token types to exclude, by default {space=true}
+-- @param options a table of options; by default, {number=true,string=true},
+-- which means convert numbers and strip string quotes.
+function lexer.scan (s,matches,filter,options)
+ --assert_arg(1,s,'string')
+ local file = type(s) ~= 'string' and s
+ filter = filter or {space=true}
+ options = options or {number=true,string=true}
+ if filter then
+ if filter.space then filter[wsdump] = true end
+ if filter.comments then
+ filter[cdump] = true
+ end
+ end
+ if not matches then
+ if not plain_matches then
+ plain_matches = {
+ {WSPACE,wsdump},
+ {NUMBER3,ndump},
+ {IDEN,plain_vdump},
+ {NUMBER1,ndump},
+ {NUMBER2,ndump},
+ {STRING3,sdump},
+ {STRING0,sdump},
+ {STRING1,sdump},
+ {'^.',tdump}
+ }
+ end
+ matches = plain_matches
+ end
+ local function lex ()
+ local i1,i2,idx,res1,res2,tok,pat,fun,capt
+ local line = 1
+ if file then s = file:read()..'\n' end
+ local sz = #s
+ local idx = 1
+ --print('sz',sz)
+ while true do
+ for _,m in ipairs(matches) do
+ pat = m[1]
+ fun = m[2]
+ i1,i2 = strfind(s,pat,idx)
+ if i1 then
+ tok = strsub(s,i1,i2)
+ idx = i2 + 1
+ if not (filter and filter[fun]) then
+ lexer.finished = idx > sz
+ res1,res2 = fun(tok,options)
+ end
+ if res1 then
+ local tp = type(res1)
+ -- insert a token list
+ if tp=='table' then
+ yield('','')
+ for _,t in ipairs(res1) do
+ yield(t[1],t[2])
+ end
+ elseif tp == 'string' then -- or search up to some special pattern
+ i1,i2 = strfind(s,res1,idx)
+ if i1 then
+ tok = strsub(s,i1,i2)
+ idx = i2 + 1
+ yield('',tok)
+ else
+ yield('','')
+ idx = sz + 1
+ end
+ --if idx > sz then return end
+ else
+ yield(line,idx)
+ end
+ end
+ if idx > sz then
+ if file then
+ --repeat -- next non-empty line
+ line = line + 1
+ s = file:read()
+ if not s then return end
+ --until not s:match '^%s*$'
+ s = s .. '\n'
+ idx ,sz = 1,#s
+ break
+ else
+ return
+ end
+ else break end
+ end
+ end
+ end
+ end
+ return wrap(lex)
+end
+
+local function isstring (s)
+ return type(s) == 'string'
+end
+
+--- insert tokens into a stream.
+-- @param tok a token stream
+-- @param a1 a string is the type, a table is a token list and
+-- a function is assumed to be a token-like iterator (returns type & value)
+-- @param a2 a string is the value
+function lexer.insert (tok,a1,a2)
+ if not a1 then return end
+ local ts
+ if isstring(a1) and isstring(a2) then
+ ts = {{a1,a2}}
+ elseif type(a1) == 'function' then
+ ts = {}
+ for t,v in a1() do
+ append(ts,{t,v})
+ end
+ else
+ ts = a1
+ end
+ tok(ts)
+end
+
+--- get everything in a stream upto a newline.
+-- @param tok a token stream
+-- @return a string
+function lexer.getline (tok)
+ local t,v = tok('.-\n')
+ return v
+end
+
+--- get current line number. <br>
+-- Only available if the input source is a file-like object.
+-- @param tok a token stream
+-- @return the line number and current column
+function lexer.lineno (tok)
+ return tok(0)
+end
+
+--- get the rest of the stream.
+-- @param tok a token stream
+-- @return a string
+function lexer.getrest (tok)
+ local t,v = tok('.+')
+ return v
+end
+
+--- get the Lua keywords as a set-like table.
+-- So <code>res["and"]</code> etc would be <code>true</code>.
+-- @return a table
+function lexer.get_keywords ()
+ if not lua_keyword then
+ lua_keyword = {
+ ["and"] = true, ["break"] = true, ["do"] = true,
+ ["else"] = true, ["elseif"] = true, ["end"] = true,
+ ["false"] = true, ["for"] = true, ["function"] = true,
+ ["if"] = true, ["in"] = true, ["local"] = true, ["nil"] = true,
+ ["not"] = true, ["or"] = true, ["repeat"] = true,
+ ["return"] = true, ["then"] = true, ["true"] = true,
+ ["until"] = true, ["while"] = true
+ }
+ end
+ return lua_keyword
+end
+
+
+--- create a Lua token iterator from a string or file-like object.
+-- Will return the token type and value.
+-- @param s the string
+-- @param filter a table of token types to exclude, by default {space=true,comments=true}
+-- @param options a table of options; by default, {number=true,string=true},
+-- which means convert numbers and strip string quotes.
+function lexer.lua(s,filter,options)
+ filter = filter or {space=true,comments=true}
+ lexer.get_keywords()
+ if not lua_matches then
+ lua_matches = {
+ {WSPACE,wsdump},
+ {NUMBER3,ndump},
+ {IDEN,lua_vdump},
+ {NUMBER4,ndump},
+ {NUMBER5,ndump},
+ {STRING3,sdump},
+ {STRING0,sdump},
+ {STRING1,sdump},
+ {'^%-%-%[%[.-%]%]',cdump},
+ {'^%-%-.-\n',cdump},
+ {'^%[%[.-%]%]',sdump_l},
+ {'^==',tdump},
+ {'^~=',tdump},
+ {'^<=',tdump},
+ {'^>=',tdump},
+ {'^%.%.%.',tdump},
+ {'^%.%.',tdump},
+ {'^.',tdump}
+ }
+ end
+ return lexer.scan(s,lua_matches,filter,options)
+end
+
+--- create a C/C++ token iterator from a string or file-like object.
+-- Will return the token type type and value.
+-- @param s the string
+-- @param filter a table of token types to exclude, by default {space=true,comments=true}
+-- @param options a table of options; by default, {number=true,string=true},
+-- which means convert numbers and strip string quotes.
+function lexer.cpp(s,filter,options)
+ filter = filter or {comments=true}
+ if not cpp_keyword then
+ cpp_keyword = {
+ ["class"] = true, ["break"] = true, ["do"] = true, ["sizeof"] = true,
+ ["else"] = true, ["continue"] = true, ["struct"] = true,
+ ["false"] = true, ["for"] = true, ["public"] = true, ["void"] = true,
+ ["private"] = true, ["protected"] = true, ["goto"] = true,
+ ["if"] = true, ["static"] = true, ["const"] = true, ["typedef"] = true,
+ ["enum"] = true, ["char"] = true, ["int"] = true, ["bool"] = true,
+ ["long"] = true, ["float"] = true, ["true"] = true, ["delete"] = true,
+ ["double"] = true, ["while"] = true, ["new"] = true,
+ ["namespace"] = true, ["try"] = true, ["catch"] = true,
+ ["switch"] = true, ["case"] = true, ["extern"] = true,
+ ["return"] = true,["default"] = true,['unsigned'] = true,['signed'] = true,
+ ["union"] = true, ["volatile"] = true, ["register"] = true,["short"] = true,
+ }
+ end
+ if not cpp_matches then
+ cpp_matches = {
+ {WSPACE,wsdump},
+ {PREPRO,pdump},
+ {NUMBER3,ndump},
+ {IDEN,cpp_vdump},
+ {NUMBER4,ndump},
+ {NUMBER5,ndump},
+ {STRING3,sdump},
+ {STRING1,chdump},
+ {'^//.-\n',cdump},
+ {'^/%*.-%*/',cdump},
+ {'^==',tdump},
+ {'^!=',tdump},
+ {'^<=',tdump},
+ {'^>=',tdump},
+ {'^->',tdump},
+ {'^&&',tdump},
+ {'^||',tdump},
+ {'^%+%+',tdump},
+ {'^%-%-',tdump},
+ {'^%+=',tdump},
+ {'^%-=',tdump},
+ {'^%*=',tdump},
+ {'^/=',tdump},
+ {'^|=',tdump},
+ {'^%^=',tdump},
+ {'^::',tdump},
+ {'^.',tdump}
+ }
+ end
+ return lexer.scan(s,cpp_matches,filter,options)
+end
+
+--- get a list of parameters separated by a delimiter from a stream.
+-- @param tok the token stream
+-- @param endtoken end of list (default ')'). Can be '\n'
+-- @param delim separator (default ',')
+-- @return a list of token lists.
+function lexer.get_separated_list(tok,endtoken,delim)
+ endtoken = endtoken or ')'
+ delim = delim or ','
+ local parm_values = {}
+ local level = 1 -- used to count ( and )
+ local tl = {}
+ local function tappend (tl,t,val)
+ val = val or t
+ append(tl,{t,val})
+ end
+ local is_end
+ if endtoken == '\n' then
+ is_end = function(t,val)
+ return t == 'space' and val:find '\n'
+ end
+ else
+ is_end = function (t)
+ return t == endtoken
+ end
+ end
+ local token,value
+ while true do
+ token,value=tok()
+ if not token then return nil,'EOS' end -- end of stream is an error!
+ if is_end(token,value) and level == 1 then
+ append(parm_values,tl)
+ break
+ elseif token == '(' then
+ level = level + 1
+ tappend(tl,'(')
+ elseif token == ')' then
+ level = level - 1
+ if level == 0 then -- finished with parm list
+ append(parm_values,tl)
+ break
+ else
+ tappend(tl,')')
+ end
+ elseif token == delim and level == 1 then
+ append(parm_values,tl) -- a new parm
+ tl = {}
+ else
+ tappend(tl,token,value)
+ end
+ end
+ return parm_values,{token,value}
+end
+
+--- get the next non-space token from the stream.
+-- @param tok the token stream.
+function lexer.skipws (tok)
+ local t,v = tok()
+ while t == 'space' do
+ t,v = tok()
+ end
+ return t,v
+end
+
+local skipws = lexer.skipws
+
+--- get the next token, which must be of the expected type.
+-- Throws an error if this type does not match!
+-- @param tok the token stream
+-- @param expected_type the token type
+-- @param no_skip_ws whether we should skip whitespace
+function lexer.expecting (tok,expected_type,no_skip_ws)
+ assert_arg(1,tok,'function')
+ assert_arg(2,expected_type,'string')
+ local t,v
+ if no_skip_ws then
+ t,v = tok()
+ else
+ t,v = skipws(tok)
+ end
+ if t ~= expected_type then error ("expecting "..expected_type,2) end
+ return v
+end
+
+return lexer
--- /dev/null
+--- Extract delimited Lua sequences from strings.
+-- Inspired by Damian Conway's Text::Balanced in Perl. <br/>
+-- <ul>
+-- <li>[1] <a href="http://lua-users.org/wiki/LuaBalanced">Lua Wiki Page</a></li>
+-- <li>[2] http://search.cpan.org/dist/Text-Balanced/lib/Text/Balanced.pm</li>
+-- </ul> <br/>
+-- <pre class=example>
+-- local lb = require "pl.luabalanced"
+-- --Extract Lua expression starting at position 4.
+-- print(lb.match_expression("if x^2 + x > 5 then print(x) end", 4))
+-- --> x^2 + x > 5 16
+-- --Extract Lua string starting at (default) position 1.
+-- print(lb.match_string([["test\"123" .. "more"]]))
+-- --> "test\"123" 12
+-- </pre>
+-- (c) 2008, David Manura, Licensed under the same terms as Lua (MIT license).
+-- @class module
+-- @name pl.luabalanced
+
+local M = {}
+
+local assert = assert
+local table_concat = table.concat
+
+-- map opening brace <-> closing brace.
+local ends = { ['('] = ')', ['{'] = '}', ['['] = ']' }
+local begins = {}; for k,v in pairs(ends) do begins[v] = k end
+
+
+-- Match Lua string in string <s> starting at position <pos>.
+-- Returns <string>, <posnew>, where <string> is the matched
+-- string (or nil on no match) and <posnew> is the character
+-- following the match (or <pos> on no match).
+-- Supports all Lua string syntax: "...", '...', [[...]], [=[...]=], etc.
+local function match_string(s, pos)
+ pos = pos or 1
+ local posa = pos
+ local c = s:sub(pos,pos)
+ if c == '"' or c == "'" then
+ pos = pos + 1
+ while 1 do
+ pos = assert(s:find("[" .. c .. "\\]", pos), 'syntax error')
+ if s:sub(pos,pos) == c then
+ local part = s:sub(posa, pos)
+ return part, pos + 1
+ else
+ pos = pos + 2
+ end
+ end
+ else
+ local sc = s:match("^%[(=*)%[", pos)
+ if sc then
+ local _; _, pos = s:find("%]" .. sc .. "%]", pos)
+ assert(pos)
+ local part = s:sub(posa, pos)
+ return part, pos + 1
+ else
+ return nil, pos
+ end
+ end
+end
+M.match_string = match_string
+
+
+-- Match bracketed Lua expression, e.g. "(...)", "{...}", "[...]", "[[...]]",
+-- [=[...]=], etc.
+-- Function interface is similar to match_string.
+local function match_bracketed(s, pos)
+ pos = pos or 1
+ local posa = pos
+ local ca = s:sub(pos,pos)
+ if not ends[ca] then
+ return nil, pos
+ end
+ local stack = {}
+ while 1 do
+ pos = s:find('[%(%{%[%)%}%]\"\']', pos)
+ assert(pos, 'syntax error: unbalanced')
+ local c = s:sub(pos,pos)
+ if c == '"' or c == "'" then
+ local part; part, pos = match_string(s, pos)
+ assert(part)
+ elseif ends[c] then -- open
+ local mid, posb
+ if c == '[' then mid, posb = s:match('^%[(=*)%[()', pos) end
+ if mid then
+ pos = s:match('%]' .. mid .. '%]()', posb)
+ assert(pos, 'syntax error: long string not terminated')
+ if #stack == 0 then
+ local part = s:sub(posa, pos-1)
+ return part, pos
+ end
+ else
+ stack[#stack+1] = c
+ pos = pos + 1
+ end
+ else -- close
+ assert(stack[#stack] == assert(begins[c]), 'syntax error: unbalanced')
+ stack[#stack] = nil
+ if #stack == 0 then
+ local part = s:sub(posa, pos)
+ return part, pos+1
+ end
+ pos = pos + 1
+ end
+ end
+end
+M.match_bracketed = match_bracketed
+
+
+-- Match Lua comment, e.g. "--...\n", "--[[...]]", "--[=[...]=]", etc.
+-- Function interface is similar to match_string.
+local function match_comment(s, pos)
+ pos = pos or 1
+ if s:sub(pos, pos+1) ~= '--' then
+ return nil, pos
+ end
+ pos = pos + 2
+ local partt, post = match_string(s, pos)
+ if partt then
+ return '--' .. partt, post
+ end
+ local part; part, pos = s:match('^([^\n]*\n?)()', pos)
+ return '--' .. part, pos
+end
+
+
+-- Match Lua expression, e.g. "a + b * c[e]".
+-- Function interface is similar to match_string.
+local wordop = {['and']=true, ['or']=true, ['not']=true}
+local is_compare = {['>']=true, ['<']=true, ['~']=true}
+local function match_expression(s, pos)
+ pos = pos or 1
+ local posa = pos
+ local lastident
+ local poscs, posce
+ while pos do
+ local c = s:sub(pos,pos)
+ if c == '"' or c == "'" or c == '[' and s:find('^[=%[]', pos+1) then
+ local part; part, pos = match_string(s, pos)
+ assert(part, 'syntax error')
+ elseif c == '-' and s:sub(pos+1,pos+1) == '-' then
+ -- note: handle adjacent comments in loop to properly support
+ -- backtracing (poscs/posce).
+ poscs = pos
+ while s:sub(pos,pos+1) == '--' do
+ local part; part, pos = match_comment(s, pos)
+ assert(part)
+ pos = s:match('^%s*()', pos)
+ posce = pos
+ end
+ elseif c == '(' or c == '{' or c == '[' then
+ local part; part, pos = match_bracketed(s, pos)
+ elseif c == '=' and s:sub(pos+1,pos+1) == '=' then
+ pos = pos + 2 -- skip over two-char op containing '='
+ elseif c == '=' and is_compare[s:sub(pos-1,pos-1)] then
+ pos = pos + 1 -- skip over two-char op containing '='
+ elseif c:match'^[%)%}%];,=]' then
+ local part = s:sub(posa, pos-1)
+ return part, pos
+ elseif c:match'^[%w_]' then
+ local newident,newpos = s:match('^([%w_]+)()', pos)
+ if pos ~= posa and not wordop[newident] then -- non-first ident
+ local pose = ((posce == pos) and poscs or pos) - 1
+ while s:match('^%s', pose) do pose = pose - 1 end
+ local ce = s:sub(pose,pose)
+ if ce:match'[%)%}\'\"%]]' or
+ ce:match'[%w_]' and not wordop[lastident]
+ then
+ local part = s:sub(posa, pos-1)
+ return part, pos
+ end
+ end
+ lastident, pos = newident, newpos
+ else
+ pos = pos + 1
+ end
+ pos = s:find('[%(%{%[%)%}%]\"\';,=%w_%-]', pos)
+ end
+ local part = s:sub(posa, #s)
+ return part, #s+1
+end
+M.match_expression = match_expression
+
+
+-- Match name list (zero or more names). E.g. "a,b,c"
+-- Function interface is similar to match_string,
+-- but returns array as match.
+local function match_namelist(s, pos)
+ pos = pos or 1
+ local list = {}
+ while 1 do
+ local c = #list == 0 and '^' or '^%s*,%s*'
+ local item, post = s:match(c .. '([%a_][%w_]*)%s*()', pos)
+ if item then pos = post else break end
+ list[#list+1] = item
+ end
+ return list, pos
+end
+M.match_namelist = match_namelist
+
+
+-- Match expression list (zero or more expressions). E.g. "a+b,b*c".
+-- Function interface is similar to match_string,
+-- but returns array as match.
+local function match_explist(s, pos)
+ pos = pos or 1
+ local list = {}
+ while 1 do
+ if #list ~= 0 then
+ local post = s:match('^%s*,%s*()', pos)
+ if post then pos = post else break end
+ end
+ local item; item, pos = match_expression(s, pos)
+ assert(item, 'syntax error')
+ list[#list+1] = item
+ end
+ return list, pos
+end
+M.match_explist = match_explist
+
+
+-- Replace snippets of code in Lua code string <s>
+-- using replacement function f(u,sin) --> sout.
+-- <u> is the type of snippet ('c' = comment, 's' = string,
+-- 'e' = any other code).
+-- Snippet is replaced with <sout> (unless <sout> is nil or false, in
+-- which case the original snippet is kept)
+-- This is somewhat analogous to string.gsub .
+local function gsub(s, f)
+ local pos = 1
+ local posa = 1
+ local sret = ''
+ while 1 do
+ pos = s:find('[%-\'\"%[]', pos)
+ if not pos then break end
+ if s:match('^%-%-', pos) then
+ local exp = s:sub(posa, pos-1)
+ if #exp > 0 then sret = sret .. (f('e', exp) or exp) end
+ local comment; comment, pos = match_comment(s, pos)
+ sret = sret .. (f('c', assert(comment)) or comment)
+ posa = pos
+ else
+ local posb = s:find('^[\'\"%[]', pos)
+ local str
+ if posb then str, pos = match_string(s, posb) end
+ if str then
+ local exp = s:sub(posa, posb-1)
+ if #exp > 0 then sret = sret .. (f('e', exp) or exp) end
+ sret = sret .. (f('s', str) or str)
+ posa = pos
+ else
+ pos = pos + 1
+ end
+ end
+ end
+ local exp = s:sub(posa)
+ if #exp > 0 then sret = sret .. (f('e', exp) or exp) end
+ return sret
+end
+M.gsub = gsub
+
+
+return M
--- /dev/null
+--- Lua operators available as functions.
+--
+-- (similar to the Python module of the same name)
+--
+-- There is a module field `optable` which maps the operator strings
+-- onto these functions, e.g. `operator.optable['()']==operator.call`
+--
+-- Operator strings like '>' and '{}' can be passed to most Penlight functions
+-- expecting a function argument.
+--
+-- Dependencies: `pl.utils`
+-- @module pl.operator
+
+local strfind = string.find
+local utils = require 'pl.utils'
+
+local operator = {}
+
+--- apply function to some arguments ()
+-- @param fn a function or callable object
+-- @param ... arguments
+function operator.call(fn,...)
+ return fn(...)
+end
+
+--- get the indexed value from a table []
+-- @param t a table or any indexable object
+-- @param k the key
+function operator.index(t,k)
+ return t[k]
+end
+
+--- returns true if arguments are equal ==
+-- @param a value
+-- @param b value
+function operator.eq(a,b)
+ return a==b
+end
+
+--- returns true if arguments are not equal ~=
+ -- @param a value
+-- @param b value
+function operator.neq(a,b)
+ return a~=b
+end
+
+--- returns true if a is less than b <
+-- @param a value
+-- @param b value
+function operator.lt(a,b)
+ return a < b
+end
+
+--- returns true if a is less or equal to b <=
+-- @param a value
+-- @param b value
+function operator.le(a,b)
+ return a <= b
+end
+
+--- returns true if a is greater than b >
+-- @param a value
+-- @param b value
+function operator.gt(a,b)
+ return a > b
+end
+
+--- returns true if a is greater or equal to b >=
+-- @param a value
+-- @param b value
+function operator.ge(a,b)
+ return a >= b
+end
+
+--- returns length of string or table #
+-- @param a a string or a table
+function operator.len(a)
+ return #a
+end
+
+--- add two values +
+-- @param a value
+-- @param b value
+function operator.add(a,b)
+ return a+b
+end
+
+--- subtract b from a -
+-- @param a value
+-- @param b value
+function operator.sub(a,b)
+ return a-b
+end
+
+--- multiply two values *
+-- @param a value
+-- @param b value
+function operator.mul(a,b)
+ return a*b
+end
+
+--- divide first value by second /
+-- @param a value
+-- @param b value
+function operator.div(a,b)
+ return a/b
+end
+
+--- raise first to the power of second ^
+-- @param a value
+-- @param b value
+function operator.pow(a,b)
+ return a^b
+end
+
+--- modulo; remainder of a divided by b %
+-- @param a value
+-- @param b value
+function operator.mod(a,b)
+ return a%b
+end
+
+--- concatenate two values (either strings or __concat defined) ..
+-- @param a value
+-- @param b value
+function operator.concat(a,b)
+ return a..b
+end
+
+--- return the negative of a value -
+-- @param a value
+-- @param b value
+function operator.unm(a)
+ return -a
+end
+
+--- false if value evaluates as true not
+-- @param a value
+function operator.lnot(a)
+ return not a
+end
+
+--- true if both values evaluate as true and
+-- @param a value
+-- @param b value
+function operator.land(a,b)
+ return a and b
+end
+
+--- true if either value evaluate as true or
+-- @param a value
+-- @param b value
+function operator.lor(a,b)
+ return a or b
+end
+
+--- make a table from the arguments {}
+-- @param ... non-nil arguments
+-- @return a table
+function operator.table (...)
+ return {...}
+end
+
+--- match two strings ~
+-- uses @{string.find}
+function operator.match (a,b)
+ return strfind(a,b)~=nil
+end
+
+--- the null operation.
+-- @param ... arguments
+-- @return the arguments
+function operator.nop (...)
+ return ...
+end
+
+ operator.optable = {
+ ['+']=operator.add,
+ ['-']=operator.sub,
+ ['*']=operator.mul,
+ ['/']=operator.div,
+ ['%']=operator.mod,
+ ['^']=operator.pow,
+ ['..']=operator.concat,
+ ['()']=operator.call,
+ ['[]']=operator.index,
+ ['<']=operator.lt,
+ ['<=']=operator.le,
+ ['>']=operator.gt,
+ ['>=']=operator.ge,
+ ['==']=operator.eq,
+ ['~=']=operator.neq,
+ ['#']=operator.len,
+ ['and']=operator.land,
+ ['or']=operator.lor,
+ ['{}']=operator.table,
+ ['~']=operator.match,
+ ['']=operator.nop,
+}
+
+return operator
--- /dev/null
+--- Path manipulation and file queries.
+--
+-- This is modelled after Python's os.path library (10.1); see @{04-paths.md|the Guide}.
+--
+-- Dependencies: `pl.utils`, `lfs`
+-- @module pl.path
+
+-- imports and locals
+local _G = _G
+local sub = string.sub
+local getenv = os.getenv
+local tmpnam = os.tmpname
+local attributes, currentdir, link_attrib
+local package = package
+local io = io
+local append = table.insert
+local ipairs = ipairs
+local utils = require 'pl.utils'
+local assert_arg,assert_string,raise = utils.assert_arg,utils.assert_string,utils.raise
+
+local attrib
+local path = {}
+
+local res,lfs = _G.pcall(_G.require,'lfs')
+if res then
+ attributes = lfs.attributes
+ currentdir = lfs.currentdir
+ link_attrib = lfs.symlinkattributes
+else
+ error("pl.path requires LuaFileSystem")
+end
+
+attrib = attributes
+path.attrib = attrib
+path.link_attrib = link_attrib
+path.dir = lfs.dir
+path.mkdir = lfs.mkdir
+path.rmdir = lfs.rmdir
+path.chdir = lfs.chdir
+
+--- is this a directory?
+-- @param P A file path
+function path.isdir(P)
+ if P:match("\\$") then
+ P = P:sub(1,-2)
+ end
+ return attrib(P,'mode') == 'directory'
+end
+
+--- is this a file?.
+-- @param P A file path
+function path.isfile(P)
+ return attrib(P,'mode') == 'file'
+end
+
+-- is this a symbolic link?
+-- @param P A file path
+function path.islink(P)
+ if link_attrib then
+ return link_attrib(P,'mode')=='link'
+ else
+ return false
+ end
+end
+
+--- return size of a file.
+-- @param P A file path
+function path.getsize(P)
+ return attrib(P,'size')
+end
+
+--- does a path exist?.
+-- @param P A file path
+-- @return the file path if it exists, nil otherwise
+function path.exists(P)
+ return attrib(P,'mode') ~= nil and P
+end
+
+--- Return the time of last access as the number of seconds since the epoch.
+-- @param P A file path
+function path.getatime(P)
+ return attrib(P,'access')
+end
+
+--- Return the time of last modification
+-- @param P A file path
+function path.getmtime(P)
+ return attrib(P,'modification')
+end
+
+---Return the system's ctime.
+-- @param P A file path
+function path.getctime(P)
+ return path.attrib(P,'change')
+end
+
+
+local function at(s,i)
+ return sub(s,i,i)
+end
+
+path.is_windows = utils.dir_separator == '\\'
+
+local other_sep
+-- !constant sep is the directory separator for this platform.
+if path.is_windows then
+ path.sep = '\\'; other_sep = '/'
+ path.dirsep = ';'
+else
+ path.sep = '/'
+ path.dirsep = ':'
+end
+local sep,dirsep = path.sep,path.dirsep
+
+--- are we running Windows?
+-- @class field
+-- @name path.is_windows
+
+--- path separator for this platform.
+-- @class field
+-- @name path.sep
+
+--- separator for PATH for this platform
+-- @class field
+-- @name path.dirsep
+
+--- given a path, return the directory part and a file part.
+-- if there's no directory part, the first value will be empty
+-- @param P A file path
+function path.splitpath(P)
+ assert_string(1,P)
+ local i = #P
+ local ch = at(P,i)
+ while i > 0 and ch ~= sep and ch ~= other_sep do
+ i = i - 1
+ ch = at(P,i)
+ end
+ if i == 0 then
+ return '',P
+ else
+ return sub(P,1,i-1), sub(P,i+1)
+ end
+end
+
+--- return an absolute path.
+-- @param P A file path
+-- @param pwd optional start path to use (default is current dir)
+function path.abspath(P,pwd)
+ assert_string(1,P)
+ local use_pwd = pwd ~= nil
+ if not use_pwd and not currentdir then return P end
+ P = P:gsub('[\\/]$','')
+ pwd = pwd or currentdir()
+ if not path.isabs(P) then
+ P = path.join(pwd,P)
+ elseif path.is_windows and not use_pwd and at(P,2) ~= ':' and at(P,2) ~= '\\' then
+ P = pwd:sub(1,2)..P -- attach current drive to path like '\\fred.txt'
+ end
+ return path.normpath(P)
+end
+
+--- given a path, return the root part and the extension part.
+-- if there's no extension part, the second value will be empty
+-- @param P A file path
+function path.splitext(P)
+ assert_string(1,P)
+ local i = #P
+ local ch = at(P,i)
+ while i > 0 and ch ~= '.' do
+ if ch == sep or ch == other_sep then
+ return P,''
+ end
+ i = i - 1
+ ch = at(P,i)
+ end
+ if i == 0 then
+ return P,''
+ else
+ return sub(P,1,i-1),sub(P,i)
+ end
+end
+
+--- return the directory part of a path
+-- @param P A file path
+function path.dirname(P)
+ assert_string(1,P)
+ local p1,p2 = path.splitpath(P)
+ return p1
+end
+
+--- return the file part of a path
+-- @param P A file path
+function path.basename(P)
+ assert_string(1,P)
+ local p1,p2 = path.splitpath(P)
+ return p2
+end
+
+--- get the extension part of a path.
+-- @param P A file path
+function path.extension(P)
+ assert_string(1,P)
+ local p1,p2 = path.splitext(P)
+ return p2
+end
+
+--- is this an absolute path?.
+-- @param P A file path
+function path.isabs(P)
+ assert_string(1,P)
+ if path.is_windows then
+ return at(P,1) == '/' or at(P,1)=='\\' or at(P,2)==':'
+ else
+ return at(P,1) == '/'
+ end
+end
+
+--- return the path resulting from combining the individual paths.
+-- if the second path is absolute, we return that path.
+-- @param p1 A file path
+-- @param p2 A file path
+-- @param ... more file paths
+function path.join(p1,p2,...)
+ if select('#',...) > 0 then
+ local p = path.join(p1,p2)
+ local args = {...}
+ for i = 1,#args do
+ p = path.join(p,args[i])
+ end
+ return p
+ end
+ assert_string(1,p1)
+ assert_string(2,p2)
+ if path.isabs(p2) then return p2 end
+ local endc = at(p1,#p1)
+ if endc ~= path.sep and endc ~= other_sep then
+ p1 = p1..path.sep
+ end
+ return p1..p2
+end
+
+--- normalize the case of a pathname. On Unix, this returns the path unchanged;
+-- for Windows, it converts the path to lowercase, and it also converts forward slashes
+-- to backward slashes.
+-- @param P A file path
+function path.normcase(P)
+ assert_string(1,P)
+ if path.is_windows then
+ return (P:lower():gsub('/','\\'))
+ else
+ return P
+ end
+end
+
+local np_gen1,np_gen2 = '[^SEP]+SEP%.%.SEP?','SEP+%.?SEP'
+local np_pat1, np_pat2
+
+--- normalize a path name.
+-- A//B, A/./B and A/foo/../B all become A/B.
+-- @param P a file path
+function path.normpath (P)
+ assert_string(1,P)
+ if path.is_windows then
+ if P:match '^\\\\' then -- UNC
+ return '\\\\'..path.normpath(P:sub(3))
+ end
+ P = P:gsub('/','\\')
+ end
+ if not np_pat1 then
+ np_pat1 = np_gen1:gsub('SEP',sep)
+ np_pat2 = np_gen2:gsub('SEP',sep)
+ end
+ local k
+ repeat -- /./ -> /
+ P,k = P:gsub(np_pat2,sep)
+ until k == 0
+ repeat -- A/../ -> (empty)
+ P,k = P:gsub(np_pat1,'')
+ until k == 0
+ if P == '' then P = '.' end
+ return P
+end
+
+local function ATS (P)
+ if at(P,#P) ~= path.sep then
+ P = P..path.sep
+ end
+ return path.normcase(P)
+end
+
+--- relative path from current directory or optional start point
+-- @param P a path
+-- @param start optional start point (default current directory)
+function path.relpath (P,start)
+ local split,normcase,min,append = utils.split, path.normcase, math.min, table.insert
+ P = normcase(path.abspath(P,start))
+ start = start or currentdir()
+ start = normcase(start)
+ local startl, Pl = split(start,sep), split(P,sep)
+ local n = min(#startl,#Pl)
+ local k = n+1 -- default value if this loop doesn't bail out!
+ for i = 1,n do
+ if startl[i] ~= Pl[i] then
+ k = i
+ break
+ end
+ end
+ local rell = {}
+ for i = 1, #startl-k+1 do rell[i] = '..' end
+ if k <= #Pl then
+ for i = k,#Pl do append(rell,Pl[i]) end
+ end
+ return table.concat(rell,sep)
+end
+
+
+--- Replace a starting '~' with the user's home directory.
+-- In windows, if HOME isn't set, then USERPROFILE is used in preference to
+-- HOMEDRIVE HOMEPATH. This is guaranteed to be writeable on all versions of Windows.
+-- @param P A file path
+function path.expanduser(P)
+ assert_string(1,P)
+ if at(P,1) == '~' then
+ local home = getenv('HOME')
+ if not home then -- has to be Windows
+ home = getenv 'USERPROFILE' or (getenv 'HOMEDRIVE' .. getenv 'HOMEPATH')
+ end
+ return home..sub(P,2)
+ else
+ return P
+ end
+end
+
+
+---Return a suitable full path to a new temporary file name.
+-- unlike os.tmpnam(), it always gives you a writeable path (uses %TMP% on Windows)
+function path.tmpname ()
+ local res = tmpnam()
+ if path.is_windows then res = getenv('TMP')..res end
+ return res
+end
+
+--- return the largest common prefix path of two paths.
+-- @param path1 a file path
+-- @param path2 a file path
+function path.common_prefix (path1,path2)
+ assert_string(1,path1)
+ assert_string(2,path2)
+ path1, path2 = path.normcase(path1), path.normcase(path2)
+ -- get them in order!
+ if #path1 > #path2 then path2,path1 = path1,path2 end
+ for i = 1,#path1 do
+ local c1 = at(path1,i)
+ if c1 ~= at(path2,i) then
+ local cp = path1:sub(1,i-1)
+ if at(path1,i-1) ~= sep then
+ cp = path.dirname(cp)
+ end
+ return cp
+ end
+ end
+ if at(path2,#path1+1) ~= sep then path1 = path.dirname(path1) end
+ return path1
+ --return ''
+end
+
+
+--- return the full path where a particular Lua module would be found.
+-- Both package.path and package.cpath is searched, so the result may
+-- either be a Lua file or a shared libarary.
+-- @param mod name of the module
+-- @return on success: path of module, lua or binary
+-- @return on error: nil,error string
+function path.package_path(mod)
+ assert_string(1,mod)
+ local res
+ mod = mod:gsub('%.',sep)
+ res = package.searchpath(mod,package.path)
+ if res then return res,true end
+ res = package.searchpath(mod,package.cpath)
+ if res then return res,false end
+ return raise 'cannot find module on path'
+end
+
+
+---- finis -----
+return path
--- /dev/null
+--- Permutation operations.
+--
+-- Dependencies: `pl.utils`, `pl.tablex`
+-- @module pl.permute
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+local copy = tablex.deepcopy
+local append = table.insert
+local coroutine = coroutine
+local resume = coroutine.resume
+local assert_arg = utils.assert_arg
+
+
+local permute = {}
+
+-- PiL, 9.3
+
+local permgen
+permgen = function (a, n, fn)
+ if n == 0 then
+ fn(a)
+ else
+ for i=1,n do
+ -- put i-th element as the last one
+ a[n], a[i] = a[i], a[n]
+
+ -- generate all permutations of the other elements
+ permgen(a, n - 1, fn)
+
+ -- restore i-th element
+ a[n], a[i] = a[i], a[n]
+
+ end
+ end
+end
+
+--- an iterator over all permutations of the elements of a list.
+-- Please note that the same list is returned each time, so do not keep references!
+-- @param a list-like table
+-- @return an iterator which provides the next permutation as a list
+function permute.iter (a)
+ assert_arg(1,a,'table')
+ local n = #a
+ local co = coroutine.create(function () permgen(a, n, coroutine.yield) end)
+ return function () -- iterator
+ local code, res = resume(co)
+ return res
+ end
+end
+
+--- construct a table containing all the permutations of a list.
+-- @param a list-like table
+-- @return a table of tables
+-- @usage permute.table {1,2,3} --> {{2,3,1},{3,2,1},{3,1,2},{1,3,2},{2,1,3},{1,2,3}}
+function permute.table (a)
+ assert_arg(1,a,'table')
+ local res = {}
+ local n = #a
+ permgen(a,n,function(t) append(res,copy(t)) end)
+ return res
+end
+
+return permute
--- /dev/null
+-- experimental support for LuaJava
+--
+local path = {}
+
+
+path.link_attrib = nil
+
+local File = luajava.bindClass("java.io.File")
+local Array = luajava.bindClass('java.lang.reflect.Array')
+
+local function file(s)
+ return luajava.new(File,s)
+end
+
+function path.dir(P)
+ local ls = file(P):list()
+ print(ls)
+ local idx,n = -1,Array:getLength(ls)
+ return function ()
+ idx = idx + 1
+ if idx == n then return nil
+ else
+ return Array:get(ls,idx)
+ end
+ end
+end
+
+function path.mkdir(P)
+ return file(P):mkdir()
+end
+
+function path.rmdir(P)
+ return file(P):delete()
+end
+
+--- is this a directory?
+-- @param P A file path
+function path.isdir(P)
+ if P:match("\\$") then
+ P = P:sub(1,-2)
+ end
+ return file(P):isDirectory()
+end
+
+--- is this a file?.
+-- @param P A file path
+function path.isfile(P)
+ return file(P):isFile()
+end
+
+-- is this a symbolic link?
+-- Direct support for symbolic links is not provided.
+-- see http://stackoverflow.com/questions/813710/java-1-6-determine-symbolic-links
+-- and the caveats therein.
+-- @param P A file path
+function path.islink(P)
+ local f = file(P)
+ local canon
+ local parent = f:getParent()
+ if not parent then
+ canon = f
+ else
+ parent = f.getParentFile():getCanonicalFile()
+ canon = luajava.new(File,parent,f:getName())
+ end
+ return canon:getCanonicalFile() ~= canon:getAbsoluteFile()
+end
+
+--- return size of a file.
+-- @param P A file path
+function path.getsize(P)
+ return file(P):length()
+end
+
+--- does a path exist?.
+-- @param P A file path
+-- @return the file path if it exists, nil otherwise
+function path.exists(P)
+ return file(P):exists() and P
+end
+
+--- Return the time of last access as the number of seconds since the epoch.
+-- @param P A file path
+function path.getatime(P)
+ return path.getmtime(P)
+end
+
+--- Return the time of last modification
+-- @param P A file path
+function path.getmtime(P)
+ -- Java time is no. of millisec since the epoch
+ return file(P):lastModified()/1000
+end
+
+---Return the system's ctime.
+-- @param P A file path
+function path.getctime(P)
+ return path.getmtime(P)
+end
+
+return path
--- /dev/null
+--- Pretty-printing Lua tables.
+-- Also provides a sandboxed Lua table reader and
+-- a function to present large numbers in human-friendly format.
+--
+-- Dependencies: `pl.utils`, `pl.lexer`
+-- @module pl.pretty
+
+local append = table.insert
+local concat = table.concat
+local utils = require 'pl.utils'
+local lexer = require 'pl.lexer'
+local assert_arg = utils.assert_arg
+
+local pretty = {}
+
+local function save_string_index ()
+ local SMT = getmetatable ''
+ if SMT then
+ SMT.old__index = SMT.__index
+ SMT.__index = nil
+ end
+ return SMT
+end
+
+local function restore_string_index (SMT)
+ if SMT then
+ SMT.__index = SMT.old__index
+ end
+end
+
+--- read a string representation of a Lua table.
+-- Uses load(), but tries to be cautious about loading arbitrary code!
+-- It is expecting a string of the form '{...}', with perhaps some whitespace
+-- before or after the curly braces. A comment may occur beforehand.
+-- An empty environment is used, and
+-- any occurance of the keyword 'function' will be considered a problem.
+-- in the given environment - the return value may be `nil`.
+-- @param s {string} string of the form '{...}', with perhaps some whitespace
+-- before or after the curly braces.
+-- @return a table
+function pretty.read(s)
+ assert_arg(1,s,'string')
+ if s:find '^%s*%-%-' then -- may start with a comment..
+ s = s:gsub('%-%-.-\n','')
+ end
+ if not s:find '^%s*%b{}%s*$' then return nil,"not a Lua table" end
+ if s:find '[^\'"%w_]function[^\'"%w_]' then
+ local tok = lexer.lua(s)
+ for t,v in tok do
+ if t == 'keyword' then
+ return nil,"cannot have functions in table definition"
+ end
+ end
+ end
+ s = 'return '..s
+ local chunk,err = utils.load(s,'tbl','t',{})
+ if not chunk then return nil,err end
+ local SMT = save_string_index()
+ local ok,ret = pcall(chunk)
+ restore_string_index(SMT)
+ if ok then return ret
+ else
+ return nil,ret
+ end
+end
+
+--- read a Lua chunk.
+-- @param s Lua code
+-- @param env optional environment
+-- @param paranoid prevent any looping constructs and disable string methods
+-- @return the environment
+function pretty.load (s, env, paranoid)
+ env = env or {}
+ if paranoid then
+ local tok = lexer.lua(s)
+ for t,v in tok do
+ if t == 'keyword'
+ and (v == 'for' or v == 'repeat' or v == 'function' or v == 'goto')
+ then
+ return nil,"looping not allowed"
+ end
+ end
+ end
+ local chunk,err = utils.load(s,'tbl','t',env)
+ if not chunk then return nil,err end
+ local SMT = paranoid and save_string_index()
+ local ok,err = pcall(chunk)
+ restore_string_index(SMT)
+ if not ok then return nil,err end
+ return env
+end
+
+local function quote_if_necessary (v)
+ if not v then return ''
+ else
+ if v:find ' ' then v = '"'..v..'"' end
+ end
+ return v
+end
+
+local keywords
+
+local function is_identifier (s)
+ return type(s) == 'string' and s:find('^[%a_][%w_]*$') and not keywords[s]
+end
+
+local function quote (s)
+ if type(s) == 'table' then
+ return pretty.write(s,'')
+ else
+ return ('%q'):format(tostring(s))
+ end
+end
+
+local function index (numkey,key)
+ if not numkey then key = quote(key) end
+ return '['..key..']'
+end
+
+
+--- Create a string representation of a Lua table.
+-- This function never fails, but may complain by returning an
+-- extra value. Normally puts out one item per line, using
+-- the provided indent; set the second parameter to '' if
+-- you want output on one line.
+-- @param tbl {table} Table to serialize to a string.
+-- @param space {string} (optional) The indent to use.
+-- Defaults to two spaces; make it the empty string for no indentation
+-- @param not_clever {bool} (optional) Use for plain output, e.g {['key']=1}.
+-- Defaults to false.
+-- @return a string
+-- @return a possible error message
+function pretty.write (tbl,space,not_clever)
+ if type(tbl) ~= 'table' then
+ local res = tostring(tbl)
+ if type(tbl) == 'string' then res = '"'..res..'"' end
+ return res, 'not a table'
+ end
+ if not keywords then
+ keywords = lexer.get_keywords()
+ end
+ local set = ' = '
+ if space == '' then set = '=' end
+ space = space or ' '
+ local lines = {}
+ local line = ''
+ local tables = {}
+
+
+ local function put(s)
+ if #s > 0 then
+ line = line..s
+ end
+ end
+
+ local function putln (s)
+ if #line > 0 then
+ line = line..s
+ append(lines,line)
+ line = ''
+ else
+ append(lines,s)
+ end
+ end
+
+ local function eat_last_comma ()
+ local n,lastch = #lines
+ local lastch = lines[n]:sub(-1,-1)
+ if lastch == ',' then
+ lines[n] = lines[n]:sub(1,-2)
+ end
+ end
+
+
+ local writeit
+ writeit = function (t,oldindent,indent)
+ local tp = type(t)
+ if tp ~= 'string' and tp ~= 'table' then
+ putln(quote_if_necessary(tostring(t))..',')
+ elseif tp == 'string' then
+ if t:find('\n') then
+ putln('[[\n'..t..']],')
+ else
+ putln(quote(t)..',')
+ end
+ elseif tp == 'table' then
+ if tables[t] then
+ putln('<cycle>,')
+ return
+ end
+ tables[t] = true
+ local newindent = indent..space
+ putln('{')
+ local used = {}
+ if not not_clever then
+ for i,val in ipairs(t) do
+ put(indent)
+ writeit(val,indent,newindent)
+ used[i] = true
+ end
+ end
+ for key,val in pairs(t) do
+ local numkey = type(key) == 'number'
+ if not_clever then
+ key = tostring(key)
+ put(indent..index(numkey,key)..set)
+ writeit(val,indent,newindent)
+ else
+ if not numkey or not used[key] then -- non-array indices
+ if numkey or not is_identifier(key) then
+ key = index(numkey,key)
+ end
+ put(indent..key..set)
+ writeit(val,indent,newindent)
+ end
+ end
+ end
+ eat_last_comma()
+ putln(oldindent..'},')
+ else
+ putln(tostring(t)..',')
+ end
+ end
+ writeit(tbl,'',space)
+ eat_last_comma()
+ return concat(lines,#space > 0 and '\n' or '')
+end
+
+--- Dump a Lua table out to a file or stdout.
+-- @param t {table} The table to write to a file or stdout.
+-- @param ... {string} (optional) File name to write too. Defaults to writing
+-- to stdout.
+function pretty.dump (t,...)
+ if select('#',...)==0 then
+ print(pretty.write(t))
+ return true
+ else
+ return utils.writefile(...,pretty.write(t))
+ end
+end
+
+local memp,nump = {'B','KiB','MiB','GiB'},{'','K','M','B'}
+
+local comma
+function comma (val)
+ local thou = math.floor(val/1000)
+ if thou > 0 then return comma(thou)..','..(val % 1000)
+ else return tostring(val) end
+end
+
+--- format large numbers nicely for human consumption.
+-- @param num a number
+-- @param kind one of 'M' (memory in KiB etc), 'N' (postfixes are 'K','M' and 'B')
+-- and 'T' (use commas as thousands separator)
+-- @param prec number of digits to use for 'M' and 'N' (default 1)
+function pretty.number (num,kind,prec)
+ local fmt = '%.'..(prec or 1)..'f%s'
+ if kind == 'T' then
+ return comma(num)
+ else
+ local postfixes, fact
+ if kind == 'M' then
+ fact = 1024
+ postfixes = memp
+ else
+ fact = 1000
+ postfixes = nump
+ end
+ local div = fact
+ local k = 1
+ while num >= div and k <= #postfixes do
+ div = div * fact
+ k = k + 1
+ end
+ div = div / fact
+ if k > #postfixes then k = k - 1; div = div/fact end
+ if k > 1 then
+ return fmt:format(num/div,postfixes[k] or 'duh')
+ else
+ return num..postfixes[1]
+ end
+ end
+end
+
+return pretty
--- /dev/null
+--- Manipulating iterators as sequences.
+-- See @{07-functional.md.Sequences|The Guide}
+--
+-- Dependencies: `pl.utils`, `debug`
+-- @module pl.seq
+
+local next,assert,type,pairs,tonumber,type,setmetatable,getmetatable,_G = next,assert,type,pairs,tonumber,type,setmetatable,getmetatable,_G
+local strfind = string.find
+local strmatch = string.match
+local format = string.format
+local mrandom = math.random
+local remove,tsort,tappend = table.remove,table.sort,table.insert
+local io = io
+local utils = require 'pl.utils'
+local function_arg = utils.function_arg
+local _List = utils.stdmt.List
+local _Map = utils.stdmt.Map
+local assert_arg = utils.assert_arg
+require 'debug'
+
+local seq = {}
+
+-- given a number, return a function(y) which returns true if y > x
+-- @param x a number
+function seq.greater_than(x)
+ return function(v)
+ return tonumber(v) > x
+ end
+end
+
+-- given a number, returns a function(y) which returns true if y < x
+-- @param x a number
+function seq.less_than(x)
+ return function(v)
+ return tonumber(v) < x
+ end
+end
+
+-- given any value, return a function(y) which returns true if y == x
+-- @param x a value
+function seq.equal_to(x)
+ if type(x) == "number" then
+ return function(v)
+ return tonumber(v) == x
+ end
+ else
+ return function(v)
+ return v == x
+ end
+ end
+end
+
+--- given a string, return a function(y) which matches y against the string.
+-- @param s a string
+function seq.matching(s)
+ return function(v)
+ return strfind(v,s)
+ end
+end
+
+--- sequence adaptor for a table. Note that if any generic function is
+-- passed a table, it will automatically use seq.list()
+-- @param t a list-like table
+-- @usage sum(list(t)) is the sum of all elements of t
+-- @usage for x in list(t) do...end
+function seq.list(t)
+ assert_arg(1,t,'table')
+ local key,value
+ return function()
+ key,value = next(t,key)
+ return value
+ end
+end
+
+--- return the keys of the table.
+-- @param t a list-like table
+-- @return iterator over keys
+function seq.keys(t)
+ assert_arg(1,t,'table')
+ local key,value
+ return function()
+ key,value = next(t,key)
+ return key
+ end
+end
+
+local list = seq.list
+local function default_iter(iter)
+ if type(iter) == 'table' then return list(iter)
+ else return iter end
+end
+
+seq.iter = default_iter
+
+--- create an iterator over a numerical range. Like the standard Python function xrange.
+-- @param start a number
+-- @param finish a number greater than start
+function seq.range(start,finish)
+ local i = start - 1
+ return function()
+ i = i + 1
+ if i > finish then return nil
+ else return i end
+ end
+end
+
+-- count the number of elements in the sequence which satisfy the predicate
+-- @param iter a sequence
+-- @param condn a predicate function (must return either true or false)
+-- @param optional argument to be passed to predicate as second argument.
+-- @return count
+function seq.count(iter,condn,arg)
+ local i = 0
+ seq.foreach(iter,function(val)
+ if condn(val,arg) then i = i + 1 end
+ end)
+ return i
+end
+
+--- return the minimum and the maximum value of the sequence.
+-- @param iter a sequence
+-- @return minimum value
+-- @return maximum value
+function seq.minmax(iter)
+ local vmin,vmax = 1e70,-1e70
+ for v in default_iter(iter) do
+ v = tonumber(v)
+ if v < vmin then vmin = v end
+ if v > vmax then vmax = v end
+ end
+ return vmin,vmax
+end
+
+--- return the sum and element count of the sequence.
+-- @param iter a sequence
+-- @param fn an optional function to apply to the values
+function seq.sum(iter,fn)
+ local s = 0
+ local i = 0
+ for v in default_iter(iter) do
+ if fn then v = fn(v) end
+ s = s + v
+ i = i + 1
+ end
+ return s,i
+end
+
+--- create a table from the sequence. (This will make the result a List.)
+-- @param iter a sequence
+-- @return a List
+-- @usage copy(list(ls)) is equal to ls
+-- @usage copy(list {1,2,3}) == List{1,2,3}
+function seq.copy(iter)
+ local res = {}
+ for v in default_iter(iter) do
+ tappend(res,v)
+ end
+ setmetatable(res,_List)
+ return res
+end
+
+--- create a table of pairs from the double-valued sequence.
+-- @param iter a double-valued sequence
+-- @param i1 used to capture extra iterator values
+-- @param i2 as with pairs & ipairs
+-- @usage copy2(ipairs{10,20,30}) == {{1,10},{2,20},{3,30}}
+-- @return a list-like table
+function seq.copy2 (iter,i1,i2)
+ local res = {}
+ for v1,v2 in iter,i1,i2 do
+ tappend(res,{v1,v2})
+ end
+ return res
+end
+
+--- create a table of 'tuples' from a multi-valued sequence.
+-- A generalization of copy2 above
+-- @param iter a multiple-valued sequence
+-- @return a list-like table
+function seq.copy_tuples (iter)
+ iter = default_iter(iter)
+ local res = {}
+ local row = {iter()}
+ while #row > 0 do
+ tappend(res,row)
+ row = {iter()}
+ end
+ return res
+end
+
+--- return an iterator of random numbers.
+-- @param n the length of the sequence
+-- @param l same as the first optional argument to math.random
+-- @param u same as the second optional argument to math.random
+-- @return a sequnce
+function seq.random(n,l,u)
+ local rand
+ assert(type(n) == 'number')
+ if u then
+ rand = function() return mrandom(l,u) end
+ elseif l then
+ rand = function() return mrandom(l) end
+ else
+ rand = mrandom
+ end
+
+ return function()
+ if n == 0 then return nil
+ else
+ n = n - 1
+ return rand()
+ end
+ end
+end
+
+--- return an iterator to the sorted elements of a sequence.
+-- @param iter a sequence
+-- @param comp an optional comparison function (comp(x,y) is true if x < y)
+function seq.sort(iter,comp)
+ local t = seq.copy(iter)
+ tsort(t,comp)
+ return list(t)
+end
+
+--- return an iterator which returns elements of two sequences.
+-- @param iter1 a sequence
+-- @param iter2 a sequence
+-- @usage for x,y in seq.zip(ls1,ls2) do....end
+function seq.zip(iter1,iter2)
+ iter1 = default_iter(iter1)
+ iter2 = default_iter(iter2)
+ return function()
+ return iter1(),iter2()
+ end
+end
+
+--- A table where the key/values are the values and value counts of the sequence.
+-- This version works with 'hashable' values like strings and numbers. <br>
+-- pl.tablex.count_map is more general.
+-- @param iter a sequence
+-- @return a map-like table
+-- @return a table
+-- @see pl.tablex.count_map
+function seq.count_map(iter)
+ local t = {}
+ local v
+ for s in default_iter(iter) do
+ v = t[s]
+ if v then t[s] = v + 1
+ else t[s] = 1 end
+ end
+ return setmetatable(t,_Map)
+end
+
+-- given a sequence, return all the unique values in that sequence.
+-- @param iter a sequence
+-- @param returns_table true if we return a table, not a sequence
+-- @return a sequence or a table; defaults to a sequence.
+function seq.unique(iter,returns_table)
+ local t = seq.count_map(iter)
+ local res = {}
+ for k in pairs(t) do tappend(res,k) end
+ table.sort(res)
+ if returns_table then
+ return res
+ else
+ return list(res)
+ end
+end
+
+--- print out a sequence iter with a separator.
+-- @param iter a sequence
+-- @param sep the separator (default space)
+-- @param nfields maximum number of values per line (default 7)
+-- @param fmt optional format function for each value
+function seq.printall(iter,sep,nfields,fmt)
+ local write = io.write
+ if not sep then sep = ' ' end
+ if not nfields then
+ if sep == '\n' then nfields = 1e30
+ else nfields = 7 end
+ end
+ if fmt then
+ local fstr = fmt
+ fmt = function(v) return format(fstr,v) end
+ end
+ local k = 1
+ for v in default_iter(iter) do
+ if fmt then v = fmt(v) end
+ if k < nfields then
+ write(v,sep)
+ k = k + 1
+ else
+ write(v,'\n')
+ k = 1
+ end
+ end
+ write '\n'
+end
+
+-- return an iterator running over every element of two sequences (concatenation).
+-- @param iter1 a sequence
+-- @param iter2 a sequence
+function seq.splice(iter1,iter2)
+ iter1 = default_iter(iter1)
+ iter2 = default_iter(iter2)
+ local iter = iter1
+ return function()
+ local ret = iter()
+ if ret == nil then
+ if iter == iter1 then
+ iter = iter2
+ return iter()
+ else return nil end
+ else
+ return ret
+ end
+ end
+end
+
+--- return a sequence where every element of a sequence has been transformed
+-- by a function. If you don't supply an argument, then the function will
+-- receive both values of a double-valued sequence, otherwise behaves rather like
+-- tablex.map.
+-- @param fn a function to apply to elements; may take two arguments
+-- @param iter a sequence of one or two values
+-- @param arg optional argument to pass to function.
+function seq.map(fn,iter,arg)
+ fn = function_arg(1,fn)
+ iter = default_iter(iter)
+ return function()
+ local v1,v2 = iter()
+ if v1 == nil then return nil end
+ if arg then return fn(v1,arg) or false
+ else return fn(v1,v2) or false
+ end
+ end
+end
+
+--- filter a sequence using a predicate function
+-- @param iter a sequence of one or two values
+-- @param pred a boolean function; may take two arguments
+-- @param arg optional argument to pass to function.
+function seq.filter (iter,pred,arg)
+ pred = function_arg(2,pred)
+ return function ()
+ local v1,v2
+ while true do
+ v1,v2 = iter()
+ if v1 == nil then return nil end
+ if arg then
+ if pred(v1,arg) then return v1,v2 end
+ else
+ if pred(v1,v2) then return v1,v2 end
+ end
+ end
+ end
+end
+
+--- 'reduce' a sequence using a binary function.
+-- @param fun a function of two arguments
+-- @param iter a sequence
+-- @param oldval optional initial value
+-- @usage seq.reduce(operator.add,seq.list{1,2,3,4}) == 10
+-- @usage seq.reduce('-',{1,2,3,4,5}) == -13
+function seq.reduce (fun,iter,oldval)
+ fun = function_arg(1,fun)
+ iter = default_iter(iter)
+ if not oldval then
+ oldval = iter()
+ end
+ local val = oldval
+ for v in iter do
+ val = fun(val,v)
+ end
+ return val
+end
+
+--- take the first n values from the sequence.
+-- @param iter a sequence of one or two values
+-- @param n number of items to take
+-- @return a sequence of at most n items
+function seq.take (iter,n)
+ local i = 1
+ iter = default_iter(iter)
+ return function()
+ if i > n then return end
+ local val1,val2 = iter()
+ if not val1 then return end
+ i = i + 1
+ return val1,val2
+ end
+end
+
+--- skip the first n values of a sequence
+-- @param iter a sequence of one or more values
+-- @param n number of items to skip
+function seq.skip (iter,n)
+ n = n or 1
+ for i = 1,n do iter() end
+ return iter
+end
+
+--- a sequence with a sequence count and the original value. <br>
+-- enum(copy(ls)) is a roundabout way of saying ipairs(ls).
+-- @param iter a single or double valued sequence
+-- @return sequence of (i,v), i = 1..n and v is from iter.
+function seq.enum (iter)
+ local i = 0
+ iter = default_iter(iter)
+ return function ()
+ local val1,val2 = iter()
+ if not val1 then return end
+ i = i + 1
+ return i,val1,val2
+ end
+end
+
+--- map using a named method over a sequence.
+-- @param iter a sequence
+-- @param name the method name
+-- @param arg1 optional first extra argument
+-- @param arg2 optional second extra argument
+function seq.mapmethod (iter,name,arg1,arg2)
+ iter = default_iter(iter)
+ return function()
+ local val = iter()
+ if not val then return end
+ local fn = val[name]
+ if not fn then error(type(val).." does not have method "..name) end
+ return fn(val,arg1,arg2)
+ end
+end
+
+--- a sequence of (last,current) values from another sequence.
+-- This will return S(i-1),S(i) if given S(i)
+-- @param iter a sequence
+function seq.last (iter)
+ iter = default_iter(iter)
+ local l = iter()
+ if l == nil then return nil end
+ return function ()
+ local val,ll
+ val = iter()
+ if val == nil then return nil end
+ ll = l
+ l = val
+ return val,ll
+ end
+end
+
+--- call the function on each element of the sequence.
+-- @param iter a sequence with up to 3 values
+-- @param fn a function
+function seq.foreach(iter,fn)
+ fn = function_arg(2,fn)
+ for i1,i2,i3 in default_iter(iter) do fn(i1,i2,i3) end
+end
+
+---------------------- Sequence Adapters ---------------------
+
+local SMT
+local callable = utils.is_callable
+
+local function SW (iter,...)
+ if callable(iter) then
+ return setmetatable({iter=iter},SMT)
+ else
+ return iter,...
+ end
+end
+
+
+-- can't directly look these up in seq because of the wrong argument order...
+local map,reduce,mapmethod = seq.map, seq.reduce, seq.mapmethod
+local overrides = {
+ map = function(self,fun,arg)
+ return map(fun,self,arg)
+ end,
+ reduce = function(self,fun)
+ return reduce(fun,self)
+ end
+}
+
+SMT = {
+ __index = function (tbl,key)
+ local s = overrides[key] or seq[key]
+ if s then
+ return function(sw,...) return SW(s(sw.iter,...)) end
+ else
+ return function(sw,...) return SW(mapmethod(sw.iter,key,...)) end
+ end
+ end,
+ __call = function (sw)
+ return sw.iter()
+ end,
+}
+
+setmetatable(seq,{
+ __call = function(tbl,iter)
+ if not callable(iter) then
+ if type(iter) == 'table' then iter = seq.list(iter)
+ else return iter
+ end
+ end
+ return setmetatable({iter=iter},SMT)
+ end
+})
+
+--- create a wrapped iterator over all lines in the file.
+-- @param f either a filename or nil (for standard input)
+-- @return a sequence wrapper
+function seq.lines (f)
+ local iter = f and io.lines(f) or io.lines()
+ return SW(iter)
+end
+
+function seq.import ()
+ _G.debug.setmetatable(function() end,{
+ __index = function(tbl,key)
+ local s = overrides[key] or seq[key]
+ if s then return s
+ else
+ return function(s,...) return seq.mapmethod(s,key,...) end
+ end
+ end
+ })
+end
+
+return seq
--- /dev/null
+--- Simple Input Patterns (SIP).
+-- SIP patterns start with '$', then a
+-- one-letter type, and then an optional variable in curly braces.
+--
+-- sip.match('$v=$q','name="dolly"',res)
+-- ==> res=={'name','dolly'}
+-- sip.match('($q{first},$q{second})','("john","smith")',res)
+-- ==> res=={second='smith',first='john'}
+--
+-- ''Type names''
+--
+-- v identifier
+-- i integer
+-- f floating-point
+-- q quoted string
+-- ([{< match up to closing bracket
+--
+-- See @{08-additional.md.Simple_Input_Patterns|the Guide}
+--
+-- @module pl.sip
+
+if not rawget(_G,'loadstring') then -- Lua 5.2 full compatibility
+ loadstring = load
+ unpack = table.unpack
+end
+
+local append,concat = table.insert,table.concat
+local concat = table.concat
+local ipairs,loadstring,type,unpack = ipairs,loadstring,type,unpack
+local io,_G = io,_G
+local print,rawget = print,rawget
+
+local patterns = {
+ FLOAT = '[%+%-%d]%d*%.?%d*[eE]?[%+%-]?%d*',
+ INTEGER = '[+%-%d]%d*',
+ IDEN = '[%a_][%w_]*',
+ FILE = '[%a%.\\][:%][%w%._%-\\]*'
+}
+
+local function assert_arg(idx,val,tp)
+ if type(val) ~= tp then
+ error("argument "..idx.." must be "..tp, 2)
+ end
+end
+
+
+--[[
+module ('pl.sip',utils._module)
+]]
+
+local sip = {}
+
+local brackets = {['<'] = '>', ['('] = ')', ['{'] = '}', ['['] = ']' }
+local stdclasses = {a=1,c=0,d=1,l=1,p=0,u=1,w=1,x=1,s=0}
+
+local _patterns = {}
+
+
+local function group(s)
+ return '('..s..')'
+end
+
+-- escape all magic characters except $, which has special meaning
+-- Also, un-escape any characters after $, so $( passes through as is.
+local function escape (spec)
+ --_G.print('spec',spec)
+ local res = spec:gsub('[%-%.%+%[%]%(%)%^%%%?%*]','%%%1'):gsub('%$%%(%S)','$%1')
+ --_G.print('res',res)
+ return res
+end
+
+local function imcompressible (s)
+ return s:gsub('%s+','\001')
+end
+
+-- [handling of spaces in patterns]
+-- spaces may be 'compressed' (i.e will match zero or more spaces)
+-- unless this occurs within a number or an identifier. So we mark
+-- the four possible imcompressible patterns first and then replace.
+-- The possible alnum patterns are v,f,a,d,x,l and u.
+local function compress_spaces (s)
+ s = s:gsub('%$[vifadxlu]%s+%$[vfadxlu]',imcompressible)
+ s = s:gsub('[%w_]%s+[%w_]',imcompressible)
+ s = s:gsub('[%w_]%s+%$[vfadxlu]',imcompressible)
+ s = s:gsub('%$[vfadxlu]%s+[%w_]',imcompressible)
+ s = s:gsub('%s+','%%s*')
+ s = s:gsub('\001',' ')
+ return s
+end
+
+local pattern_map = {
+ v = group(patterns.IDEN),
+ i = group(patterns.INTEGER),
+ f = group(patterns.FLOAT),
+ r = '(%S.*)',
+ p = '([%a]?[:]?[\\/%.%w_]+)'
+}
+
+function sip.custom_pattern(flag,patt)
+ pattern_map[flag] = patt
+end
+
+--- convert a SIP pattern into the equivalent Lua string pattern.
+-- @param spec a SIP pattern
+-- @param options a table; only the <code>at_start</code> field is
+-- currently meaningful and esures that the pattern is anchored
+-- at the start of the string.
+-- @return a Lua string pattern.
+function sip.create_pattern (spec,options)
+ assert_arg(1,spec,'string')
+ local fieldnames,fieldtypes = {},{}
+
+ if type(spec) == 'string' then
+ spec = escape(spec)
+ else
+ local res = {}
+ for i,s in ipairs(spec) do
+ res[i] = escape(s)
+ end
+ spec = concat(res,'.-')
+ end
+
+ local kount = 1
+
+ local function addfield (name,type)
+ if not name then name = kount end
+ if fieldnames then append(fieldnames,name) end
+ if fieldtypes then fieldtypes[name] = type end
+ kount = kount + 1
+ end
+
+ local named_vars, pattern
+ named_vars = spec:find('{%a+}')
+ pattern = '%$%S'
+
+ if options and options.at_start then
+ spec = '^'..spec
+ end
+ if spec:sub(-1,-1) == '$' then
+ spec = spec:sub(1,-2)..'$r'
+ if named_vars then spec = spec..'{rest}' end
+ end
+
+
+ local names
+
+ if named_vars then
+ names = {}
+ spec = spec:gsub('{(%a+)}',function(name)
+ append(names,name)
+ return ''
+ end)
+ end
+ spec = compress_spaces(spec)
+
+ local k = 1
+ local err
+ local r = (spec:gsub(pattern,function(s)
+ local type,name
+ type = s:sub(2,2)
+ if names then name = names[k]; k=k+1 end
+ -- this kludge is necessary because %q generates two matches, and
+ -- we want to ignore the first. Not a problem for named captures.
+ if not names and type == 'q' then
+ addfield(nil,'Q')
+ else
+ addfield(name,type)
+ end
+ local res
+ if pattern_map[type] then
+ res = pattern_map[type]
+ elseif type == 'q' then
+ -- some Lua pattern matching voodoo; we want to match '...' as
+ -- well as "...", and can use the fact that %n will match a
+ -- previous capture. Adding the extra field above comes from needing
+ -- to accomodate the extra spurious match (which is either ' or ")
+ addfield(name,type)
+ res = '(["\'])(.-)%'..(kount-2)
+ else
+ local endbracket = brackets[type]
+ if endbracket then
+ res = '(%b'..type..endbracket..')'
+ elseif stdclasses[type] or stdclasses[type:lower()] then
+ res = '(%'..type..'+)'
+ else
+ err = "unknown format type or character class"
+ end
+ end
+ return res
+ end))
+ --print(r,err)
+ if err then
+ return nil,err
+ else
+ return r,fieldnames,fieldtypes
+ end
+end
+
+
+local function tnumber (s)
+ return s == 'd' or s == 'i' or s == 'f'
+end
+
+function sip.create_spec_fun(spec,options)
+ local fieldtypes,fieldnames
+ local ls = {}
+ spec,fieldnames,fieldtypes = sip.create_pattern(spec,options)
+ if not spec then return spec,fieldnames end
+ local named_vars = type(fieldnames[1]) == 'string'
+ for i = 1,#fieldnames do
+ append(ls,'mm'..i)
+ end
+ local fun = ('return (function(s,res)\n\tlocal %s = s:match(%q)\n'):format(concat(ls,','),spec)
+ fun = fun..'\tif not mm1 then return false end\n'
+ local k=1
+ for i,f in ipairs(fieldnames) do
+ if f ~= '_' then
+ local var = 'mm'..i
+ if tnumber(fieldtypes[f]) then
+ var = 'tonumber('..var..')'
+ elseif brackets[fieldtypes[f]] then
+ var = var..':sub(2,-2)'
+ end
+ if named_vars then
+ fun = ('%s\tres.%s = %s\n'):format(fun,f,var)
+ else
+ if fieldtypes[f] ~= 'Q' then -- we skip the string-delim capture
+ fun = ('%s\tres[%d] = %s\n'):format(fun,k,var)
+ k = k + 1
+ end
+ end
+ end
+ end
+ return fun..'\treturn true\nend)\n', named_vars
+end
+
+--- convert a SIP pattern into a matching function.
+-- The returned function takes two arguments, the line and an empty table.
+-- If the line matched the pattern, then this function return true
+-- and the table is filled with field-value pairs.
+-- @param spec a SIP pattern
+-- @param options optional table; {anywhere=true} will stop pattern anchoring at start
+-- @return a function if successful, or nil,<error>
+function sip.compile(spec,options)
+ assert_arg(1,spec,'string')
+ local fun,names = sip.create_spec_fun(spec,options)
+ if not fun then return nil,names end
+ if rawget(_G,'_DEBUG') then print(fun) end
+ local chunk,err = loadstring(fun,'tmp')
+ if err then return nil,err end
+ return chunk(),names
+end
+
+local cache = {}
+
+--- match a SIP pattern against a string.
+-- @param spec a SIP pattern
+-- @param line a string
+-- @param res a table to receive values
+-- @param options (optional) option table
+-- @return true or false
+function sip.match (spec,line,res,options)
+ assert_arg(1,spec,'string')
+ assert_arg(2,line,'string')
+ assert_arg(3,res,'table')
+ if not cache[spec] then
+ cache[spec] = sip.compile(spec,options)
+ end
+ return cache[spec](line,res)
+end
+
+--- match a SIP pattern against the start of a string.
+-- @param spec a SIP pattern
+-- @param line a string
+-- @param res a table to receive values
+-- @return true or false
+function sip.match_at_start (spec,line,res)
+ return sip.match(spec,line,res,{at_start=true})
+end
+
+--- given a pattern and a file object, return an iterator over the results
+-- @param spec a SIP pattern
+-- @param f a file - use standard input if not specified.
+function sip.fields (spec,f)
+ assert_arg(1,spec,'string')
+ f = f or io.stdin
+ local fun,err = sip.compile(spec)
+ if not fun then return nil,err end
+ local res = {}
+ return function()
+ while true do
+ local line = f:read()
+ if not line then return end
+ if fun(line,res) then
+ local values = res
+ res = {}
+ return unpack(values)
+ end
+ end
+ end
+end
+
+--- register a match which will be used in the read function.
+-- @param spec a SIP pattern
+-- @param fun a function to be called with the results of the match
+-- @see read
+function sip.pattern (spec,fun)
+ assert_arg(1,spec,'string')
+ local pat,named = sip.compile(spec)
+ append(_patterns,{pat=pat,named=named,callback=fun or false})
+end
+
+--- enter a loop which applies all registered matches to the input file.
+-- @param f a file object; if nil, then io.stdin is assumed.
+function sip.read (f)
+ local owned,err
+ f = f or io.stdin
+ if type(f) == 'string' then
+ f,err = io.open(f)
+ if not f then return nil,err end
+ owned = true
+ end
+ local res = {}
+ for line in f:lines() do
+ for _,item in ipairs(_patterns) do
+ if item.pat(line,res) then
+ if item.callback then
+ if item.named then
+ item.callback(res)
+ else
+ item.callback(unpack(res))
+ end
+ end
+ res = {}
+ break
+ end
+ end
+ end
+ if owned then f:close() end
+end
+
+return sip
--- /dev/null
+--- Checks uses of undeclared global variables.
+-- All global variables must be 'declared' through a regular assignment
+-- (even assigning nil will do) in a main chunk before being used
+-- anywhere or assigned to inside a function.
+-- @module pl.strict
+
+require 'debug'
+local getinfo, error, rawset, rawget = debug.getinfo, error, rawset, rawget
+local handler,hooked
+
+local mt = getmetatable(_G)
+if mt == nil then
+ mt = {}
+ setmetatable(_G, mt)
+elseif mt.hook then
+ hooked = true
+end
+
+-- predeclaring _PROMPT keeps the Lua Interpreter happy
+mt.__declared = {_PROMPT=true}
+
+local function what ()
+ local d = getinfo(3, "S")
+ return d and d.what or "C"
+end
+
+mt.__newindex = function (t, n, v)
+ if not mt.__declared[n] then
+ local w = what()
+ if w ~= "main" and w ~= "C" then
+ error("assign to undeclared variable '"..n.."'", 2)
+ end
+ mt.__declared[n] = true
+ end
+ rawset(t, n, v)
+end
+
+handler = function(t,n)
+ if not mt.__declared[n] and what() ~= "C" then
+ error("variable '"..n.."' is not declared", 2)
+ end
+ return rawget(t, n)
+end
+
+function package.strict (mod)
+ local mt = getmetatable(mod)
+ if mt == nil then
+ mt = {}
+ setmetatable(mod, mt)
+ end
+ mt.__declared = {}
+ mt.__newindex = function(t, n, v)
+ mt.__declared[n] = true
+ rawset(t, n, v)
+ end
+ mt.__index = function(t,n)
+ if not mt.__declared[n] then
+ error("variable '"..n.."' is not declared", 2)
+ end
+ return rawget(t, n)
+ end
+end
+
+if not hooked then
+ mt.__index = handler
+else
+ mt.hook(handler)
+end
+
+
--- /dev/null
+--- Reading and writing strings using file-like objects. <br>
+--
+-- f = stringio.open(text)
+-- l1 = f:read() -- read first line
+-- n,m = f:read ('*n','*n') -- read two numbers
+-- for line in f:lines() do print(line) end -- iterate over all lines
+-- f = stringio.create()
+-- f:write('hello')
+-- f:write('dolly')
+-- assert(f:value(),'hellodolly')
+--
+-- See @{03-strings.md.File_style_I_O_on_Strings|the Guide}.
+-- @module pl.stringio
+
+if not rawget(_G,'loadstring') then -- Lua 5.2 full compatibility
+ unpack = table.unpack
+end
+
+local getmetatable,tostring,unpack,tonumber = getmetatable,tostring,unpack,tonumber
+local concat,append = table.concat,table.insert
+
+local stringio = {}
+
+--- Writer class
+local SW = {}
+SW.__index = SW
+
+local function xwrite(self,...)
+ local args = {...} --arguments may not be nil!
+ for i = 1, #args do
+ append(self.tbl,args[i])
+ end
+end
+
+function SW:write(arg1,arg2,...)
+ if arg2 then
+ xwrite(self,arg1,arg2,...)
+ else
+ append(self.tbl,arg1)
+ end
+end
+
+function SW:writef(fmt,...)
+ self:write(fmt:format(...))
+end
+
+function SW:value()
+ return concat(self.tbl)
+end
+
+function SW:__tostring()
+ return self:value()
+end
+
+function SW:close() -- for compatibility only
+end
+
+function SW:seek()
+end
+
+--- Reader class
+local SR = {}
+SR.__index = SR
+
+function SR:_read(fmt)
+ local i,str = self.i,self.str
+ local sz = #str
+ if i > sz then return nil end
+ local res
+ if fmt == '*l' or fmt == '*L' then
+ local idx = str:find('\n',i) or (sz+1)
+ res = str:sub(i,fmt == '*l' and idx-1 or idx)
+ self.i = idx+1
+ elseif fmt == '*a' then
+ res = str:sub(i)
+ self.i = sz
+ elseif fmt == '*n' then
+ local _,i2,i2,idx
+ _,idx = str:find ('%s*%d+',i)
+ _,i2 = str:find ('^%.%d+',idx+1)
+ if i2 then idx = i2 end
+ _,i2 = str:find ('^[eE][%+%-]*%d+',idx+1)
+ if i2 then idx = i2 end
+ local val = str:sub(i,idx)
+ res = tonumber(val)
+ self.i = idx+1
+ elseif type(fmt) == 'number' then
+ res = str:sub(i,i+fmt-1)
+ self.i = i + fmt
+ else
+ error("bad read format",2)
+ end
+ return res
+end
+
+function SR:read(...)
+ if select('#',...) == 0 then
+ return self:_read('*l')
+ else
+ local res, fmts = {},{...}
+ for i = 1, #fmts do
+ res[i] = self:_read(fmts[i])
+ end
+ return unpack(res)
+ end
+end
+
+function SR:seek(whence,offset)
+ local base
+ whence = whence or 'cur'
+ offset = offset or 0
+ if whence == 'set' then
+ base = 1
+ elseif whence == 'cur' then
+ base = self.i
+ elseif whence == 'end' then
+ base = #self.str
+ end
+ self.i = base + offset
+ return self.i
+end
+
+function SR:lines(...)
+ local n, args = select('#',...)
+ if n > 0 then
+ args = {...}
+ end
+ return function()
+ if n == 0 then
+ return self:_read '*l'
+ else
+ return self:read(unpack(args))
+ end
+ end
+end
+
+function SR:close() -- for compatibility only
+end
+
+--- create a file-like object which can be used to construct a string.
+-- The resulting object has an extra <code>value()</code> method for
+-- retrieving the string value.
+-- @usage f = create(); f:write('hello, dolly\n'); print(f:value())
+function stringio.create()
+ return setmetatable({tbl={}},SW)
+end
+
+--- create a file-like object for reading from a given string.
+-- @param s The input string.
+function stringio.open(s)
+ return setmetatable({str=s,i=1},SR)
+end
+
+function stringio.lines(s,...)
+ return stringio.open(s):lines(...)
+end
+
+return stringio
--- /dev/null
+--- Python-style extended string library.
+--
+-- see 3.6.1 of the Python reference.
+-- If you want to make these available as string methods, then say
+-- `stringx.import()` to bring them into the standard `string` table.
+--
+-- See @{03-strings.md|the Guide}
+--
+-- Dependencies: `pl.utils`
+-- @module pl.stringx
+local utils = require 'pl.utils'
+local string = string
+local find = string.find
+local type,setmetatable,getmetatable,ipairs,unpack = type,setmetatable,getmetatable,ipairs,unpack
+local error,tostring = error,tostring
+local gsub = string.gsub
+local rep = string.rep
+local sub = string.sub
+local concat = table.concat
+local escape = utils.escape
+local ceil = math.ceil
+local _G = _G
+local assert_arg,usplit,list_MT = utils.assert_arg,utils.split,utils.stdmt.List
+local lstrip
+
+local function assert_string (n,s)
+ assert_arg(n,s,'string')
+end
+
+local function non_empty(s)
+ return #s > 0
+end
+
+local function assert_nonempty_string(n,s)
+ assert_arg(n,s,'string',non_empty,'must be a non-empty string')
+end
+
+local stringx = {}
+
+--- does s only contain alphabetic characters?.
+-- @param s a string
+function stringx.isalpha(s)
+ assert_string(1,s)
+ return find(s,'^%a+$') == 1
+end
+
+--- does s only contain digits?.
+-- @param s a string
+function stringx.isdigit(s)
+ assert_string(1,s)
+ return find(s,'^%d+$') == 1
+end
+
+--- does s only contain alphanumeric characters?.
+-- @param s a string
+function stringx.isalnum(s)
+ assert_string(1,s)
+ return find(s,'^%w+$') == 1
+end
+
+--- does s only contain spaces?.
+-- @param s a string
+function stringx.isspace(s)
+ assert_string(1,s)
+ return find(s,'^%s+$') == 1
+end
+
+--- does s only contain lower case characters?.
+-- @param s a string
+function stringx.islower(s)
+ assert_string(1,s)
+ return find(s,'^[%l%s]+$') == 1
+end
+
+--- does s only contain upper case characters?.
+-- @param s a string
+function stringx.isupper(s)
+ assert_string(1,s)
+ return find(s,'^[%u%s]+$') == 1
+end
+
+--- concatenate the strings using this string as a delimiter.
+-- @param self the string
+-- @param seq a table of strings or numbers
+-- @usage (' '):join {1,2,3} == '1 2 3'
+function stringx.join (self,seq)
+ assert_string(1,self)
+ return concat(seq,self)
+end
+
+--- does string start with the substring?.
+-- @param self the string
+-- @param s2 a string
+function stringx.startswith(self,s2)
+ assert_string(1,self)
+ assert_string(2,s2)
+ return find(self,s2,1,true) == 1
+end
+
+local function _find_all(s,sub,first,last)
+ if sub == '' then return #s+1,#s end
+ local i1,i2 = find(s,sub,first,true)
+ local res
+ local k = 0
+ while i1 do
+ res = i1
+ k = k + 1
+ i1,i2 = find(s,sub,i2+1,true)
+ if last and i1 > last then break end
+ end
+ return res,k
+end
+
+--- does string end with the given substring?.
+-- @param s a string
+-- @param send a substring or a table of suffixes
+function stringx.endswith(s,send)
+ assert_string(1,s)
+ if type(send) == 'string' then
+ return #s >= #send and s:find(send, #s-#send+1, true) and true or false
+ elseif type(send) == 'table' then
+ local endswith = stringx.endswith
+ for _,suffix in ipairs(send) do
+ if endswith(s,suffix) then return true end
+ end
+ return false
+ else
+ error('argument #2: either a substring or a table of suffixes expected')
+ end
+end
+
+-- break string into a list of lines
+-- @param self the string
+-- @param keepends (currently not used)
+function stringx.splitlines (self,keepends)
+ assert_string(1,self)
+ local res = usplit(self,'[\r\n]')
+ -- we are currently hacking around a problem with utils.split (see stringx.split)
+ if #res == 0 then res = {''} end
+ return setmetatable(res,list_MT)
+end
+
+local function tab_expand (self,n)
+ return (gsub(self,'([^\t]*)\t', function(s)
+ return s..(' '):rep(n - #s % n)
+ end))
+end
+
+--- replace all tabs in s with n spaces. If not specified, n defaults to 8.
+-- with 0.9.5 this now correctly expands to the next tab stop (if you really
+-- want to just replace tabs, use :gsub('\t',' ') etc)
+-- @param self the string
+-- @param n number of spaces to expand each tab, (default 8)
+function stringx.expandtabs(self,n)
+ assert_string(1,self)
+ n = n or 8
+ if not self:find '\n' then return tab_expand(self,n) end
+ local res,i = {},1
+ for line in stringx.lines(self) do
+ res[i] = tab_expand(line,n)
+ i = i + 1
+ end
+ return table.concat(res,'\n')
+end
+
+--- find index of first instance of sub in s from the left.
+-- @param self the string
+-- @param sub substring
+-- @param i1 start index
+function stringx.lfind(self,sub,i1)
+ assert_string(1,self)
+ assert_string(2,sub)
+ local idx = find(self,sub,i1,true)
+ if idx then return idx else return nil end
+end
+
+--- find index of first instance of sub in s from the right.
+-- @param self the string
+-- @param sub substring
+-- @param first first index
+-- @param last last index
+function stringx.rfind(self,sub,first,last)
+ assert_string(1,self)
+ assert_string(2,sub)
+ local idx = _find_all(self,sub,first,last)
+ if idx then return idx else return nil end
+end
+
+--- replace up to n instances of old by new in the string s.
+-- if n is not present, replace all instances.
+-- @param s the string
+-- @param old the target substring
+-- @param new the substitution
+-- @param n optional maximum number of substitutions
+-- @return result string
+-- @return the number of substitutions
+function stringx.replace(s,old,new,n)
+ assert_string(1,s)
+ assert_string(1,old)
+ return (gsub(s,escape(old),new:gsub('%%','%%%%'),n))
+end
+
+--- split a string into a list of strings using a delimiter.
+-- @class function
+-- @name split
+-- @param self the string
+-- @param re a delimiter (defaults to whitespace)
+-- @param n maximum number of results
+-- @usage #(('one two'):split()) == 2
+-- @usage ('one,two,three'):split(',') == List{'one','two','three'}
+-- @usage ('one,two,three'):split(',',2) == List{'one','two,three'}
+function stringx.split(self,re,n)
+ local s = self
+ local plain = true
+ if not re then -- default spaces
+ s = lstrip(s)
+ plain = false
+ end
+ local res = usplit(s,re,plain,n)
+ if re and re ~= '' and find(s,re,-#re,true) then
+ res[#res+1] = ""
+ end
+ return setmetatable(res,list_MT)
+end
+
+--- split a string using a pattern. Note that at least one value will be returned!
+-- @param self the string
+-- @param re a Lua string pattern (defaults to whitespace)
+-- @return the parts of the string
+-- @usage a,b = line:splitv('=')
+function stringx.splitv (self,re)
+ assert_string(1,self)
+ return utils.splitv(self,re)
+end
+
+local function copy(self)
+ return self..''
+end
+
+--- count all instances of substring in string.
+-- @param self the string
+-- @param sub substring
+function stringx.count(self,sub)
+ assert_string(1,self)
+ local i,k = _find_all(self,sub,1)
+ return k
+end
+
+local function _just(s,w,ch,left,right)
+ local n = #s
+ if w > n then
+ if not ch then ch = ' ' end
+ local f1,f2
+ if left and right then
+ local ln = ceil((w-n)/2)
+ local rn = w - n - ln
+ f1 = rep(ch,ln)
+ f2 = rep(ch,rn)
+ elseif right then
+ f1 = rep(ch,w-n)
+ f2 = ''
+ else
+ f2 = rep(ch,w-n)
+ f1 = ''
+ end
+ return f1..s..f2
+ else
+ return copy(s)
+ end
+end
+
+--- left-justify s with width w.
+-- @param self the string
+-- @param w width of justification
+-- @param ch padding character, default ' '
+function stringx.ljust(self,w,ch)
+ assert_string(1,self)
+ assert_arg(2,w,'number')
+ return _just(self,w,ch,true,false)
+end
+
+--- right-justify s with width w.
+-- @param s the string
+-- @param w width of justification
+-- @param ch padding character, default ' '
+function stringx.rjust(s,w,ch)
+ assert_string(1,s)
+ assert_arg(2,w,'number')
+ return _just(s,w,ch,false,true)
+end
+
+--- center-justify s with width w.
+-- @param s the string
+-- @param w width of justification
+-- @param ch padding character, default ' '
+function stringx.center(s,w,ch)
+ assert_string(1,s)
+ assert_arg(2,w,'number')
+ return _just(s,w,ch,true,true)
+end
+
+local function _strip(s,left,right,chrs)
+ if not chrs then
+ chrs = '%s'
+ else
+ chrs = '['..escape(chrs)..']'
+ end
+ if left then
+ local i1,i2 = find(s,'^'..chrs..'*')
+ if i2 >= i1 then
+ s = sub(s,i2+1)
+ end
+ end
+ if right then
+ local i1,i2 = find(s,chrs..'*$')
+ if i2 >= i1 then
+ s = sub(s,1,i1-1)
+ end
+ end
+ return s
+end
+
+--- trim any whitespace on the left of s.
+-- @param self the string
+-- @param chrs default space, can be a string of characters to be trimmed
+function stringx.lstrip(self,chrs)
+ assert_string(1,self)
+ return _strip(self,true,false,chrs)
+end
+lstrip = stringx.lstrip
+
+--- trim any whitespace on the right of s.
+-- @param s the string
+-- @param chrs default space, can be a string of characters to be trimmed
+function stringx.rstrip(s,chrs)
+ assert_string(1,s)
+ return _strip(s,false,true,chrs)
+end
+
+--- trim any whitespace on both left and right of s.
+-- @param self the string
+-- @param chrs default space, can be a string of characters to be trimmed
+function stringx.strip(self,chrs)
+ assert_string(1,self)
+ return _strip(self,true,true,chrs)
+end
+
+-- The partition functions split a string using a delimiter into three parts:
+-- the part before, the delimiter itself, and the part afterwards
+local function _partition(p,delim,fn)
+ local i1,i2 = fn(p,delim)
+ if not i1 or i1 == -1 then
+ return p,'',''
+ else
+ if not i2 then i2 = i1 end
+ return sub(p,1,i1-1),sub(p,i1,i2),sub(p,i2+1)
+ end
+end
+
+--- partition the string using first occurance of a delimiter
+-- @param self the string
+-- @param ch delimiter
+-- @return part before ch
+-- @return ch
+-- @return part after ch
+function stringx.partition(self,ch)
+ assert_string(1,self)
+ assert_nonempty_string(2,ch)
+ return _partition(self,ch,stringx.lfind)
+end
+
+--- partition the string p using last occurance of a delimiter
+-- @param self the string
+-- @param ch delimiter
+-- @return part before ch
+-- @return ch
+-- @return part after ch
+function stringx.rpartition(self,ch)
+ assert_string(1,self)
+ assert_nonempty_string(2,ch)
+ return _partition(self,ch,stringx.rfind)
+end
+
+--- return the 'character' at the index.
+-- @param self the string
+-- @param idx an index (can be negative)
+-- @return a substring of length 1 if successful, empty string otherwise.
+function stringx.at(self,idx)
+ assert_string(1,self)
+ assert_arg(2,idx,'number')
+ return sub(self,idx,idx)
+end
+
+--- return an interator over all lines in a string
+-- @param self the string
+-- @return an iterator
+function stringx.lines (self)
+ assert_string(1,self)
+ local s = self
+ if not s:find '\n$' then s = s..'\n' end
+ return s:gmatch('([^\n]*)\n')
+end
+
+--- iniital word letters uppercase ('title case').
+-- Here 'words' mean chunks of non-space characters.
+-- @param self the string
+-- @return a string with each word's first letter uppercase
+function stringx.title(self)
+ return (self:gsub('(%S)(%S*)',function(f,r)
+ return f:upper()..r:lower()
+ end))
+end
+
+stringx.capitalize = stringx.title
+
+local elipsis = '...'
+local n_elipsis = #elipsis
+
+--- return a shorted version of a string.
+-- @param self the string
+-- @param sz the maxinum size allowed
+-- @param tail true if we want to show the end of the string (head otherwise)
+function stringx.shorten(self,sz,tail)
+ if #self > sz then
+ if sz < n_elipsis then return elipsis:sub(1,sz) end
+ if tail then
+ local i = #self - sz + 1 + n_elipsis
+ return elipsis .. self:sub(i)
+ else
+ return self:sub(1,sz-n_elipsis) .. elipsis
+ end
+ end
+ return self
+end
+
+function stringx.import(dont_overload)
+ utils.import(stringx,string)
+end
+
+return stringx
--- /dev/null
+--- Extended operations on Lua tables.
+--
+-- See @{02-arrays.md.Useful_Operations_on_Tables|the Guide}
+--
+-- Dependencies: `pl.utils`
+-- @module pl.tablex
+local utils = require ('pl.utils')
+local getmetatable,setmetatable,require = getmetatable,setmetatable,require
+local append,remove = table.insert,table.remove
+local min,max = math.min,math.max
+local pairs,type,unpack,next,select,tostring = pairs,type,unpack,next,select,tostring
+local function_arg = utils.function_arg
+local Set = utils.stdmt.Set
+local List = utils.stdmt.List
+local Map = utils.stdmt.Map
+local assert_arg = utils.assert_arg
+
+local tablex = {}
+
+-- generally, functions that make copies of tables try to preserve the metatable.
+-- However, when the source has no obvious type, then we attach appropriate metatables
+-- like List, Map, etc to the result.
+local function setmeta (res,tbl,def)
+ local mt = getmetatable(tbl) or def
+--~ if mt then
+--~ local mmt = getmetatable(mt)
+--~ if mmt and mmt.__call then return mt(res) end
+--~ end
+ return setmetatable(res, mt)
+end
+
+local function makelist (res)
+ return setmetatable(res,List)
+end
+
+local function assert_arg_indexable (idx,val)
+ if type(val) == 'table' then return end
+ local mt = getmetatable(val)
+ if not(mt and mt.__len and mt.__index) then
+ error(('argument %d is not indexable'):format(idx),2)
+ end
+end
+
+--- copy a table into another, in-place.
+-- @param t1 destination table
+-- @param t2 source table
+-- @return first table
+function tablex.update (t1,t2)
+ assert_arg_indexable(1,t1)
+ assert_arg_indexable(2,t2)
+ for k,v in pairs(t2) do
+ t1[k] = v
+ end
+ return t1
+end
+
+--- total number of elements in this table.
+-- Note that this is distinct from `#t`, which is the number
+-- of values in the array part; this value will always
+-- be greater or equal. The difference gives the size of
+-- the hash part, for practical purposes.
+-- @param t a table
+-- @return the size
+function tablex.size (t)
+ assert_arg_indexable(1,t)
+ local i = 0
+ for k in pairs(t) do i = i + 1 end
+ return i
+end
+
+--- make a shallow copy of a table
+-- @param t source table
+-- @return new table
+function tablex.copy (t)
+ assert_arg_indexable(1,t)
+ local res = {}
+ for k,v in pairs(t) do
+ res[k] = v
+ end
+ return res
+end
+
+--- make a deep copy of a table, recursively copying all the keys and fields.
+-- This will also set the copied table's metatable to that of the original.
+-- @param t A table
+-- @return new table
+function tablex.deepcopy(t)
+ assert_arg_indexable(1,t)
+ if type(t) ~= 'table' then return t end
+ local mt = getmetatable(t)
+ local res = {}
+ for k,v in pairs(t) do
+ if type(v) == 'table' then
+ v = tablex.deepcopy(v)
+ end
+ res[k] = v
+ end
+ setmetatable(res,mt)
+ return res
+end
+
+local abs = math.abs
+
+--- compare two values.
+-- if they are tables, then compare their keys and fields recursively.
+-- @param t1 A value
+-- @param t2 A value
+-- @param ignore_mt if true, ignore __eq metamethod (default false)
+-- @param eps if defined, then used for any number comparisons
+-- @return true or false
+function tablex.deepcompare(t1,t2,ignore_mt,eps)
+ local ty1 = type(t1)
+ local ty2 = type(t2)
+ if ty1 ~= ty2 then return false end
+ -- non-table types can be directly compared
+ if ty1 ~= 'table' then
+ if ty1 == 'number' and eps then return abs(t1-t2) < eps end
+ return t1 == t2
+ end
+ -- as well as tables which have the metamethod __eq
+ local mt = getmetatable(t1)
+ if not ignore_mt and mt and mt.__eq then return t1 == t2 end
+ for k1,v1 in pairs(t1) do
+ local v2 = t2[k1]
+ if v2 == nil or not tablex.deepcompare(v1,v2,ignore_mt,eps) then return false end
+ end
+ for k2,v2 in pairs(t2) do
+ local v1 = t1[k2]
+ if v1 == nil or not tablex.deepcompare(v1,v2,ignore_mt,eps) then return false end
+ end
+ return true
+end
+
+--- compare two arrays using a predicate.
+-- @param t1 an array
+-- @param t2 an array
+-- @param cmp A comparison function
+function tablex.compare (t1,t2,cmp)
+ assert_arg_indexable(1,t1)
+ assert_arg_indexable(2,t2)
+ if #t1 ~= #t2 then return false end
+ cmp = function_arg(3,cmp)
+ for k = 1,#t1 do
+ if not cmp(t1[k],t2[k]) then return false end
+ end
+ return true
+end
+
+--- compare two list-like tables using an optional predicate, without regard for element order.
+-- @param t1 a list-like table
+-- @param t2 a list-like table
+-- @param cmp A comparison function (may be nil)
+function tablex.compare_no_order (t1,t2,cmp)
+ assert_arg_indexable(1,t1)
+ assert_arg_indexable(2,t2)
+ if cmp then cmp = function_arg(3,cmp) end
+ if #t1 ~= #t2 then return false end
+ local visited = {}
+ for i = 1,#t1 do
+ local val = t1[i]
+ local gotcha
+ for j = 1,#t2 do if not visited[j] then
+ local match
+ if cmp then match = cmp(val,t2[j]) else match = val == t2[j] end
+ if match then
+ gotcha = j
+ break
+ end
+ end end
+ if not gotcha then return false end
+ visited[gotcha] = true
+ end
+ return true
+end
+
+
+--- return the index of a value in a list.
+-- Like string.find, there is an optional index to start searching,
+-- which can be negative.
+-- @param t A list-like table (i.e. with numerical indices)
+-- @param val A value
+-- @param idx index to start; -1 means last element,etc (default 1)
+-- @return index of value or nil if not found
+-- @usage find({10,20,30},20) == 2
+-- @usage find({'a','b','a','c'},'a',2) == 3
+function tablex.find(t,val,idx)
+ assert_arg_indexable(1,t)
+ idx = idx or 1
+ if idx < 0 then idx = #t + idx + 1 end
+ for i = idx,#t do
+ if t[i] == val then return i end
+ end
+ return nil
+end
+
+--- return the index of a value in a list, searching from the end.
+-- Like string.find, there is an optional index to start searching,
+-- which can be negative.
+-- @param t A list-like table (i.e. with numerical indices)
+-- @param val A value
+-- @param idx index to start; -1 means last element,etc (default 1)
+-- @return index of value or nil if not found
+-- @usage rfind({10,10,10},10) == 3
+function tablex.rfind(t,val,idx)
+ assert_arg_indexable(1,t)
+ idx = idx or #t
+ if idx < 0 then idx = #t + idx + 1 end
+ for i = idx,1,-1 do
+ if t[i] == val then return i end
+ end
+ return nil
+end
+
+
+--- return the index (or key) of a value in a table using a comparison function.
+-- @param t A table
+-- @param cmp A comparison function
+-- @param arg an optional second argument to the function
+-- @return index of value, or nil if not found
+-- @return value returned by comparison function
+function tablex.find_if(t,cmp,arg)
+ assert_arg_indexable(1,t)
+ cmp = function_arg(2,cmp)
+ for k,v in pairs(t) do
+ local c = cmp(v,arg)
+ if c then return k,c end
+ end
+ return nil
+end
+
+--- return a list of all values in a table indexed by another list.
+-- @param tbl a table
+-- @param idx an index table (a list of keys)
+-- @return a list-like table
+-- @usage index_by({10,20,30,40},{2,4}) == {20,40}
+-- @usage index_by({one=1,two=2,three=3},{'one','three'}) == {1,3}
+function tablex.index_by(tbl,idx)
+ assert_arg_indexable(1,tbl)
+ assert_arg_indexable(2,idx)
+ local res = {}
+ for i = 1,#idx do
+ res[i] = tbl[idx[i]]
+ end
+ return setmeta(res,tbl,List)
+end
+
+--- apply a function to all values of a table.
+-- This returns a table of the results.
+-- Any extra arguments are passed to the function.
+-- @param fun A function that takes at least one argument
+-- @param t A table
+-- @param ... optional arguments
+-- @usage map(function(v) return v*v end, {10,20,30,fred=2}) is {100,400,900,fred=4}
+function tablex.map(fun,t,...)
+ assert_arg_indexable(1,t)
+ fun = function_arg(1,fun)
+ local res = {}
+ for k,v in pairs(t) do
+ res[k] = fun(v,...)
+ end
+ return setmeta(res,t)
+end
+
+--- apply a function to all values of a list.
+-- This returns a table of the results.
+-- Any extra arguments are passed to the function.
+-- @param fun A function that takes at least one argument
+-- @param t a table (applies to array part)
+-- @param ... optional arguments
+-- @return a list-like table
+-- @usage imap(function(v) return v*v end, {10,20,30,fred=2}) is {100,400,900}
+function tablex.imap(fun,t,...)
+ assert_arg_indexable(1,t)
+ fun = function_arg(1,fun)
+ local res = {}
+ for i = 1,#t do
+ res[i] = fun(t[i],...) or false
+ end
+ return setmeta(res,t,List)
+end
+
+--- apply a named method to values from a table.
+-- @param name the method name
+-- @param t a list-like table
+-- @param ... any extra arguments to the method
+function tablex.map_named_method (name,t,...)
+ assert_arg_indexable(1,name,'string')
+ assert_arg_indexable(2,t)
+ local res = {}
+ for i = 1,#t do
+ local val = t[i]
+ local fun = val[name]
+ res[i] = fun(val,...)
+ end
+ return setmeta(res,t,List)
+end
+
+
+--- apply a function to all values of a table, in-place.
+-- Any extra arguments are passed to the function.
+-- @param fun A function that takes at least one argument
+-- @param t a table
+-- @param ... extra arguments
+function tablex.transform (fun,t,...)
+ assert_arg_indexable(1,t)
+ fun = function_arg(1,fun)
+ for k,v in pairs(t) do
+ t[v] = fun(v,...)
+ end
+end
+
+--- generate a table of all numbers in a range
+-- @param start number
+-- @param finish number
+-- @param step optional increment (default 1 for increasing, -1 for decreasing)
+function tablex.range (start,finish,step)
+ if start == finish then return {start}
+ elseif start > finish then return {}
+ end
+ local res = {}
+ local k = 1
+ if not step then
+ if finish > start then step = finish > start and 1 or -1 end
+ end
+ for i=start,finish,step do res[k]=i; k=k+1 end
+ return res
+end
+
+--- apply a function to values from two tables.
+-- @param fun a function of at least two arguments
+-- @param t1 a table
+-- @param t2 a table
+-- @param ... extra arguments
+-- @return a table
+-- @usage map2('+',{1,2,3,m=4},{10,20,30,m=40}) is {11,22,23,m=44}
+function tablex.map2 (fun,t1,t2,...)
+ assert_arg_indexable(1,t1)
+ assert_arg_indexable(2,t2)
+ fun = function_arg(1,fun)
+ local res = {}
+ for k,v in pairs(t1) do
+ res[k] = fun(v,t2[k],...)
+ end
+ return setmeta(res,t1,List)
+end
+
+--- apply a function to values from two arrays.
+-- @param fun a function of at least two arguments
+-- @param t1 a list-like table
+-- @param t2 a list-like table
+-- @param ... extra arguments
+-- @usage imap2('+',{1,2,3,m=4},{10,20,30,m=40}) is {11,22,23}
+function tablex.imap2 (fun,t1,t2,...)
+ assert_arg_indexable(2,t1)
+ assert_arg_indexable(3,t2)
+ fun = function_arg(1,fun)
+ local res = {}
+ for i = 1,#t1 do
+ res[i] = fun(t1[i],t2[i],...)
+ end
+ return res
+end
+
+--- 'reduce' a list using a binary function.
+-- @param fun a function of two arguments
+-- @param t a list-like table
+-- @return the result of the function
+-- @usage reduce('+',{1,2,3,4}) == 10
+function tablex.reduce (fun,t)
+ assert_arg_indexable(2,t)
+ fun = function_arg(1,fun)
+ local n = #t
+ local res = t[1]
+ for i = 2,n do
+ res = fun(res,t[i])
+ end
+ return res
+end
+
+--- apply a function to all elements of a table.
+-- The arguments to the function will be the value,
+-- the key and <i>finally</i> any extra arguments passed to this function.
+-- Note that the Lua 5.0 function table.foreach passed the <i>key</i> first.
+-- @param t a table
+-- @param fun a function with at least one argument
+-- @param ... extra arguments
+function tablex.foreach(t,fun,...)
+ assert_arg_indexable(1,t)
+ fun = function_arg(2,fun)
+ for k,v in pairs(t) do
+ fun(v,k,...)
+ end
+end
+
+--- apply a function to all elements of a list-like table in order.
+-- The arguments to the function will be the value,
+-- the index and <i>finally</i> any extra arguments passed to this function
+-- @param t a table
+-- @param fun a function with at least one argument
+-- @param ... optional arguments
+function tablex.foreachi(t,fun,...)
+ assert_arg_indexable(1,t)
+ fun = function_arg(2,fun)
+ for i = 1,#t do
+ fun(t[i],i,...)
+ end
+end
+
+
+--- Apply a function to a number of tables.
+-- A more general version of map
+-- The result is a table containing the result of applying that function to the
+-- ith value of each table. Length of output list is the minimum length of all the lists
+-- @param fun a function of n arguments
+-- @param ... n tables
+-- @usage mapn(function(x,y,z) return x+y+z end, {1,2,3},{10,20,30},{100,200,300}) is {111,222,333}
+-- @usage mapn(math.max, {1,20,300},{10,2,3},{100,200,100}) is {100,200,300}
+-- @param fun A function that takes as many arguments as there are tables
+function tablex.mapn(fun,...)
+ fun = function_arg(1,fun)
+ local res = {}
+ local lists = {...}
+ local minn = 1e40
+ for i = 1,#lists do
+ minn = min(minn,#(lists[i]))
+ end
+ for i = 1,minn do
+ local args = {}
+ for j = 1,#lists do
+ args[#args+1] = lists[j][i]
+ end
+ res[#res+1] = fun(unpack(args))
+ end
+ return res
+end
+
+--- call the function with the key and value pairs from a table.
+-- The function can return a value and a key (note the order!). If both
+-- are not nil, then this pair is inserted into the result. If only value is not nil, then
+-- it is appended to the result.
+-- @param fun A function which will be passed each key and value as arguments, plus any extra arguments to pairmap.
+-- @param t A table
+-- @param ... optional arguments
+-- @usage pairmap({fred=10,bonzo=20},function(k,v) return v end) is {10,20}
+-- @usage pairmap({one=1,two=2},function(k,v) return {k,v},k end) is {one={'one',1},two={'two',2}}
+function tablex.pairmap(fun,t,...)
+ assert_arg_indexable(1,t)
+ fun = function_arg(1,fun)
+ local res = {}
+ for k,v in pairs(t) do
+ local rv,rk = fun(k,v,...)
+ if rk then
+ res[rk] = rv
+ else
+ res[#res+1] = rv
+ end
+ end
+ return res
+end
+
+local function keys_op(i,v) return i end
+
+--- return all the keys of a table in arbitrary order.
+-- @param t A table
+function tablex.keys(t)
+ assert_arg_indexable(1,t)
+ return makelist(tablex.pairmap(keys_op,t))
+end
+
+local function values_op(i,v) return v end
+
+--- return all the values of the table in arbitrary order
+-- @param t A table
+function tablex.values(t)
+ assert_arg_indexable(1,t)
+ return makelist(tablex.pairmap(values_op,t))
+end
+
+local function index_map_op (i,v) return i,v end
+
+--- create an index map from a list-like table. The original values become keys,
+-- and the associated values are the indices into the original list.
+-- @param t a list-like table
+-- @return a map-like table
+function tablex.index_map (t)
+ assert_arg_indexable(1,t)
+ return setmetatable(tablex.pairmap(index_map_op,t),Map)
+end
+
+local function set_op(i,v) return true,v end
+
+--- create a set from a list-like table. A set is a table where the original values
+-- become keys, and the associated values are all true.
+-- @param t a list-like table
+-- @return a set (a map-like table)
+function tablex.makeset (t)
+ assert_arg_indexable(1,t)
+ return setmetatable(tablex.pairmap(set_op,t),Set)
+end
+
+
+--- combine two tables, either as union or intersection. Corresponds to
+-- set operations for sets () but more general. Not particularly
+-- useful for list-like tables.
+-- @param t1 a table
+-- @param t2 a table
+-- @param dup true for a union, false for an intersection.
+-- @usage merge({alice=23,fred=34},{bob=25,fred=34}) is {fred=34}
+-- @usage merge({alice=23,fred=34},{bob=25,fred=34},true) is {bob=25,fred=34,alice=23}
+-- @see tablex.index_map
+function tablex.merge (t1,t2,dup)
+ assert_arg_indexable(1,t1)
+ assert_arg_indexable(2,t2)
+ local res = {}
+ for k,v in pairs(t1) do
+ if dup or t2[k] then res[k] = v end
+ end
+ for k,v in pairs(t2) do
+ if dup or t1[k] then res[k] = v end
+ end
+ return setmeta(res,t1,Map)
+end
+
+--- a new table which is the difference of two tables.
+-- With sets (where the values are all true) this is set difference and
+-- symmetric difference depending on the third parameter.
+-- @param s1 a map-like table or set
+-- @param s2 a map-like table or set
+-- @param symm symmetric difference (default false)
+-- @return a map-like table or set
+function tablex.difference (s1,s2,symm)
+ assert_arg_indexable(1,s1)
+ assert_arg_indexable(2,s2)
+ local res = {}
+ for k,v in pairs(s1) do
+ if not s2[k] then res[k] = v end
+ end
+ if symm then
+ for k,v in pairs(s2) do
+ if not s1[k] then res[k] = v end
+ end
+ end
+ return setmeta(res,s1,Map)
+end
+
+--- A table where the key/values are the values and value counts of the table.
+-- @param t a list-like table
+-- @param cmp a function that defines equality (otherwise uses ==)
+-- @return a map-like table
+-- @see seq.count_map
+function tablex.count_map (t,cmp)
+ assert_arg_indexable(1,t)
+ local res,mask = {},{}
+ cmp = function_arg(2,cmp)
+ local n = #t
+ for i = 1,#t do
+ local v = t[i]
+ if not mask[v] then
+ mask[v] = true
+ -- check this value against all other values
+ res[v] = 1 -- there's at least one instance
+ for j = i+1,n do
+ local w = t[j]
+ if cmp and cmp(v,w) or v == w then
+ res[v] = res[v] + 1
+ mask[w] = true
+ end
+ end
+ end
+ end
+ return setmetatable(res,Map)
+end
+
+--- filter a table's values using a predicate function
+-- @param t a list-like table
+-- @param pred a boolean function
+-- @param arg optional argument to be passed as second argument of the predicate
+function tablex.filter (t,pred,arg)
+ assert_arg_indexable(1,t)
+ pred = function_arg(2,pred)
+ local res,k = {},1
+ for i = 1,#t do
+ local v = t[i]
+ if pred(v,arg) then
+ res[k] = v
+ k = k + 1
+ end
+ end
+ return setmeta(res,t,List)
+end
+
+--- return a table where each element is a table of the ith values of an arbitrary
+-- number of tables. It is equivalent to a matrix transpose.
+-- @usage zip({10,20,30},{100,200,300}) is {{10,100},{20,200},{30,300}}
+function tablex.zip(...)
+ return tablex.mapn(function(...) return {...} end,...)
+end
+
+local _copy
+function _copy (dest,src,idest,isrc,nsrc,clean_tail)
+ idest = idest or 1
+ isrc = isrc or 1
+ local iend
+ if not nsrc then
+ nsrc = #src
+ iend = #src
+ else
+ iend = isrc + min(nsrc-1,#src-isrc)
+ end
+ if dest == src then -- special case
+ if idest > isrc and iend >= idest then -- overlapping ranges
+ src = tablex.sub(src,isrc,nsrc)
+ isrc = 1; iend = #src
+ end
+ end
+ for i = isrc,iend do
+ dest[idest] = src[i]
+ idest = idest + 1
+ end
+ if clean_tail then
+ tablex.clear(dest,idest)
+ end
+ return dest
+end
+
+--- copy an array into another one, resizing the destination if necessary. <br>
+-- @param dest a list-like table
+-- @param src a list-like table
+-- @param idest where to start copying values from source (default 1)
+-- @param isrc where to start copying values into destination (default 1)
+-- @param nsrc number of elements to copy from source (default source size)
+function tablex.icopy (dest,src,idest,isrc,nsrc)
+ assert_arg_indexable(1,dest)
+ assert_arg_indexable(2,src)
+ return _copy(dest,src,idest,isrc,nsrc,true)
+end
+
+--- copy an array into another one. <br>
+-- @param dest a list-like table
+-- @param src a list-like table
+-- @param idest where to start copying values from source (default 1)
+-- @param isrc where to start copying values into destination (default 1)
+-- @param nsrc number of elements to copy from source (default source size)
+function tablex.move (dest,src,idest,isrc,nsrc)
+ assert_arg_indexable(1,dest)
+ assert_arg_indexable(2,src)
+ return _copy(dest,src,idest,isrc,nsrc,false)
+end
+
+function tablex._normalize_slice(self,first,last)
+ local sz = #self
+ if not first then first=1 end
+ if first<0 then first=sz+first+1 end
+ -- make the range _inclusive_!
+ if not last then last=sz end
+ if last < 0 then last=sz+1+last end
+ return first,last
+end
+
+--- Extract a range from a table, like 'string.sub'.
+-- If first or last are negative then they are relative to the end of the list
+-- eg. sub(t,-2) gives last 2 entries in a list, and
+-- sub(t,-4,-2) gives from -4th to -2nd
+-- @param t a list-like table
+-- @param first An index
+-- @param last An index
+-- @return a new List
+function tablex.sub(t,first,last)
+ assert_arg_indexable(1,t)
+ first,last = tablex._normalize_slice(t,first,last)
+ local res={}
+ for i=first,last do append(res,t[i]) end
+ return setmeta(res,t,List)
+end
+
+--- set an array range to a value. If it's a function we use the result
+-- of applying it to the indices.
+-- @param t a list-like table
+-- @param val a value
+-- @param i1 start range (default 1)
+-- @param i2 end range (default table size)
+function tablex.set (t,val,i1,i2)
+ i1,i2 = i1 or 1,i2 or #t
+ if utils.is_callable(val) then
+ for i = i1,i2 do
+ t[i] = val(i)
+ end
+ else
+ for i = i1,i2 do
+ t[i] = val
+ end
+ end
+end
+
+--- create a new array of specified size with initial value.
+-- @param n size
+-- @param val initial value (can be nil, but don't expect # to work!)
+-- @return the table
+function tablex.new (n,val)
+ local res = {}
+ tablex.set(res,val,1,n)
+ return res
+end
+
+--- clear out the contents of a table.
+-- @param t a table
+-- @param istart optional start position
+function tablex.clear(t,istart)
+ istart = istart or 1
+ for i = istart,#t do remove(t) end
+end
+
+--- insert values into a table. <br>
+-- insertvalues(t, [pos,] values) <br>
+-- similar to table.insert but inserts values from given table "values",
+-- not the object itself, into table "t" at position "pos".
+function tablex.insertvalues(t, ...)
+ local pos, values
+ if select('#', ...) == 1 then
+ pos,values = #t+1, ...
+ else
+ pos,values = ...
+ end
+ if #values > 0 then
+ for i=#t,pos,-1 do
+ t[i+#values] = t[i]
+ end
+ local offset = 1 - pos
+ for i=pos,pos+#values-1 do
+ t[i] = values[i + offset]
+ end
+ end
+ return t
+end
+
+--- remove a range of values from a table.
+-- @param t a list-like table
+-- @param i1 start index
+-- @param i2 end index
+-- @return the table
+function tablex.removevalues (t,i1,i2)
+ i1,i2 = tablex._normalize_slice(t,i1,i2)
+ for i = i1,i2 do
+ remove(t,i1)
+ end
+ return t
+end
+
+local _find
+_find = function (t,value,tables)
+ for k,v in pairs(t) do
+ if v == value then return k end
+ end
+ for k,v in pairs(t) do
+ if not tables[v] and type(v) == 'table' then
+ tables[v] = true
+ local res = _find(v,value,tables)
+ if res then
+ res = tostring(res)
+ if type(k) ~= 'string' then
+ return '['..k..']'..res
+ else
+ return k..'.'..res
+ end
+ end
+ end
+ end
+end
+
+--- find a value in a table by recursive search.
+-- @param t the table
+-- @param value the value
+-- @param exclude any tables to avoid searching
+-- @usage search(_G,math.sin,{package.path}) == 'math.sin'
+-- @return a fieldspec, e.g. 'a.b' or 'math.sin'
+function tablex.search (t,value,exclude)
+ assert_arg_indexable(1,t)
+ local tables = {[t]=true}
+ if exclude then
+ for _,v in pairs(exclude) do tables[v] = true end
+ end
+ return _find(t,value,tables)
+end
+
+return tablex
--- /dev/null
+--- A template preprocessor.
+-- Originally by [Ricki Lake](http://lua-users.org/wiki/SlightlyLessSimpleLuaPreprocessor)
+--
+-- There are two rules:
+--
+-- * lines starting with # are Lua
+-- * otherwise, `$(expr)` is the result of evaluating `expr`
+--
+-- Example:
+--
+-- # for i = 1,3 do
+-- $(i) Hello, Word!
+-- # end
+-- ===>
+-- 1 Hello, Word!
+-- 2 Hello, Word!
+-- 3 Hello, Word!
+--
+-- Other escape characters can be used, when the defaults conflict
+-- with the output language.
+--
+-- > for _,n in pairs{'one','two','three'} do
+-- static int l_${n} (luaState *state);
+-- > end
+--
+-- See @{03-strings.md.Another_Style_of_Template|the Guide}.
+--
+-- Dependencies: `pl.utils`
+-- @module pl.template
+
+local utils = require 'pl.utils'
+local append,format = table.insert,string.format
+
+local function parseHashLines(chunk,brackets,esc)
+ local exec_pat = "()$(%b"..brackets..")()"
+
+ local function parseDollarParen(pieces, chunk, s, e)
+ local s = 1
+ for term, executed, e in chunk:gmatch (exec_pat) do
+ executed = '('..executed:sub(2,-2)..')'
+ append(pieces,
+ format("%q..(%s or '')..",chunk:sub(s, term - 1), executed))
+ s = e
+ end
+ append(pieces, format("%q", chunk:sub(s)))
+ end
+
+ local esc_pat = esc.."+([^\n]*\n?)"
+ local esc_pat1, esc_pat2 = "^"..esc_pat, "\n"..esc_pat
+ local pieces, s = {"return function(_put) ", n = 1}, 1
+ while true do
+ local ss, e, lua = chunk:find (esc_pat1, s)
+ if not e then
+ ss, e, lua = chunk:find(esc_pat2, s)
+ append(pieces, "_put(")
+ parseDollarParen(pieces, chunk:sub(s, ss))
+ append(pieces, ")")
+ if not e then break end
+ end
+ append(pieces, lua)
+ s = e + 1
+ end
+ append(pieces, " end")
+ return table.concat(pieces)
+end
+
+local template = {}
+
+--- expand the template using the specified environment.
+-- @param str the template string
+-- @param env the environment (by default empty). <br>
+-- There are three special fields in the environment table <ul>
+-- <li><code>_parent</code> continue looking up in this table</li>
+-- <li><code>_brackets</code>; default is '()', can be any suitable bracket pair</li>
+-- <li><code>_escape</code>; default is '#' </li>
+-- </ul>
+function template.substitute(str,env)
+ env = env or {}
+ if rawget(env,"_parent") then
+ setmetatable(env,{__index = env._parent})
+ end
+ local brackets = rawget(env,"_brackets") or '()'
+ local escape = rawget(env,"_escape") or '#'
+ local code = parseHashLines(str,brackets,escape)
+ local fn,err = utils.load(code,'TMP','t',env)
+ if not fn then return nil,err end
+ fn = fn()
+ local out = {}
+ local res,err = xpcall(function() fn(function(s)
+ out[#out+1] = s
+ end) end,debug.traceback)
+ if not res then
+ if env._debug then print(code) end
+ return nil,err
+ end
+ return table.concat(out)
+end
+
+return template
+
+
+
+
--- /dev/null
+--- Useful test utilities.
+--
+-- test.asserteq({1,2},{1,2}) -- can compare tables
+-- test.asserteq(1.2,1.19,0.02) -- compare FP numbers within precision
+-- T = test.tuple -- used for comparing multiple results
+-- test.asserteq(T(string.find(" me","me")),T(2,3))
+--
+-- Dependencies: `pl.utils`, `pl.tablex`, `pl.pretty`, `pl.path`, `debug`
+-- @module pl.test
+
+local tablex = require 'pl.tablex'
+local utils = require 'pl.utils'
+local pretty = require 'pl.pretty'
+local path = require 'pl.path'
+local print,type = print,type
+local clock = os.clock
+local debug = require 'debug'
+local io,debug = io,debug
+
+local function dump(x)
+ if type(x) == 'table' and not (getmetatable(x) and getmetatable(x).__tostring) then
+ return pretty.write(x,' ',true)
+ else
+ return tostring(x)
+ end
+end
+
+local test = {}
+
+local function complain (x,y,msg)
+ local i = debug.getinfo(3)
+ local err = io.stderr
+ err:write(path.basename(i.short_src)..':'..i.currentline..': assertion failed\n')
+ err:write("got:\t",dump(x),'\n')
+ err:write("needed:\t",dump(y),'\n')
+ utils.quit(1,msg or "these values were not equal")
+end
+
+--- like assert, except takes two arguments that must be equal and can be tables.
+-- If they are plain tables, it will use tablex.deepcompare.
+-- @param x any value
+-- @param y a value equal to x
+-- @param eps an optional tolerance for numerical comparisons
+function test.asserteq (x,y,eps)
+ local res = x == y
+ if not res then
+ res = tablex.deepcompare(x,y,true,eps)
+ end
+ if not res then
+ complain(x,y)
+ end
+end
+
+--- assert that the first string matches the second.
+-- @param s1 a string
+-- @param s2 a string
+function test.assertmatch (s1,s2)
+ if not s1:match(s2) then
+ complain (s1,s2,"these strings did not match")
+ end
+end
+
+function test.assertraise(fn,e)
+ local ok, err = pcall(unpack(fn))
+ if not err or err:match(e)==nil then
+ complain (err,e,"these errors did not match")
+ end
+end
+
+--- a version of asserteq that takes two pairs of values.
+-- <code>x1==y1 and x2==y2</code> must be true. Useful for functions that naturally
+-- return two values.
+-- @param x1 any value
+-- @param x2 any value
+-- @param y1 any value
+-- @param y2 any value
+function test.asserteq2 (x1,x2,y1,y2)
+ if x1 ~= y1 then complain(x1,y1) end
+ if x2 ~= y2 then complain(x2,y2) end
+end
+
+-- tuple type --
+
+local tuple_mt = {}
+
+function tuple_mt.__tostring(self)
+ local ts = {}
+ for i=1, self.n do
+ local s = self[i]
+ ts[i] = type(s) == 'string' and string.format('%q', s) or tostring(s)
+ end
+ return 'tuple(' .. table.concat(ts, ', ') .. ')'
+end
+
+function tuple_mt.__eq(a, b)
+ if a.n ~= b.n then return false end
+ for i=1, a.n do
+ if a[i] ~= b[i] then return false end
+ end
+ return true
+end
+
+--- encode an arbitrary argument list as a tuple.
+-- This can be used to compare to other argument lists, which is
+-- very useful for testing functions which return a number of values.
+-- @usage asserteq(tuple( ('ab'):find 'a'), tuple(1,1))
+function test.tuple(...)
+ return setmetatable({n=select('#', ...), ...}, tuple_mt)
+end
+
+--- Time a function. Call the function a given number of times, and report the number of seconds taken,
+-- together with a message. Any extra arguments will be passed to the function.
+-- @param msg a descriptive message
+-- @param n number of times to call the function
+-- @param fun the function
+-- @param ... optional arguments to fun
+function test.timer(msg,n,fun,...)
+ local start = clock()
+ for i = 1,n do fun(...) end
+ utils.printf("%s: took %7.2f sec\n",msg,clock()-start)
+end
+
+return test
--- /dev/null
+--- Text processing utilities.
+--
+-- This provides a Template class (modeled after the same from the Python
+-- libraries, see string.Template). It also provides similar functions to those
+-- found in the textwrap module.
+--
+-- See @{03-strings.md.String_Templates|the Guide}.
+--
+-- Calling `text.format_operator()` overloads the % operator for strings to give Python/Ruby style formated output.
+-- This is extended to also do template-like substitution for map-like data.
+--
+-- > require 'pl.text'.format_operator()
+-- > = '%s = %5.3f' % {'PI',math.pi}
+-- PI = 3.142
+-- > = '$name = $value' % {name='dog',value='Pluto'}
+-- dog = Pluto
+--
+-- Dependencies: `pl.utils`
+-- @module pl.text
+
+local gsub = string.gsub
+local concat,append = table.concat,table.insert
+local utils = require 'pl.utils'
+local bind1,usplit,assert_arg,is_callable = utils.bind1,utils.split,utils.assert_arg,utils.is_callable
+
+local function lstrip(str) return (str:gsub('^%s+','')) end
+local function strip(str) return (lstrip(str):gsub('%s+$','')) end
+local function make_list(l) return setmetatable(l,utils.stdmt.List) end
+local function split(s,delim) return make_list(usplit(s,delim)) end
+
+local function imap(f,t,...)
+ local res = {}
+ for i = 1,#t do res[i] = f(t[i],...) end
+ return res
+end
+
+--[[
+module ('pl.text',utils._module)
+]]
+
+local text = {}
+
+local function _indent (s,sp)
+ local sl = split(s,'\n')
+ return concat(imap(bind1('..',sp),sl),'\n')..'\n'
+end
+
+--- indent a multiline string.
+-- @param s the string
+-- @param n the size of the indent
+-- @param ch the character to use when indenting (default ' ')
+-- @return indented string
+function text.indent (s,n,ch)
+ assert_arg(1,s,'string')
+ assert_arg(2,s,'number')
+ return _indent(s,string.rep(ch or ' ',n))
+end
+
+--- dedent a multiline string by removing any initial indent.
+-- useful when working with [[..]] strings.
+-- @param s the string
+-- @return a string with initial indent zero.
+function text.dedent (s)
+ assert_arg(1,s,'string')
+ local sl = split(s,'\n')
+ local i1,i2 = sl[1]:find('^%s*')
+ sl = imap(string.sub,sl,i2+1)
+ return concat(sl,'\n')..'\n'
+end
+
+--- format a paragraph into lines so that they fit into a line width.
+-- It will not break long words, so lines can be over the length
+-- to that extent.
+-- @param s the string
+-- @param width the margin width, default 70
+-- @return a list of lines
+function text.wrap (s,width)
+ assert_arg(1,s,'string')
+ width = width or 70
+ s = s:gsub('\n',' ')
+ local i,nxt = 1
+ local lines,line = {}
+ while i < #s do
+ nxt = i+width
+ if s:find("[%w']",nxt) then -- inside a word
+ nxt = s:find('%W',nxt+1) -- so find word boundary
+ end
+ line = s:sub(i,nxt)
+ i = i + #line
+ append(lines,strip(line))
+ end
+ return make_list(lines)
+end
+
+--- format a paragraph so that it fits into a line width.
+-- @param s the string
+-- @param width the margin width, default 70
+-- @return a string
+-- @see wrap
+function text.fill (s,width)
+ return concat(text.wrap(s,width),'\n') .. '\n'
+end
+
+local Template = {}
+text.Template = Template
+Template.__index = Template
+setmetatable(Template, {
+ __call = function(obj,tmpl)
+ return Template.new(tmpl)
+ end})
+
+function Template.new(tmpl)
+ assert_arg(1,tmpl,'string')
+ local res = {}
+ res.tmpl = tmpl
+ setmetatable(res,Template)
+ return res
+end
+
+local function _substitute(s,tbl,safe)
+ local subst
+ if is_callable(tbl) then
+ subst = tbl
+ else
+ function subst(f)
+ local s = tbl[f]
+ if not s then
+ if safe then
+ return f
+ else
+ error("not present in table "..f)
+ end
+ else
+ return s
+ end
+ end
+ end
+ local res = gsub(s,'%${([%w_]+)}',subst)
+ return (gsub(res,'%$([%w_]+)',subst))
+end
+
+--- substitute values into a template, throwing an error.
+-- This will throw an error if no name is found.
+-- @param tbl a table of name-value pairs.
+function Template:substitute(tbl)
+ assert_arg(1,tbl,'table')
+ return _substitute(self.tmpl,tbl,false)
+end
+
+--- substitute values into a template.
+-- This version just passes unknown names through.
+-- @param tbl a table of name-value pairs.
+function Template:safe_substitute(tbl)
+ assert_arg(1,tbl,'table')
+ return _substitute(self.tmpl,tbl,true)
+end
+
+--- substitute values into a template, preserving indentation. <br>
+-- If the value is a multiline string _or_ a template, it will insert
+-- the lines at the correct indentation. <br>
+-- Furthermore, if a template, then that template will be subsituted
+-- using the same table.
+-- @param tbl a table of name-value pairs.
+function Template:indent_substitute(tbl)
+ assert_arg(1,tbl,'table')
+ if not self.strings then
+ self.strings = split(self.tmpl,'\n')
+ end
+ -- the idea is to substitute line by line, grabbing any spaces as
+ -- well as the $var. If the value to be substituted contains newlines,
+ -- then we split that into lines and adjust the indent before inserting.
+ local function subst(line)
+ return line:gsub('(%s*)%$([%w_]+)',function(sp,f)
+ local subtmpl
+ local s = tbl[f]
+ if not s then error("not present in table "..f) end
+ if getmetatable(s) == Template then
+ subtmpl = s
+ s = s.tmpl
+ else
+ s = tostring(s)
+ end
+ if s:find '\n' then
+ s = _indent(s,sp)
+ end
+ if subtmpl then return _substitute(s,tbl)
+ else return s
+ end
+ end)
+ end
+ local lines = imap(subst,self.strings)
+ return concat(lines,'\n')..'\n'
+end
+
+------- Python-style formatting operator ------
+-- (see <a href="http://lua-users.org/wiki/StringInterpolation">the lua-users wiki</a>) --
+
+function text.format_operator()
+
+ local format = string.format
+
+ -- a more forgiving version of string.format, which applies
+ -- tostring() to any value with a %s format.
+ local function formatx (fmt,...)
+ local args = {...}
+ local i = 1
+ for p in fmt:gmatch('%%.') do
+ if p == '%s' and type(args[i]) ~= 'string' then
+ args[i] = tostring(args[i])
+ end
+ i = i + 1
+ end
+ return format(fmt,unpack(args))
+ end
+
+ local function basic_subst(s,t)
+ return (s:gsub('%$([%w_]+)',t))
+ end
+
+ -- Note this goes further than the original, and will allow these cases:
+ -- 1. a single value
+ -- 2. a list of values
+ -- 3. a map of var=value pairs
+ -- 4. a function, as in gsub
+ -- For the second two cases, it uses $-variable substituion.
+ getmetatable("").__mod = function(a, b)
+ if b == nil then
+ return a
+ elseif type(b) == "table" and getmetatable(b) == nil then
+ if #b == 0 then -- assume a map-like table
+ return _substitute(a,b,true)
+ else
+ return formatx(a,unpack(b))
+ end
+ elseif type(b) == 'function' then
+ return basic_subst(a,b)
+ else
+ return formatx(a,b)
+ end
+ end
+end
+
+return text
--- /dev/null
+--- Generally useful routines.
+-- See @{01-introduction.md.Generally_useful_functions|the Guide}.
+-- @module pl.utils
+local format,gsub,byte = string.format,string.gsub,string.byte
+local clock = os.clock
+local stdout = io.stdout
+local append = table.insert
+
+local collisions = {}
+
+local utils = {}
+
+utils._VERSION = "1.0.2"
+
+local lua51 = rawget(_G,'setfenv')
+
+utils.lua51 = lua51
+if not lua51 then -- Lua 5.2 compatibility
+ unpack = table.unpack
+ loadstring = load
+end
+
+utils.dir_separator = _G.package.config:sub(1,1)
+
+--- end this program gracefully.
+-- @param code The exit code or a message to be printed
+-- @param ... extra arguments for message's format'
+-- @see utils.fprintf
+function utils.quit(code,...)
+ if type(code) == 'string' then
+ utils.fprintf(io.stderr,code,...)
+ code = -1
+ else
+ utils.fprintf(io.stderr,...)
+ end
+ io.stderr:write('\n')
+ os.exit(code)
+end
+
+--- print an arbitrary number of arguments using a format.
+-- @param fmt The format (see string.format)
+-- @param ... Extra arguments for format
+function utils.printf(fmt,...)
+ utils.fprintf(stdout,fmt,...)
+end
+
+--- write an arbitrary number of arguments to a file using a format.
+-- @param f File handle to write to.
+-- @param fmt The format (see string.format).
+-- @param ... Extra arguments for format
+function utils.fprintf(f,fmt,...)
+ utils.assert_string(2,fmt)
+ f:write(format(fmt,...))
+end
+
+local function import_symbol(T,k,v,libname)
+ local key = rawget(T,k)
+ -- warn about collisions!
+ if key and k ~= '_M' and k ~= '_NAME' and k ~= '_PACKAGE' and k ~= '_VERSION' then
+ utils.printf("warning: '%s.%s' overrides existing symbol\n",libname,k)
+ end
+ rawset(T,k,v)
+end
+
+local function lookup_lib(T,t)
+ for k,v in pairs(T) do
+ if v == t then return k end
+ end
+ return '?'
+end
+
+local already_imported = {}
+
+--- take a table and 'inject' it into the local namespace.
+-- @param t The Table
+-- @param T An optional destination table (defaults to callers environment)
+function utils.import(t,T)
+ T = T or _G
+ t = t or utils
+ if type(t) == 'string' then
+ t = require (t)
+ end
+ local libname = lookup_lib(T,t)
+ if already_imported[t] then return end
+ already_imported[t] = libname
+ for k,v in pairs(t) do
+ import_symbol(T,k,v,libname)
+ end
+end
+
+utils.patterns = {
+ FLOAT = '[%+%-%d]%d*%.?%d*[eE]?[%+%-]?%d*',
+ INTEGER = '[+%-%d]%d*',
+ IDEN = '[%a_][%w_]*',
+ FILE = '[%a%.\\][:%][%w%._%-\\]*'
+}
+
+--- escape any 'magic' characters in a string
+-- @param s The input string
+function utils.escape(s)
+ utils.assert_string(1,s)
+ return (s:gsub('[%-%.%+%[%]%(%)%$%^%%%?%*]','%%%1'))
+end
+
+--- return either of two values, depending on a condition.
+-- @param cond A condition
+-- @param value1 Value returned if cond is true
+-- @param value2 Value returned if cond is false (can be optional)
+function utils.choose(cond,value1,value2)
+ if cond then return value1
+ else return value2
+ end
+end
+
+local raise
+
+--- return the contents of a file as a string
+-- @param filename The file path
+-- @param is_bin open in binary mode
+-- @return file contents
+function utils.readfile(filename,is_bin)
+ local mode = is_bin and 'b' or ''
+ utils.assert_string(1,filename)
+ local f,err = io.open(filename,'r'..mode)
+ if not f then return utils.raise (err) end
+ local res,err = f:read('*a')
+ f:close()
+ if not res then return raise (err) end
+ return res
+end
+
+--- write a string to a file
+-- @param filename The file path
+-- @param str The string
+-- @return true or nil
+-- @return error message
+-- @raise error if filename or str aren't strings
+function utils.writefile(filename,str)
+ utils.assert_string(1,filename)
+ utils.assert_string(2,str)
+ local f,err = io.open(filename,'w')
+ if not f then return raise(err) end
+ f:write(str)
+ f:close()
+ return true
+end
+
+--- return the contents of a file as a list of lines
+-- @param filename The file path
+-- @return file contents as a table
+-- @raise errror if filename is not a string
+function utils.readlines(filename)
+ utils.assert_string(1,filename)
+ local f,err = io.open(filename,'r')
+ if not f then return raise(err) end
+ local res = {}
+ for line in f:lines() do
+ append(res,line)
+ end
+ f:close()
+ return res
+end
+
+--- split a string into a list of strings separated by a delimiter.
+-- @param s The input string
+-- @param re A Lua string pattern; defaults to '%s+'
+-- @param plain don't use Lua patterns
+-- @param n optional maximum number of splits
+-- @return a list-like table
+-- @raise error if s is not a string
+function utils.split(s,re,plain,n)
+ utils.assert_string(1,s)
+ local find,sub,append = string.find, string.sub, table.insert
+ local i1,ls = 1,{}
+ if not re then re = '%s+' end
+ if re == '' then return {s} end
+ while true do
+ local i2,i3 = find(s,re,i1,plain)
+ if not i2 then
+ local last = sub(s,i1)
+ if last ~= '' then append(ls,last) end
+ if #ls == 1 and ls[1] == '' then
+ return {}
+ else
+ return ls
+ end
+ end
+ append(ls,sub(s,i1,i2-1))
+ if n and #ls == n then
+ ls[#ls] = sub(s,i1)
+ return ls
+ end
+ i1 = i3+1
+ end
+end
+
+--- split a string into a number of values.
+-- @param s the string
+-- @param re the delimiter, default space
+-- @return n values
+-- @usage first,next = splitv('jane:doe',':')
+-- @see split
+function utils.splitv (s,re)
+ return unpack(utils.split(s,re))
+end
+
+local lua51_load = load
+
+if utils.lua51 then -- define Lua 5.2 style load()
+ function utils.load(str,src,mode,env)
+ local chunk,err
+ if type(str) == 'string' then
+ chunk,err = loadstring(str,src)
+ else
+ chunk,err = lua51_load(str,src)
+ end
+ if chunk and env then setfenv(chunk,env) end
+ return chunk,err
+ end
+else
+ utils.load = load
+ -- setfenv/getfenv replacements for Lua 5.2
+ -- by Sergey Rozhenko
+ -- http://lua-users.org/lists/lua-l/2010-06/msg00313.html
+ -- Roberto Ierusalimschy notes that it is possible for getfenv to return nil
+ -- in the case of a function with no globals:
+ -- http://lua-users.org/lists/lua-l/2010-06/msg00315.html
+ function setfenv(f, t)
+ f = (type(f) == 'function' and f or debug.getinfo(f + 1, 'f').func)
+ local name
+ local up = 0
+ repeat
+ up = up + 1
+ name = debug.getupvalue(f, up)
+ until name == '_ENV' or name == nil
+ if name then
+ debug.upvaluejoin(f, up, function() return name end, 1) -- use unique upvalue
+ debug.setupvalue(f, up, t)
+ end
+ if f ~= 0 then return f end
+ end
+
+ function getfenv(f)
+ local f = f or 0
+ f = (type(f) == 'function' and f or debug.getinfo(f + 1, 'f').func)
+ local name, val
+ local up = 0
+ repeat
+ up = up + 1
+ name, val = debug.getupvalue(f, up)
+ until name == '_ENV' or name == nil
+ return val
+ end
+end
+
+
+--- execute a shell command.
+-- This is a compatibility function that returns the same for Lua 5.1 and Lua 5.2
+-- @param cmd a shell command
+-- @return true if successful
+-- @return actual return code
+function utils.execute (cmd)
+ local res1,res2,res2 = os.execute(cmd)
+ if lua51 then
+ return res1==0,res1
+ else
+ return res1,res2
+ end
+end
+
+if lua51 then
+ function table.pack (...)
+ local n = select('#',...)
+ return {n=n; ...}
+ end
+ local sep = package.config:sub(1,1)
+ function package.searchpath (mod,path)
+ mod = mod:gsub('%.',sep)
+ for m in path:gmatch('[^;]+') do
+ local nm = m:gsub('?',mod)
+ local f = io.open(nm,'r')
+ if f then f:close(); return nm end
+ end
+ end
+end
+
+if not table.pack then table.pack = _G.pack end
+if not rawget(_G,"pack") then _G.pack = table.pack end
+
+--- take an arbitrary set of arguments and make into a table.
+-- This returns the table and the size; works fine for nil arguments
+-- @param ... arguments
+-- @return table
+-- @return table size
+-- @usage local t,n = utils.args(...)
+
+--- 'memoize' a function (cache returned value for next call).
+-- This is useful if you have a function which is relatively expensive,
+-- but you don't know in advance what values will be required, so
+-- building a table upfront is wasteful/impossible.
+-- @param func a function of at least one argument
+-- @return a function with at least one argument, which is used as the key.
+function utils.memoize(func)
+ return setmetatable({}, {
+ __index = function(self, k, ...)
+ local v = func(k,...)
+ self[k] = v
+ return v
+ end,
+ __call = function(self, k) return self[k] end
+ })
+end
+
+--- is the object either a function or a callable object?.
+-- @param obj Object to check.
+function utils.is_callable (obj)
+ return type(obj) == 'function' or getmetatable(obj) and getmetatable(obj).__call
+end
+
+--- is the object of the specified type?.
+-- If the type is a string, then use type, otherwise compare with metatable
+-- @param obj An object to check
+-- @param tp String of what type it should be
+function utils.is_type (obj,tp)
+ if type(tp) == 'string' then return type(obj) == tp end
+ local mt = getmetatable(obj)
+ return tp == mt
+end
+
+local fileMT = getmetatable(io.stdout)
+
+--- a string representation of a type.
+-- For tables with metatables, we assume that the metatable has a `_name`
+-- field. Knows about Lua file objects.
+-- @param obj an object
+-- @return a string like 'number', 'table' or 'List'
+function utils.type (obj)
+ local t = type(obj)
+ if t == 'table' or t == 'userdata' then
+ local mt = getmetatable(obj)
+ if mt == fileMT then
+ return 'file'
+ else
+ return mt._name or "unknown "..t
+ end
+ else
+ return t
+ end
+end
+
+--- is this number an integer?
+-- @param x a number
+-- @raise error if x is not a number
+function utils.is_integer (x)
+ return math.ceil(x)==x
+end
+
+utils.stdmt = {
+ List = {_name='List'}, Map = {_name='Map'},
+ Set = {_name='Set'}, MultiMap = {_name='MultiMap'}
+}
+
+local _function_factories = {}
+
+--- associate a function factory with a type.
+-- A function factory takes an object of the given type and
+-- returns a function for evaluating it
+-- @param mt metatable
+-- @param fun a callable that returns a function
+function utils.add_function_factory (mt,fun)
+ _function_factories[mt] = fun
+end
+
+local function _string_lambda(f)
+ local raise = utils.raise
+ if f:find '^|' or f:find '_' then
+ local args,body = f:match '|([^|]*)|(.+)'
+ if f:find '_' then
+ args = '_'
+ body = f
+ else
+ if not args then return raise 'bad string lambda' end
+ end
+ local fstr = 'return function('..args..') return '..body..' end'
+ local fn,err = loadstring(fstr)
+ if not fn then return raise(err) end
+ fn = fn()
+ return fn
+ else return raise 'not a string lambda'
+ end
+end
+
+--- an anonymous function as a string. This string is either of the form
+-- '|args| expression' or is a function of one argument, '_'
+-- @param lf function as a string
+-- @return a function
+-- @usage string_lambda '|x|x+1' (2) == 3
+-- @usage string_lambda '_+1 (2) == 3
+-- @function utils.string_lambda
+utils.string_lambda = utils.memoize(_string_lambda)
+
+local ops
+
+--- process a function argument.
+-- This is used throughout Penlight and defines what is meant by a function:
+-- Something that is callable, or an operator string as defined by <code>pl.operator</code>,
+-- such as '>' or '#'. If a function factory has been registered for the type, it will
+-- be called to get the function.
+-- @param idx argument index
+-- @param f a function, operator string, or callable object
+-- @param msg optional error message
+-- @return a callable
+-- @raise if idx is not a number or if f is not callable
+-- @see utils.is_callable
+function utils.function_arg (idx,f,msg)
+ utils.assert_arg(1,idx,'number')
+ local tp = type(f)
+ if tp == 'function' then return f end -- no worries!
+ -- ok, a string can correspond to an operator (like '==')
+ if tp == 'string' then
+ if not ops then ops = require 'pl.operator'.optable end
+ local fn = ops[f]
+ if fn then return fn end
+ local fn, err = utils.string_lambda(f)
+ if not fn then error(err..': '..f) end
+ return fn
+ elseif tp == 'table' or tp == 'userdata' then
+ local mt = getmetatable(f)
+ if not mt then error('not a callable object',2) end
+ local ff = _function_factories[mt]
+ if not ff then
+ if not mt.__call then error('not a callable object',2) end
+ return f
+ else
+ return ff(f) -- we have a function factory for this type!
+ end
+ end
+ if not msg then msg = " must be callable" end
+ if idx > 0 then
+ error("argument "..idx..": "..msg,2)
+ else
+ error(msg,2)
+ end
+end
+
+--- bind the first argument of the function to a value.
+-- @param fn a function of at least two values (may be an operator string)
+-- @param p a value
+-- @return a function such that f(x) is fn(p,x)
+-- @raise same as @{function_arg}
+-- @see pl.func.curry
+function utils.bind1 (fn,p)
+ fn = utils.function_arg(1,fn)
+ return function(...) return fn(p,...) end
+end
+
+--- bind the second argument of the function to a value.
+-- @param fn a function of at least two values (may be an operator string)
+-- @param p a value
+-- @return a function such that f(x) is fn(x,p)
+-- @raise same as @{function_arg}
+function utils.bind2 (fn,p)
+ fn = utils.function_arg(1,fn)
+ return function(x,...) return fn(x,p,...) end
+end
+
+
+--- assert that the given argument is in fact of the correct type.
+-- @param n argument index
+-- @param val the value
+-- @param tp the type
+-- @param verify an optional verfication function
+-- @param msg an optional custom message
+-- @param lev optional stack position for trace, default 2
+-- @raise if the argument n is not the correct type
+-- @usage assert_arg(1,t,'table')
+-- @usage assert_arg(n,val,'string',path.isdir,'not a directory')
+function utils.assert_arg (n,val,tp,verify,msg,lev)
+ if type(val) ~= tp then
+ error(("argument %d expected a '%s', got a '%s'"):format(n,tp,type(val)),2)
+ end
+ if verify and not verify(val) then
+ error(("argument %d: '%s' %s"):format(n,val,msg),lev or 2)
+ end
+end
+
+--- assert the common case that the argument is a string.
+-- @param n argument index
+-- @param val a value that must be a string
+-- @raise val must be a string
+function utils.assert_string (n,val)
+ utils.assert_arg(n,val,'string',nil,nil,nil,3)
+end
+
+local err_mode = 'default'
+
+--- control the error strategy used by Penlight.
+-- Controls how <code>utils.raise</code> works; the default is for it
+-- to return nil and the error string, but if the mode is 'error' then
+-- it will throw an error. If mode is 'quit' it will immediately terminate
+-- the program.
+-- @param mode - either 'default', 'quit' or 'error'
+-- @see utils.raise
+function utils.on_error (mode)
+ err_mode = mode
+end
+
+--- used by Penlight functions to return errors. Its global behaviour is controlled
+-- by <code>utils.on_error</code>
+-- @param err the error string.
+-- @see utils.on_error
+function utils.raise (err)
+ if err_mode == 'default' then return nil,err
+ elseif err_mode == 'quit' then utils.quit(err)
+ else error(err,2)
+ end
+end
+
+raise = utils.raise
+
+--- load a code string or bytecode chunk.
+-- @param code Lua code as a string or bytecode
+-- @param name for source errors
+-- @param mode kind of chunk, 't' for text, 'b' for bytecode, 'bt' for all (default)
+-- @param env the environment for the new chunk (default nil)
+-- @return compiled chunk
+-- @return error message (chunk is nil)
+-- @function utils.load
+
+
+--- Lua 5.2 Compatible Functions
+-- @section lua52
+
+--- pack an argument list into a table.
+-- @param ... any arguments
+-- @return a table with field n set to the length
+-- @return the length
+-- @function table.pack
+
+------
+-- return the full path where a Lua module name would be matched.
+-- @param mod module name, possibly dotted
+-- @param path a path in the same form as package.path or package.cpath
+-- @see path.package_path
+-- @function package.searchpath
+
+return utils
+
+
--- /dev/null
+--- XML LOM Utilities.
+--
+-- This implements some useful things on [LOM](http://matthewwild.co.uk/projects/luaexpat/lom.html) documents, such as returned by `lxp.lom.parse`.
+-- In particular, it can convert LOM back into XML text, with optional pretty-printing control.
+-- It is s based on stanza.lua from [Prosody](http://hg.prosody.im/trunk/file/4621c92d2368/util/stanza.lua)
+--
+-- > d = xml.parse "<nodes><node id='1'>alice</node></nodes>"
+-- > = d
+-- <nodes><node id='1'>alice</node></nodes>
+-- > = xml.tostring(d,'',' ')
+-- <nodes>
+-- <node id='1'>alice</node>
+-- </nodes>
+--
+-- Can be used as a lightweight one-stop-shop for simple XML processing; a simple XML parser is included
+-- but the default is to use `lxp.lom` if it can be found.
+-- <pre>
+-- Prosody IM
+-- Copyright (C) 2008-2010 Matthew Wild
+-- Copyright (C) 2008-2010 Waqas Hussain--
+-- classic Lua XML parser by Roberto Ierusalimschy.
+-- modified to output LOM format.
+-- http://lua-users.org/wiki/LuaXml
+-- </pre>
+-- See @{06-data.md.XML|the Guide}
+--
+-- Dependencies: `pl.utils`
+--
+-- Soft Dependencies: `lxp.lom` (fallback is to use basic Lua parser)
+-- @module pl.xml
+
+local split = require 'pl.utils'.split
+local t_insert = table.insert;
+local t_concat = table.concat;
+local t_remove = table.remove;
+local s_format = string.format;
+local s_match = string.match;
+local tostring = tostring;
+local setmetatable = setmetatable;
+local getmetatable = getmetatable;
+local pairs = pairs;
+local ipairs = ipairs;
+local type = type;
+local next = next;
+local print = print;
+local unpack = unpack;
+local s_gsub = string.gsub;
+local s_char = string.char;
+local s_find = string.find;
+local os = os;
+local pcall,require,io = pcall,require,io
+
+local _M = {}
+local Doc = { __type = "doc" };
+Doc.__index = Doc;
+
+--- create a new document node.
+-- @param tag the tag name
+-- @param attr optional attributes (table of name-value pairs)
+function _M.new(tag, attr)
+ local doc = { tag = tag, attr = attr or {}, last_add = {}};
+ return setmetatable(doc, Doc);
+end
+
+--- parse an XML document. By default, this uses lxp.lom.parse, but
+-- falls back to basic_parse, or if use_basic is true
+-- @param text_or_file file or string representation
+-- @param is_file whether text_or_file is a file name or not
+-- @param use_basic do a basic parse
+-- @return a parsed LOM document with the document metatatables set
+-- @return nil, error the error can either be a file error or a parse error
+function _M.parse(text_or_file, is_file, use_basic)
+ local parser,status,lom
+ if use_basic then parser = _M.basic_parse
+ else
+ status,lom = pcall(require,'lxp.lom')
+ if not status then parser = _M.basic_parse else parser = lom.parse end
+ end
+ if is_file then
+ local f,err = io.open(text_or_file)
+ if not f then return nil,err end
+ text_or_file = f:read '*a'
+ f:close()
+ end
+ local doc,err = parser(text_or_file)
+ if not doc then return nil,err end
+ if lom then
+ _M.walk(doc,false,function(_,d)
+ setmetatable(d,Doc)
+ end)
+ end
+ return doc
+end
+
+---- convenient function to add a document node, This updates the last inserted position.
+-- @param tag a tag name
+-- @param attrs optional set of attributes (name-string pairs)
+function Doc:addtag(tag, attrs)
+ local s = _M.new(tag, attrs);
+ (self.last_add[#self.last_add] or self):add_direct_child(s);
+ t_insert(self.last_add, s);
+ return self;
+end
+
+--- convenient function to add a text node. This updates the last inserted position.
+-- @param text a string
+function Doc:text(text)
+ (self.last_add[#self.last_add] or self):add_direct_child(text);
+ return self;
+end
+
+---- go up one level in a document
+function Doc:up()
+ t_remove(self.last_add);
+ return self;
+end
+
+function Doc:reset()
+ local last_add = self.last_add;
+ for i = 1,#last_add do
+ last_add[i] = nil;
+ end
+ return self;
+end
+
+--- append a child to a document directly.
+-- @param child a child node (either text or a document)
+function Doc:add_direct_child(child)
+ t_insert(self, child);
+end
+
+--- append a child to a document at the last element added
+-- @param child a child node (either text or a document)
+function Doc:add_child(child)
+ (self.last_add[#self.last_add] or self):add_direct_child(child);
+ return self;
+end
+
+--accessing attributes: useful not to have to expose implementation (attr)
+--but also can allow attr to be nil in any future optimizations
+
+--- set attributes of a document node.
+-- @param t a table containing attribute/value pairs
+function Doc:set_attribs (t)
+ for k,v in pairs(t) do
+ self.attr[k] = v
+ end
+end
+
+--- set a single attribute of a document node.
+-- @param a attribute
+-- @param v its value
+function Doc:set_attrib(a,v)
+ self.attr[a] = v
+end
+
+--- access the attributes of a document node.
+function Doc:get_attribs()
+ return self.attr
+end
+
+--- function to create an element with a given tag name and a set of children.
+-- @param tag a tag name
+-- @param items either text or a table where the hash part is the attributes and the list part is the children.
+function _M.elem(tag,items)
+ local s = _M.new(tag)
+ if type(items) == 'string' then items = {items} end
+ if _M.is_tag(items) then
+ t_insert(s,items)
+ elseif type(items) == 'table' then
+ for k,v in pairs(items) do
+ if type(k) == 'string' then
+ s.attr[k] = v
+ t_insert(s.attr,k)
+ else
+ s[k] = v
+ end
+ end
+ end
+ return s
+end
+
+--- given a list of names, return a number of element constructors.
+-- @param list a list of names, or a comma-separated string.
+-- @usage local parent,children = doc.tags 'parent,children' <br>
+-- doc = parent {child 'one', child 'two'}
+function _M.tags(list)
+ local ctors = {}
+ local elem = _M.elem
+ if type(list) == 'string' then list = split(list,'%s*,%s*') end
+ for _,tag in ipairs(list) do
+ local ctor = function(items) return _M.elem(tag,items) end
+ t_insert(ctors,ctor)
+ end
+ return unpack(ctors)
+end
+
+local templ_cache = {}
+
+local function is_data(data)
+ return #data == 0 or type(data[1]) ~= 'table'
+end
+
+local function prepare_data(data)
+ -- a hack for ensuring that $1 maps to first element of data, etc.
+ -- Either this or could change the gsub call just below.
+ for i,v in ipairs(data) do
+ data[tostring(i)] = v
+ end
+end
+
+--- create a substituted copy of a document,
+-- @param templ may be a document or a string representation which will be parsed and cached
+-- @param data a table of name-value pairs or a list of such tables
+-- @return an XML document
+function Doc.subst(templ, data)
+ if type(data) ~= 'table' or not next(data) then return nil, "data must be a non-empty table" end
+ if is_data(data) then
+ prepare_data(data)
+ end
+ if type(templ) == 'string' then
+ if templ_cache[templ] then
+ templ = templ_cache[templ]
+ else
+ local str,err = templ
+ templ,err = _M.parse(str)
+ if not templ then return nil,err end
+ templ_cache[str] = templ
+ end
+ end
+ local function _subst(item)
+ return _M.clone(templ,function(s)
+ return s:gsub('%$(%w+)',item)
+ end)
+ end
+ if is_data(data) then return _subst(data) end
+ local list = {}
+ for _,item in ipairs(data) do
+ prepare_data(item)
+ t_insert(list,_subst(item))
+ end
+ if data.tag then
+ list = _M.elem(data.tag,list)
+ end
+ return list
+end
+
+
+--- get the first child with a given tag name.
+-- @param tag the tag name
+function Doc:child_with_name(tag)
+ for _, child in ipairs(self) do
+ if child.tag == tag then return child; end
+ end
+end
+
+local _children_with_name
+function _children_with_name(self,tag,list,recurse)
+ for _, child in ipairs(self) do if type(child) == 'table' then
+ if child.tag == tag then t_insert(list,child) end
+ if recurse then _children_with_name(child,tag,list,recurse) end
+ end end
+end
+
+--- get all elements in a document that have a given tag.
+-- @param tag a tag name
+-- @param dont_recurse optionally only return the immediate children with this tag name
+-- @return a list of elements
+function Doc:get_elements_with_name(tag,dont_recurse)
+ local res = {}
+ _children_with_name(self,tag,res,not dont_recurse)
+ return res
+end
+
+-- iterate over all children of a document node, including text nodes.
+function Doc:children()
+ local i = 0;
+ return function (a)
+ i = i + 1
+ return a[i];
+ end, self, i;
+end
+
+-- return the first child element of a node, if it exists.
+function Doc:first_childtag()
+ if #self == 0 then return end
+ for _,t in ipairs(self) do
+ if type(t) == 'table' then return t end
+ end
+end
+
+function Doc:matching_tags(tag, xmlns)
+ xmlns = xmlns or self.attr.xmlns;
+ local tags = self;
+ local start_i, max_i = 1, #tags;
+ return function ()
+ for i=start_i,max_i do
+ v = tags[i];
+ if (not tag or v.tag == tag)
+ and (not xmlns or xmlns == v.attr.xmlns) then
+ start_i = i+1;
+ return v;
+ end
+ end
+ end, tags, i;
+end
+
+--- iterate over all child elements of a document node.
+function Doc:childtags()
+ local i = 0;
+ return function (a)
+ local v
+ repeat
+ i = i + 1
+ v = self[i]
+ if v and type(v) == 'table' then return v; end
+ until not v
+ end, self[1], i;
+end
+
+--- visit child element of a node and call a function, possibility modifying the document.
+-- @param callback a function passed the node (text or element). If it returns nil, that node will be removed.
+-- If it returns a value, that will replace the current node.
+function Doc:maptags(callback)
+ local is_tag = _M.is_tag
+ local i = 1;
+ while i <= #self do
+ if is_tag(self[i]) then
+ local ret = callback(self[i]);
+ if ret == nil then
+ t_remove(self, i);
+ else
+ self[i] = ret;
+ i = i + 1;
+ end
+ end
+ end
+ return self;
+end
+
+local xml_escape
+do
+ local escape_table = { ["'"] = "'", ["\""] = """, ["<"] = "<", [">"] = ">", ["&"] = "&" };
+ function xml_escape(str) return (s_gsub(str, "['&<>\"]", escape_table)); end
+ _M.xml_escape = xml_escape;
+end
+
+-- pretty printing
+-- if indent, then put each new tag on its own line
+-- if attr_indent, put each new attribute on its own line
+local function _dostring(t, buf, self, xml_escape, parentns, idn, indent, attr_indent)
+ local nsid = 0;
+ local tag = t.tag
+ local lf,alf = ""," "
+ if indent then lf = '\n'..idn end
+ if attr_indent then alf = '\n'..idn..attr_indent end
+ t_insert(buf, lf.."<"..tag);
+ for k, v in pairs(t.attr) do
+ if type(k) ~= 'number' then -- LOM attr table has list-like part
+ if s_find(k, "\1", 1, true) then
+ local ns, attrk = s_match(k, "^([^\1]*)\1?(.*)$");
+ nsid = nsid + 1;
+ t_insert(buf, " xmlns:ns"..nsid.."='"..xml_escape(ns).."' ".."ns"..nsid..":"..attrk.."='"..xml_escape(v).."'");
+ elseif not(k == "xmlns" and v == parentns) then
+ t_insert(buf, alf..k.."='"..xml_escape(v).."'");
+ end
+ end
+ end
+ local len,has_children = #t;
+ if len == 0 then
+ local out = "/>"
+ if attr_indent then out = '\n'..idn..out end
+ t_insert(buf, out);
+ else
+ t_insert(buf, ">");
+ for n=1,len do
+ local child = t[n];
+ if child.tag then
+ self(child, buf, self, xml_escape, t.attr.xmlns,idn and idn..indent, indent, attr_indent );
+ has_children = true
+ else -- text element
+ t_insert(buf, xml_escape(child));
+ end
+ end
+ t_insert(buf, (has_children and lf or '').."</"..tag..">");
+ end
+end
+
+---- pretty-print an XML document
+--- @param t an XML document
+--- @param idn an initial indent (indents are all strings)
+--- @param indent an indent for each level
+--- @param attr_indent if given, indent each attribute pair and put on a separate line
+--- @return a string representation
+function _M.tostring(t,idn,indent, attr_indent)
+ local buf = {};
+ _dostring(t, buf, _dostring, xml_escape, nil,idn,indent, attr_indent);
+ return t_concat(buf);
+end
+
+Doc.__tostring = _M.tostring
+
+--- get the full text value of an element
+function Doc:get_text()
+ local res = {}
+ for i,el in ipairs(self) do
+ if type(el) == 'string' then t_insert(res,el) end
+ end
+ return t_concat(res);
+end
+
+--- make a copy of a document
+-- @param doc the original document
+-- @param strsubst an optional function for handling string copying which could do substitution, etc.
+function _M.clone(doc, strsubst)
+ local lookup_table = {};
+ local function _copy(object)
+ if type(object) ~= "table" then
+ if strsubst and type(object) == 'string' then return strsubst(object)
+ else return object;
+ end
+ elseif lookup_table[object] then
+ return lookup_table[object];
+ end
+ local new_table = {};
+ lookup_table[object] = new_table;
+ for index, value in pairs(object) do
+ new_table[_copy(index)] = _copy(value); -- is cloning keys much use, hm?
+ end
+ return setmetatable(new_table, getmetatable(object));
+ end
+
+ return _copy(doc)
+end
+
+--- compare two documents.
+-- @param t1 any value
+-- @param t2 any value
+function _M.compare(t1,t2)
+ local ty1 = type(t1)
+ local ty2 = type(t2)
+ if ty1 ~= ty2 then return false, 'type mismatch' end
+ if ty1 == 'string' then
+ return t1 == t2 and true or 'text '..t1..' ~= text '..t2
+ end
+ if ty1 ~= 'table' or ty2 ~= 'table' then return false, 'not a document' end
+ if t1.tag ~= t2.tag then return false, 'tag '..t1.tag..' ~= tag '..t2.tag end
+ if #t1 ~= #t2 then return false, 'size '..#t1..' ~= size '..#t2..' for tag '..t1.tag end
+ -- compare attributes
+ for k,v in pairs(t1.attr) do
+ if t2.attr[k] ~= v then return false, 'mismatch attrib' end
+ end
+ for k,v in pairs(t2.attr) do
+ if t1.attr[k] ~= v then return false, 'mismatch attrib' end
+ end
+ -- compare children
+ for i = 1,#t1 do
+ local yes,err = _M.compare(t1[i],t2[i])
+ if not yes then return err end
+ end
+ return true
+end
+
+--- is this value a document element?
+-- @param d any value
+function _M.is_tag(d)
+ return type(d) == 'table' and type(d.tag) == 'string'
+end
+
+--- call the desired function recursively over the document.
+-- @param doc the document
+-- @param depth_first visit child notes first, then the current node
+-- @param operation a function which will receive the current tag name and current node.
+function _M.walk (doc, depth_first, operation)
+ if not depth_first then operation(doc.tag,doc) end
+ for _,d in ipairs(doc) do
+ if _M.is_tag(d) then
+ _M.walk(d,depth_first,operation)
+ end
+ end
+ if depth_first then operation(doc.tag,doc) end
+end
+
+local escapes = { quot = "\"", apos = "'", lt = "<", gt = ">", amp = "&" }
+local function unescape(str) return (str:gsub( "&(%a+);", escapes)); end
+
+local function parseargs(s)
+ local arg = {}
+ s:gsub("([%w:]+)%s*=%s*([\"'])(.-)%2", function (w, _, a)
+ arg[w] = unescape(a)
+ end)
+ return arg
+end
+
+--- Parse a simple XML document using a pure Lua parser based on Robero Ierusalimschy's original version.
+-- @param s the XML document to be parsed.
+-- @param all_text if true, preserves all whitespace. Otherwise only text containing non-whitespace is included.
+function _M.basic_parse(s,all_text)
+ local t_insert,t_remove = table.insert,table.remove
+ local s_find,s_sub = string.find,string.sub
+ local stack = {}
+ local top = {}
+ t_insert(stack, top)
+ local ni,c,label,xarg, empty
+ local i, j = 1, 1
+ -- we're not interested in <?xml version="1.0"?>
+ local _,istart = s_find(s,'^%s*<%?[^%?]+%?>%s*')
+ if istart then i = istart+1 end
+ while true do
+ ni,j,c,label,xarg, empty = s_find(s, "<(%/?)([%w:%-_]+)(.-)(%/?)>", i)
+ if not ni then break end
+ local text = s_sub(s, i, ni-1)
+ if all_text or not s_find(text, "^%s*$") then
+ t_insert(top, unescape(text))
+ end
+ if empty == "/" then -- empty element tag
+ t_insert(top, setmetatable({tag=label, attr=parseargs(xarg), empty=1},Doc))
+ elseif c == "" then -- start tag
+ top = setmetatable({tag=label, attr=parseargs(xarg)},Doc)
+ t_insert(stack, top) -- new level
+ else -- end tag
+ local toclose = t_remove(stack) -- remove top
+ top = stack[#stack]
+ if #stack < 1 then
+ error("nothing to close with "..label)
+ end
+ if toclose.tag ~= label then
+ error("trying to close "..toclose.tag.." with "..label)
+ end
+ t_insert(top, toclose)
+ end
+ i = j+1
+ end
+ local text = s_sub(s, i)
+ if all_text or not s_find(text, "^%s*$") then
+ t_insert(stack[#stack], unescape(text))
+ end
+ if #stack > 1 then
+ error("unclosed "..stack[#stack].tag)
+ end
+ local res = stack[1]
+ return type(res[1])=='string' and res[2] or res[1]
+end
+
+local function empty(attr) return not attr or not next(attr) end
+local function is_text(s) return type(s) == 'string' end
+local function is_element(d) return type(d) == 'table' and d.tag ~= nil end
+
+-- returns the key,value pair from a table if it has exactly one entry
+local function has_one_element(t)
+ local key,value = next(t)
+ if next(t,key) ~= nil then return false end
+ return key,value
+end
+
+local function append_capture(res,tbl)
+ if not empty(tbl) then -- no point in capturing empty tables...
+ local key
+ if tbl._ then -- if $_ was set then it is meant as the top-level key for the captured table
+ key = tbl._
+ tbl._ = nil
+ if empty(tbl) then return end
+ end
+ -- a table with only one pair {[0]=value} shall be reduced to that value
+ local numkey,val = has_one_element(tbl)
+ if numkey == 0 then tbl = val end
+ if key then
+ res[key] = tbl
+ else -- otherwise, we append the captured table
+ t_insert(res,tbl)
+ end
+ end
+end
+
+local function make_number(pat)
+ if pat:find '^%d+$' then -- $1 etc means use this as an array location
+ pat = tonumber(pat)
+ end
+ return pat
+end
+
+local function capture_attrib(res,pat,value)
+ pat = make_number(pat:sub(2))
+ res[pat] = value
+ return true
+end
+
+local match
+function match(d,pat,res,keep_going)
+ local ret = true
+ if d == nil then return false end
+ -- attribute string matching is straight equality, except if the pattern is a $ capture,
+ -- which always succeeds.
+ if type(d) == 'string' then
+ if type(pat) ~= 'string' then return false end
+ if _M.debug then print(d,pat) end
+ if pat:find '^%$' then
+ return capture_attrib(res,pat,d)
+ else
+ return d == pat
+ end
+ else
+ if _M.debug then print(d.tag,pat.tag) end
+ -- this is an element node. For a match to succeed, the attributes must
+ -- match as well.
+ -- a tagname in the pattern ending with '-' is a wildcard and matches like an attribute
+ local tagpat = pat.tag:match '^(.-)%-$'
+ if tagpat then
+ tagpat = make_number(tagpat)
+ res[tagpat] = d.tag
+ end
+ if d.tag == pat.tag or tagpat then
+
+ if not empty(pat.attr) then
+ if empty(d.attr) then ret = false
+ else
+ for prop,pval in pairs(pat.attr) do
+ local dval = d.attr[prop]
+ if not match(dval,pval,res) then ret = false; break end
+ end
+ end
+ end
+ -- the pattern may have child nodes. We match partially, so that {P1,P2} shall match {X,P1,X,X,P2,..}
+ if ret and #pat > 0 then
+ local i,j = 1,1
+ local function next_elem()
+ j = j + 1 -- next child element of data
+ if is_text(d[j]) then j = j + 1 end
+ return j <= #d
+ end
+ repeat
+ local p = pat[i]
+ -- repeated {{<...>}} patterns shall match one or more elements
+ -- so e.g. {P+} will match {X,X,P,P,X,P,X,X,X}
+ if is_element(p) and p.repeated then
+ local found
+ repeat
+ local tbl = {}
+ ret = match(d[j],p,tbl,false)
+ if ret then
+ found = false --true
+ append_capture(res,tbl)
+ end
+ until not next_elem() or (found and not ret)
+ i = i + 1
+ else
+ ret = match(d[j],p,res,false)
+ if ret then i = i + 1 end
+ end
+ until not next_elem() or i > #pat -- run out of elements or patterns to match
+ -- if every element in our pattern matched ok, then it's been a successful match
+ if i > #pat then return true end
+ end
+ if ret then return true end
+ else
+ ret = false
+ end
+ -- keep going anyway - look at the children!
+ if keep_going then
+ for child in d:childtags() do
+ ret = match(child,pat,res,keep_going)
+ if ret then break end
+ end
+ end
+ end
+ return ret
+end
+
+function Doc:match(pat)
+ if is_text(pat) then
+ pat = _M.parse(pat,false,true)
+ end
+ _M.walk(pat,false,function(_,d)
+ if is_text(d[1]) and is_element(d[2]) and is_text(d[3]) and
+ d[1]:find '%s*{{' and d[3]:find '}}%s*' then
+ t_remove(d,1)
+ t_remove(d,2)
+ d[1].repeated = true
+ end
+ end)
+
+ local res = {}
+ local ret = match(self,pat,res,true)
+ return res,ret
+end
+
+
+return _M
+
--- /dev/null
+-- running the tests and examples
+require 'pl'
+local lfs = require 'lfs'
+
+local function quote_if_needed (s)
+ if s:match '%s' then
+ s = '"'..s..'"'
+ end
+ return s
+end
+
+-- get the Lua command-line used to invoke this script
+local cmd = app.lua()
+
+function do_lua_files ()
+ for _,f in ipairs(dir.getfiles('.','*.lua')) do
+ print(cmd..' '..f)
+ local res = utils.execute(cmd..' '..f)
+ if not res then
+ print ('process failed with non-zero result: '..f)
+ os.exit(1)
+ end
+ end
+end
+
+if #arg == 0 then arg[1] = 'tests'; arg[2] = 'examples' end
+
+for _,dir in ipairs(arg) do
+ print('directory',dir)
+ lfs.chdir(dir)
+ do_lua_files()
+ lfs.chdir('..')
+end
+
--- /dev/null
+--- test module for demonstrating app.require_here()\r
+local bar = {}\r
+\r
+function bar.name ()\r
+ return 'bar'\r
+end\r
+\r
+return bar\r
+\r
+\r
--- /dev/null
+--- test module for demonstrating app.require_here()\r
+local args = {}\r
+\r
+function args.answer ()\r
+ return 42\r
+end\r
+\r
+return args\r
+\r
--- /dev/null
+-- testing app.parse_args\r
+asserteq = require 'pl.test'.asserteq\r
+app = require 'pl.app'\r
+path = require 'pl.path'\r
+parse_args = app.parse_args\r
+\r
+-- shows the use of plain flags, long and short:\r
+flags,args = parse_args({'-abc','--flag','-v','one'})\r
+\r
+asserteq(flags,{a=true,b=true,c=true,flag=true,v=true})\r
+asserteq(args,{'one'})\r
+\r
+-- flags may be given values using these three syntaxes:\r
+flags,args = parse_args({'-n10','--out=20','-v:2'})\r
+\r
+asserteq(flags,{n='10',out='20',v='2'})\r
+\r
+-- a flag can be specified as taking a value:\r
+flags,args = parse_args({'-k','-b=23','-o','hello','--out'},{o=true})\r
+\r
+asserteq(flags,{out=true,o="hello",k=true,b="23"})\r
+\r
+-- modify this script's module path so it looks in the 'lua' subdirectory\r
+-- for its modules\r
+app.require_here 'lua'\r
+\r
+asserteq(require 'foo.args'.answer(),42)\r
+asserteq(require 'bar'.name(),'bar')\r
+\r
+asserteq(\r
+ app.appfile 'config',\r
+ path.expanduser('~/.test-args/config'):gsub('/',path.sep)\r
+)\r
+\r
+\r
+\r
+\r
--- /dev/null
+local array = require 'pl.array2d'
+local asserteq = require('pl.test').asserteq
+local L = require 'pl.utils'. string_lambda
+
+local A = {
+ {1,2,3,4},
+ {10,20,30,40},
+ {100,200,300,400},
+ {1000,2000,3000,4000},
+}
+
+asserteq(array.column(A,2),{2,20,200,2000})
+asserteq(array.reduce_rows('+',A),{10,100,1000,10000})
+asserteq(array.reduce_cols('+',A),{1111,2222,3333,4444})
+
+--array.write(A)
+
+local dump = require 'pl.pretty'.dump
+
+asserteq(array.range(A,'A1:B1'),{1,2})
+
+asserteq(array.range(A,'A1:B2'),{{1,2},{10,20}})
+
+asserteq(
+ array.product('..',{1,2,3},{'a','b','c'}),
+ {{'1a','2a','3a'},{'1b','2b','3b'},{'1c','2c','3c'}}
+)
+
+asserteq(
+ array.product('{}',{1,2},{'a','b','c'}),
+ {{{1,'a'},{2,'a'}},{{1,'b'},{2,'b'}},{{1,'c'},{2,'c'}}}
+)
+
+asserteq(
+ array.flatten {{1,2},{3,4},{5,6}},
+ {1,2,3,4,5,6}
+)
+
+
+A = {{1,2,3},{4,5,6}}
+
+-- flatten in column order!
+asserteq(
+ array.reshape(A,1,true),
+ {{1,4,2,5,3,6}}
+)
+
+-- regular row-order reshape
+asserteq(
+ array.reshape(A,3),
+ {{1,2},{3,4},{5,6}}
+)
+
+asserteq(
+ array.new(3,3,0),
+ {{0,0,0},{0,0,0},{0,0,0}}
+)
+
+asserteq(
+ array.new(3,3,L'|i,j| i==j and 1 or 0'),
+ {{1,0,0},{0,1,0},{0,0,1}}
+)
+
+asserteq(
+ array.reduce2('+','*',{{1,10},{2,10},{3,10}}),
+ 60 -- i.e. 1*10 + 2*10 + 3*10
+)
+
+A = array.new(4,4,0)
+B = array.new(3,3,1)
+array.move(A,2,2,B)
+asserteq(A,{{0,0,0,0},{0,1,1,1},{0,1,1,1},{0,1,1,1}})
+
+
+
+
+
--- /dev/null
+require 'pl'
+asserteq = test.asserteq
+T = test.tuple
+
+A = class()
+
+function A:_init ()
+ self.a = 1
+end
+
+-- calling base class' ctor automatically
+A1 = class(A)
+
+asserteq(A1(),{a=1})
+
+-- explicitly calling base ctor with super
+
+B = class(A)
+
+function B:_init ()
+ self:super()
+ self.b = 2
+end
+
+asserteq(B(),{a=1,b=2})
+
+-- can continue this chain
+
+C = class(B)
+
+function C:_init ()
+ self:super()
+ self.c = 3
+end
+
+--- metamethods!
+
+function C:__tostring ()
+ return ("%d:%d:%d"):format(self.a,self.b,self.c)
+end
+
+function C.__eq (c1,c2)
+ return c1.a == c2.a and c1.b == c2.b and c1.c == c2.c
+end
+
+asserteq(C(),{a=1,b=2,c=3})
+
+asserteq(tostring(C()),"1:2:3")
+
+asserteq(C()==C(),true)
+
+----- properties -----
+
+local MyProps = class(class.properties)
+local setted_a, got_b
+
+function MyProps:_init ()
+ self._a = 1
+ self._b = 2
+end
+
+function MyProps:set_a (v)
+ setted_a = true
+ self._a = v
+end
+
+function MyProps:get_b ()
+ got_b = true
+ return self._b
+end
+
+function MyProps:set (a,b)
+ self._a = a
+ self._b = b
+end
+
+local mp = MyProps()
+
+mp.a = 10
+
+asserteq(mp.a,10)
+asserteq(mp.b,2)
+asserteq(setted_a and got_b, true)
+
+class.MoreProps(MyProps)
+local setted_c
+
+function MoreProps:_init()
+ self:super()
+ self._c = 3
+end
+
+function MoreProps:set_c (c)
+ setted_c = true
+ self._c = c
+end
+
+mm = MoreProps()
+
+mm:set(10,20)
+mm.c = 30
+
+asserteq(setted_c, true)
+asserteq(T(mm.a, mm.b, mm.c),T(10,20,30))
+
+
+
+
+
--- /dev/null
+-- test-compare-no-order.lua\r
+\r
+local T = require 'pl.tablex'\r
+local P = require 'pl.permute'\r
+\r
+local t = {10,20,5,5,10,'one',555}\r
+\r
+local permutations = P.table(t)\r
+print('permutations',#permutations)\r
+for _,p in ipairs(permutations) do\r
+ if not T.compare_no_order(t,p) then return print 'different!' end\r
+end\r
+print 'DONE'\r
--- /dev/null
+-- test-comprehension.lua\r
+-- test of comprehension.lua\r
+local comp = require 'pl.comprehension' . new()\r
+local asserteq = require 'pl.test' . asserteq\r
+\r
+-- test of list building\r
+asserteq(comp 'x for x' {}, {})\r
+asserteq(comp 'x for x' {2,3}, {2,3})\r
+asserteq(comp 'x^2 for x' {2,3}, {2^2,3^2})\r
+asserteq(comp 'x for x if x % 2 == 0' {4,5,6,7}, {4,6})\r
+asserteq(comp '{x,y} for x for y if x>2 if y>4' ({2,3},{4,5}), {{3,5}})\r
+\r
+-- test of table building\r
+local t = comp 'table(x,x+1 for x)' {3,4}\r
+assert(t[3] == 3+1 and t[4] == 4+1)\r
+local t = comp 'table(x,x+y for x for y)' ({3,4}, {2})\r
+assert(t[3] == 3+2 and t[4] == 4+2)\r
+local t = comp 'table(v,k for k,v in pairs(_1))' {[3]=5, [5]=7}\r
+assert(t[5] == 3 and t[7] == 5)\r
+\r
+-- test of sum\r
+assert(comp 'sum(x for x)' {} == 0)\r
+assert(comp 'sum(x for x)' {2,3} == 2+3)\r
+assert(comp 'sum(x^2 for x)' {2,3} == 2^2+3^2)\r
+assert(comp 'sum(x*y for x for y)' ({2,3}, {4,5}) == 2*4+2*5+3*4+3*5)\r
+assert(comp 'sum(x^2 for x if x % 2 == 0)' {4,5,6,7} == 4^2+6^2)\r
+assert(comp 'sum(x*y for x for y if x>2 if y>4)' ({2,3}, {4,5}) == 3*5)\r
+\r
+-- test of min/max\r
+assert(comp 'min(x for x)' {3,5,2,4} == 2)\r
+assert(comp 'max(x for x)' {3,5,2,4} == 5)\r
+\r
+-- test of placeholder parameters --\r
+assert(comp 'sum(x^_1 + _3 for x if x >= _4)' (2, nil, 3, 4, {3,4,5})\r
+ == 4^2+3 + 5^2+3)\r
+\r
+-- test of for =\r
+assert(comp 'sum(x^2 for x=2,3)' () == 2^2+3^2)\r
+assert(comp 'sum(x^2 for x=2,6,1+1)' () == 2^2+4^2+6^2)\r
+assert(comp 'sum(x*y*z for x=1,2 for y=3,3 for z)' {5,6} ==\r
+ 1*3*5 + 2*3*5 + 1*3*6 + 2*3*6)\r
+assert(comp 'sum(x*y*z for z for x=1,2 for y=3,3)' {5,6} ==\r
+ 1*3*5 + 2*3*5 + 1*3*6 + 2*3*6)\r
+\r
+-- test of for in\r
+assert(comp 'sum(i*v for i,v in ipairs(_1))' {2,3} == 1*2+2*3)\r
+assert(comp 'sum(i*v for i,v in _1,_2,_3)' (ipairs{2,3}) == 1*2+2*3)\r
+\r
+-- test of difficult syntax\r
+asserteq(comp '" x for x " for x' {2}, {' x for x '})\r
+asserteq(comp 'x --[=[for x\n\n]=] for x' {2}, {2})\r
+asserteq(comp '(function() for i = 1,1 do return x*2 end end)() for x'\r
+ {2}, {4})\r
+assert(comp 'sum(("_5" and x)^_1 --[[_6]] for x)' (2, {4,5}) == 4^2 + 5^2)\r
+\r
+-- error checking\r
+assert(({pcall(function() comp 'x for __result' end)})[2]\r
+ :find'not contain __ prefix')\r
+\r
+-- environment.\r
+-- Note: generated functions are set to the environment of the 'new' call.\r
+ assert(5 == (function()\r
+ local env = {d = 5}\r
+ setfenv(1, env)\r
+ local comp = comp.new()\r
+ return comp 'sum(d for x)' {1}\r
+ end)());\r
+print 'DONE'\r
--- /dev/null
+require 'pl'\r
+asserteq = require 'pl.test'.asserteq\r
+\r
+function testconfig(test,tbl,cfg)\r
+ local f = stringio.open(test)\r
+ local c = config.read(f,cfg)\r
+ f:close()\r
+ if not tbl then\r
+ print(pretty.write(c))\r
+ else\r
+ asserteq(c,tbl)\r
+ end\r
+end\r
+\r
+testconfig ([[\r
+ ; comment 2 (an ini file)\r
+[section!]\r
+bonzo.dog=20,30\r
+config_parm=here we go again\r
+depth = 2\r
+[another]\r
+felix="cat"\r
+]],{\r
+ section_ = {\r
+ bonzo_dog = { -- comma-sep values get split by default\r
+ 20,\r
+ 30\r
+ },\r
+ depth = 2,\r
+ config_parm = "here we go again"\r
+ },\r
+ another = {\r
+ felix = "\"cat\""\r
+ }\r
+})\r
+\r
+\r
+testconfig ([[\r
+# this is a more Unix-y config file\r
+fred = 1\r
+alice = 2\r
+home.dog = /bonzo/dog/etc\r
+]],{\r
+ home_dog = "/bonzo/dog/etc", -- note the default is {variablilize = true}\r
+ fred = 1,\r
+ alice = 2\r
+})\r
+\r
+-- backspace line continuation works, thanks to config.lines function\r
+testconfig ([[\r
+foo=frodo,a,c,d, \\r
+ frank, alice, boyo\r
+]],\r
+{\r
+ foo = {\r
+ "frodo",\r
+ "a",\r
+ "c",\r
+ "d",\r
+ "frank",\r
+ "alice",\r
+ "boyo"\r
+ }\r
+}\r
+)\r
+\r
+------ options to control default behaviour -----\r
+\r
+-- want to keep key names as is!\r
+testconfig ([[\r
+alpha.dog=10\r
+# comment here\r
+]],{\r
+ ["alpha.dog"]=10\r
+},{variabilize=false})\r
+\r
+-- don't convert strings to numbers\r
+testconfig ([[\r
+alpha.dog=10\r
+; comment here\r
+]],{\r
+ alpha_dog="10"\r
+},{convert_numbers=false})\r
+\r
+-- don't split comma-lists by setting the list delimiter to something else\r
+testconfig ([[\r
+extra=10,'hello',42\r
+]],{\r
+ extra="10,'hello',42"\r
+},{list_delim='@'})\r
+\r
+-- Unix-style password file\r
+testconfig([[\r
+lp:x:7:7:lp:/var/spool/lpd:/bin/sh\r
+mail:x:8:8:mail:/var/mail:/bin/sh\r
+news:x:9:9:news:/var/spool/news:/bin/sh\r
+]],\r
+{\r
+ {\r
+ "lp",\r
+ "x",\r
+ 7,\r
+ 7,\r
+ "lp",\r
+ "/var/spool/lpd",\r
+ "/bin/sh"\r
+ },\r
+ {\r
+ "mail",\r
+ "x",\r
+ 8,\r
+ 8,\r
+ "mail",\r
+ "/var/mail",\r
+ "/bin/sh"\r
+ },\r
+ {\r
+ "news",\r
+ "x",\r
+ 9,\r
+ 9,\r
+ "news",\r
+ "/var/spool/news",\r
+ "/bin/sh"\r
+ }\r
+},\r
+{list_delim=':'})\r
+\r
+-- Unix updatedb.conf is in shell script form, but config.read\r
+-- copes by extracting the variables as keys and the export\r
+-- commands as the array part; there is an option to remove quotes\r
+-- from values\r
+testconfig([[\r
+# Global options for invocations of find(1)\r
+FINDOPTIONS='-ignore_readdir_race'\r
+export FINDOPTIONS\r
+]],{\r
+ "export FINDOPTIONS",\r
+ FINDOPTIONS = "-ignore_readdir_race"\r
+},{trim_quotes=true})\r
+\r
+-- Unix fstab format. No key/value assignments so use `ignore_assign`;\r
+-- list values are separated by a number of spaces\r
+testconfig([[\r
+# <file system> <mount point> <type> <options> <dump> <pass>\r
+proc /proc proc defaults 0 0\r
+/dev/sda1 / ext3 defaults,errors=remount-ro 0 1\r
+]],\r
+{\r
+ {\r
+ "proc",\r
+ "/proc",\r
+ "proc",\r
+ "defaults",\r
+ 0,\r
+ 0\r
+ },\r
+ {\r
+ "/dev/sda1",\r
+ "/",\r
+ "ext3",\r
+ "defaults,errors=remount-ro",\r
+ 0,\r
+ 1\r
+ }\r
+},\r
+{list_delim='%s+',ignore_assign=true}\r
+)\r
+\r
+\r
+-- altho this works, rather use pl.data.read for this kind of purpose.\r
+testconfig ([[\r
+# this is just a set of comma-separated values\r
+1000,444,222\r
+44,555,224\r
+]],{\r
+ {\r
+ 1000,\r
+ 444,\r
+ 222\r
+ },\r
+ {\r
+ 44,\r
+ 555,\r
+ 224\r
+ }\r
+})\r
+\r
+\r
+\r
--- /dev/null
+--_DEBUG=true
+data = require 'pl.data'
+List = require 'pl.List'
+array = require 'pl.array2d'
+seq = require 'pl.seq'
+utils = require 'pl.utils'
+stringio = require 'pl.stringio'
+open = stringio. open
+asserteq = require 'pl.test' . asserteq
+T = require 'pl.test'. tuple
+
+-- tab-separated data, explicit column names
+t1f = open [[
+EventID Magnitude LocationX LocationY LocationZ LocationError EventDate DataFile
+981124001 2.0 18988.4 10047.1 4149.7 33.8 24/11/1998 11:18:05 981124DF.AAB
+981125001 0.8 19104.0 9970.4 5088.7 3.0 25/11/1998 05:44:54 981125DF.AAB
+981127003 0.5 19012.5 9946.9 3831.2 46.0 27/11/1998 17:15:17 981127DF.AAD
+981127005 0.6 18676.4 10606.2 3761.9 4.4 27/11/1998 17:46:36 981127DF.AAF
+981127006 0.2 19109.9 9716.5 3612.0 11.8 27/11/1998 19:29:51 981127DF.AAG
+]]
+
+t1 = data.read (t1f)
+-- column_by_name returns a List
+asserteq(t1:column_by_name 'Magnitude',List{2,0.8,0.5,0.6,0.2})
+-- can use array.column as well
+asserteq(array.column(t1,2),{2,0.8,0.5,0.6,0.2})
+
+-- only numerical columns (deduced from first data row) are converted by default
+-- can look up indices in the list fieldnames.
+EDI = t1.fieldnames:index 'EventDate'
+assert(type(t1[1][EDI]) == 'string')
+
+-- select method returns a sequence, in this case single-valued.
+-- (Note that seq.copy returns a List)
+asserteq(seq(t1:select 'LocationX where Magnitude > 0.5'):copy(),List{18988.4,19104,18676.4})
+
+--[[
+--a common select usage pattern:
+for event,mag in t1:select 'EventID,Magnitude sort by Magnitude desc' do
+ print(event,mag)
+end
+--]]
+
+-- space-separated, but with last field containing spaces.
+t2f = open [[
+USER PID %MEM %CPU COMMAND
+sdonovan 2333 0.3 0.1 background --n=2
+root 2332 0.4 0.2 fred --start=yes
+root 2338 0.2 0.1 backyard-process
+]]
+
+t2,err = data.read(t2f,{last_field_collect=true})
+if not t2 then return print (err) end
+
+-- the last_field_collect option is useful with space-delimited data where the last
+-- field may contain spaces. Otherwise, a record count mismatch should be an error!
+lt2 = List(t2[2])
+asserteq(lt2:join ',','root,2332,0.4,0.2,fred --start=yes')
+
+-- fieldnames are converted into valid identifiers by substituting _
+-- (we do this to make select queries parseable by Lua)
+asserteq(t2.fieldnames,List{'USER','PID','_MEM','_CPU','COMMAND'})
+
+-- select queries are NOT SQL so remember to use == ! (and no 'between' operator, sorry)
+--s,err = t2:select('_MEM where USER="root"')
+--assert(err == [[[string "tmp"]:9: unexpected symbol near '=']])
+
+s = t2:select('_MEM where USER=="root"')
+assert(s() == 0.4)
+assert(s() == 0.2)
+assert(s() == nil)
+
+-- CSV, Excel style
+t3f = open [[
+Department Name,Employee ID,Project,Hours Booked
+sales,1231,overhead,4
+sales,1255,overhead,3
+engineering,1501,development,5
+engineering,1501,maintenance,3
+engineering,1433,maintenance,10
+]]
+
+t3 = data.read(t3f)
+
+-- a common operation is to select using a given list of columns, and each row
+-- on some explicit condition. The select() method can take a table with these
+-- parameters
+keepcols = {'Employee_ID','Hours_Booked'}
+
+q = t3:select { fields = keepcols,
+ where = function(row) return row[1]=='engineering' end
+ }
+
+asserteq(seq.copy2(q),{{1501,5},{1501,3},{1433,10}})
+
+-- another pattern is doing a select to restrict rows & columns, process some
+-- fields and write out the modified rows.
+
+utils.import 'pl.func'
+
+outf = stringio.create()
+
+names = {[1501]='don',[1433]='dilbert'}
+
+t3:write_row (outf,{'Employee','Hours_Booked'})
+q = t3:select_row {fields=keepcols,where=Eq(_1[1],'engineering')}
+for row in q do
+ row[1] = names[row[1]]
+ t3:write_row(outf,row)
+end
+
+asserteq(outf:value(),
+[[
+Employee,Hours_Booked
+don,5
+don,3
+dilbert,10
+]])
+
+-- data may not always have column headers. When creating a data object
+-- from a two-dimensional array, must specify the fieldnames, as a list or a string.
+-- The delimiter is deduced from the fieldname string, so a string just containing
+-- the delimiter will set it, and the fieldnames will be empty.
+local dat = List()
+local row = List.range(1,10)
+for i = 1,10 do
+ dat:append(row:map('*',i))
+end
+dat = data.new(dat,',')
+local out = stringio.create()
+dat:write(out,',')
+asserteq(out:value(), [[
+1,2,3,4,5,6,7,8,9,10
+2,4,6,8,10,12,14,16,18,20
+3,6,9,12,15,18,21,24,27,30
+4,8,12,16,20,24,28,32,36,40
+5,10,15,20,25,30,35,40,45,50
+6,12,18,24,30,36,42,48,54,60
+7,14,21,28,35,42,49,56,63,70
+8,16,24,32,40,48,56,64,72,80
+9,18,27,36,45,54,63,72,81,90
+10,20,30,40,50,60,70,80,90,100
+]])
+
+-- you can always use numerical field indices, AWK-style;
+-- note how the copy_select method gives you a data object instead of an
+-- iterator over the fields
+local res = dat:copy_select '$1,$3 where $1 > 5'
+local L = List
+asserteq(L(res),L{
+ L{6, 18},
+ L{7,21},
+ L{8,24},
+ L{9,27},
+ L{10,30},
+})
+
+-- the column_by_name method may take a fieldname or an index
+asserteq(dat:column_by_name(2), L{2,4,6,8,10,12,14,16,18,20})
+
+-- the field list may contain expressions or even constants
+local q = dat:select '$3,2*$4 where $1 == 8'
+asserteq(T(q()),T(24,64))
+
+dat = data.read(open [[
+1.0 0.1
+0.2 1.3
+]])
+
+-- if a method cannot be found, then we look up in array2d
+-- array2d.flatten(t) makes a 1D list out of a 2D array,
+-- and then List.minmax() gets the extrema.
+
+asserteq(T(dat:flatten():minmax()),T(0.1,1.3))
+
--- /dev/null
+local test = require 'pl.test'
+local asserteq, assertmatch = test.asserteq, test.assertmatch
+local dump = require 'pl.pretty'.dump
+local T = require 'pl.test'.tuple
+
+local Date = require 'pl.Date'
+
+--[[
+d = Date()
+print(d)
+print(d:year())
+d:day(20)
+print(d)
+d:add {day = 2}
+print(d:day())
+d = Date() -- 'now'
+print(d:last_day():day())
+print(d:month(7):last_day())
+--]]
+
+function check_df(fmt,str,no_check)
+ local df = Date.Format(fmt)
+ local d = df:parse(str)
+ --print(str,d)
+ if not no_check then
+ asserteq(df:tostring(d),str)
+ end
+end
+
+check_df('dd/mm/yy','02/04/10')
+check_df('mm/dd/yyyy','04/02/2010')
+check_df('yyyy-mm-dd','2011-02-20')
+check_df('yyyymmdd','20070320')
+
+-- use single fields for 'slack' parsing
+check_df('m/d/yyyy','1/5/2001',true)
+
+check_df('HH:MM','23:10')
+
+iso = Date.Format 'yyyy-mm-dd' -- ISO date
+d = iso:parse '2010-04-10'
+asserteq(T(d:day(),d:month(),d:year()),T(10,4,2010))
+amer = Date.Format 'mm/dd/yyyy' -- American style
+s = amer:tostring(d)
+dc = amer:parse(s)
+asserteq(d,dc)
+
+d = Date() -- today
+d:add { day = 1 } -- tomorrow
+assert(d > Date())
+
+-------- testing 'flexible' date parsing ---------
+
+
+local df = Date.Format()
+
+function parse_date (s)
+ return df:parse(s)
+end
+
+-- ISO 8601
+-- specified as UTC plus/minus offset
+
+function parse_utc (s)
+ local d = parse_date(s)
+ d:toUTC()
+ return d
+end
+
+asserteq(parse_utc '2010-05-10 12:35:23Z', Date(2010,05,10,12,35,23))
+asserteq(parse_utc '2008-10-03T14:30+02', Date(2008,10,03,12,30))
+asserteq(parse_utc '2008-10-03T14:00-02:00',Date(2008,10,03,16,0))
+
+---- can't do anything before 1970, which is somewhat unfortunate....
+--parse_date '20/03/59'
+
+asserteq(parse_date '15:30', Date {hour=15,min=30})
+asserteq(parse_date '8.05pm', Date {hour=20,min=5})
+asserteq(parse_date '28/10/02', Date {year=2002,month=10,day=28})
+asserteq(parse_date ' 5 Feb 2012 ', Date {year=2012,month=2,day=5})
+asserteq(parse_date '20 Jul ', Date {month=7,day=20})
+asserteq(parse_date '05/04/02 15:30:43', Date{year=2002,month=4,day=5,hour=15,min=30,sec=43})
+asserteq(parse_date 'march', Date {month=3})
+asserteq(parse_date '2010-05-23T0130', Date{year=2010,month=5,day=23,hour=1,min=30})
+asserteq(parse_date '2008-10-03T14:30:45', Date{year=2008,month=10,day=3,hour=14,min=30,sec=45})
+
+function err (status,e)
+ return e
+end
+
+assertmatch(err(parse_date('2005-10-40 01:30')),'40 is not between 1 and 31')
+assertmatch(err(parse_date('14.20pm')),'14 is not between 0 and 12')
+
+
--- /dev/null
+-- This test file expects to be ran from 'run.lua' in the root Penlight directory.
+
+local dir = require( "pl.dir" )
+local file = require( "pl.file" )
+local path = require( "pl.path" )
+local asserteq = require( "pl.test" ).asserteq
+local pretty = require( "pl.pretty" )
+
+local normpath = path.normpath
+
+local expected = {normpath "../docs/config.ld"}
+
+local files = dir.getallfiles( normpath "../docs/", "*.ld" )
+
+asserteq( files, expected )
+
+-- Test move files -----------------------------------------
+
+-- Create a dummy file
+local fileName = path.tmpname()
+file.write( fileName, string.rep( "poot ", 1000 ) )
+
+local newFileName = path.tmpname()
+local err, msg = dir.movefile( fileName, newFileName )
+
+-- Make sure the move is successful
+assert( err, msg )
+
+-- Check to make sure the original file is gone
+
+asserteq( path.exists( fileName ), false )
+
+-- Check to make sure the new file is there
+asserteq (path.exists( newFileName ) , newFileName)
+
+-- Clean up
+file.delete( newFileName )
+
--- /dev/null
+--- testing Lua 5.1/5.2 compatibility functions
+-- these are global side-effects of pl.utils
+local utils = require 'pl.utils'
+local asserteq = require 'pl.test'.asserteq
+local _,lua = require 'pl.app'. lua()
+
+-- utils.execute is a compromise between 5.1 and 5.2 for os.execute changes
+-- can we call Lua ?
+local ok,code = utils.execute(lua..' -v')
+assert(ok == true and code == 0)
+
+-- table.pack is defined for 5.1
+local t = table.pack(1,nil,'hello')
+asserteq(t.n,3)
+assert(t[1] == 1 and t[3] == 'hello')
+
+-- unpack is globally available for 5.2
+local a,b = unpack{10,'wow'}
+assert(a == 10 and b == 'wow')
+
+-- utils.load() is Lua 5.2 style
+chunk = utils.load('return x+y','tmp','t',{x=1,y=2})
+asserteq(chunk(),3)
+
+-- package.searchpath for Lua 5.1
+-- nota bene: depends on ./?.lua being in the package.path!
+-- So we hack it if not found
+if not package.path:find '.[/\\]%?' then
+ package.path = './?.lua;'..package.path
+end
+asserteq(
+ package.searchpath('test-fenv',package.path):gsub('\\','/'),
+ './test-fenv.lua'
+)
+
+-- testing getfenv and setfenv for both interpreters
+
+function test()
+ return X + Y + Z
+end
+
+t = {X = 1, Y = 2, Z = 3}
+
+setfenv(test,t)
+
+assert(test(),6)
+
+t.X = 10
+
+assert(test(),15)
+
+local getfenv,_G = getfenv,_G
+
+function test2()
+ local env = {x=2}
+ setfenv(1,env)
+ asserteq(getfenv(1),env)
+ asserteq(x,2)
+end
+
+test2()
+
+
+
--- /dev/null
+require 'pl'\r
+asserteq = require('pl.test').asserteq\r
+utils.import('pl.func')\r
+\r
+ -- _DEBUG = true\r
+\r
+function pprint (t)\r
+ print(pretty.write(t))\r
+end\r
+\r
+function test (e)\r
+ local v = {}\r
+ print('test',collect_values(e,v))\r
+ if #v > 0 then pprint(v) end\r
+ local rep = repr(e)\r
+ print(rep)\r
+end\r
+\r
+import ('math')\r
+\r
+test(_1+_2('hello'))\r
+test(sin(_1))\r
+test(_1:method())\r
+test(Not(_1))\r
+\r
+asserteq(instantiate(_1+_2)(10,20),30)\r
+asserteq(instantiate(_1+20)(10),30)\r
+asserteq(instantiate(Or(Not(_1),_2))(true,true),true)\r
+test(_1() + _2() + _3())\r
+print(I(_1+_2)(10,20))\r
+test(sin(_1)+cos(_2))\r
+\r
+\r
+asserteq(instantiate(_1+_2)(10,20),30)\r
+\r
+ls = List {1,2,3,4}\r
+res = ls:map(10*_1 - 1)\r
+asserteq(res,List {9,19,29,39})\r
+\r
+-- note that relational operators can't be overloaded for _different_ types\r
+ls = List {10,20,30,40}\r
+asserteq(ls:filter(Gt(_1,20)),List {30,40})\r
+\r
+\r
+local map,map2 = tablex.map,tablex.map2\r
+\r
+--~ test(Len(_1))\r
+\r
+-- methods can be applied to all items in a table with map\r
+asserteq (map(_1:sub(1,2),{'one','four'}),{'on','fo'})\r
+\r
+--~ -- or you can do this using List:map\r
+asserteq( List({'one','four'}):map(_1:sub(1,2)), List{'on','fo'})\r
+\r
+--~ -- note that Len can't be represented generally by #, since this can only be overriden by userdata\r
+asserteq( map(Len(_1),{'one','four'}), {3,4} )\r
+\r
+--~ -- simularly, 'and' and 'or' are not really operators in Lua, so we need a function notation for them\r
+asserteq (map2(Or(_1,_2),{false,'b'},{'.lua',false}),{'.lua','b'})\r
+\r
+--~ -- binary operators: + - * / % ^ ..\r
+asserteq (map2(_1.._2,{'a','b'},{'.lua','.c'}),{'a.lua','b.c'})\r
+\r
+t1 = {alice=23,fred=34}\r
+t2 = {bob=25,fred=34}\r
+\r
+intersection = bind(tablex.merge,_1,_2,false)\r
+\r
+asserteq(intersection(t1,t2),{fred=34})\r
+\r
+union = bind(tablex.merge,_1,_2,true)\r
+\r
+asserteq(union(t1,t2),{bob=25,fred=34,alice=23})\r
+\r
+asserteq(repr(_1+_2),"_1 + _2")\r
+\r
+\r
+\r
+\r
+\r
+\r
+\r
+\r
+\r
--- /dev/null
+require 'pl'\r
+\r
+utils.on_error 'quit'\r
+\r
+stuff = [[\r
+Department Name,Employee ID,Project,Hours Booked\r
+sales, 1231,overhead,4\r
+sales,1255,overhead,3\r
+engineering,1501,development,5\r
+engineering,1501,maintenance,3\r
+engineering,1433,maintenance,10\r
+]]\r
+\r
+t = data.read(stringio.open(stuff))\r
+\r
+q = t:select 'Employee_ID,Hours_Booked where Department_Name == "engineering"'\r
+\r
+test.asserteq2(1501,5,q())\r
+test.asserteq2(1501,3,q())\r
+test.asserteq2(1433,10,q())\r
--- /dev/null
+\r
+local test = require 'pl.test'\r
+local lapp = require 'pl.lapp'\r
+\r
+local k = 1\r
+function check (spec,args,match)\r
+ local args = lapp(spec,args)\r
+ for k,v in pairs(args) do\r
+ if type(v) == 'userdata' then args[k]:close(); args[k] = '<file>' end\r
+ end\r
+ test.asserteq(args,match)\r
+end\r
+\r
+-- force Lapp to throw an error, rather than just calling os.exit()\r
+lapp.show_usage_error = 'throw'\r
+\r
+function check_error(spec,args,msg)\r
+ arg = args\r
+ local ok,err = pcall(lapp,spec)\r
+ test.assertmatch(err,msg)\r
+end\r
+\r
+local parmtest = [[\r
+Testing 'array' parameter handling\r
+ -o,--output... (string)\r
+ -v...\r
+]]\r
+\r
+\r
+check (parmtest,{'-o','one'},{output={'one'},v={false}})\r
+check (parmtest,{'-o','one','-v'},{output={'one'},v={true}})\r
+check (parmtest,{'-o','one','-vv'},{output={'one'},v={true,true}})\r
+check (parmtest,{'-o','one','-o','two'},{output={'one','two'},v={false}})\r
+\r
+\r
+local simple = [[\r
+Various flags and option types\r
+ -p A simple optional flag, defaults to false\r
+ -q,--quiet A simple flag with long name\r
+ -o (string) A required option with argument\r
+ <input> (default stdin) Optional input file parameter...\r
+]]\r
+\r
+check(simple,\r
+ {'-o','in'},\r
+ {quiet=false,p=false,o='in',input='<file>'})\r
+\r
+check(simple,\r
+ {'-o','help','-q','test-lapp.lua'},\r
+ {quiet=true,p=false,o='help',input='<file>',input_name='test-lapp.lua'})\r
+\r
+local longs = [[\r
+ --open (string)\r
+]]\r
+\r
+check(longs,{'--open','folder'},{open='folder'})\r
+\r
+local extras1 = [[\r
+ <files...> (string) A bunch of files\r
+]]\r
+\r
+check(extras1,{'one','two'},{files={'one','two'}})\r
+\r
+-- any extra parameters go into the array part of the result\r
+local extras2 = [[\r
+ <file> (string) A file\r
+]]\r
+\r
+check(extras2,{'one','two'},{file='one','two'})\r
+\r
+local extended = [[\r
+ --foo (string default 1)\r
+ -s,--speed (slow|medium|fast default medium)\r
+ -n (1..10 default 1)\r
+ -p print\r
+ -v verbose\r
+]]\r
+\r
+\r
+\r
+check(extended,{},{foo='1',speed='medium',n=1,p=false,v=false})\r
+check(extended,{'-pv'},{foo='1',speed='medium',n=1,p=true,v=true})\r
+check(extended,{'--foo','2','-s','fast'},{foo='2',speed='fast',n=1,p=false,v=false})\r
+check(extended,{'--foo=2','-s=fast','-n2'},{foo='2',speed='fast',n=2,p=false,v=false})\r
+\r
+check_error(extended,{'--speed','massive'},"value 'massive' not in slow|medium|fast")\r
+\r
+check_error(extended,{'-n','x'},"unable to convert to number: x")\r
+\r
+check_error(extended,{'-n','12'},"n out of range")\r
+\r
+\r
+\r
--- /dev/null
+asserteq = require('pl.test').asserteq\r
+T = require 'pl.test' . tuple\r
+lexer = require 'pl.lexer'\r
+seq = require 'pl.seq'\r
+List = require ('pl.List')\r
+copy2 = seq.copy2\r
+\r
+s = '20 = hello'\r
+ asserteq(copy2(lexer.scan (s,nil,{space=false},{number=false})),\r
+ {{'number','20'},{'space',' '},{'=','='},{'space',' '},{'iden','hello'}})\r
+\r
+ asserteq(copy2(lexer.scan (s,nil,{space=true},{number=true})),\r
+ {{'number',20},{'=','='},{'iden','hello'}})\r
+\r
+asserteq(copy2(lexer.lua('test(20 and a > b)',{space=true})),\r
+ {{'iden','test'},{'(','('},{'number',20},{'keyword','and'},{'iden','a'},\r
+ {'>','>'},{'iden','b'},{')',')'}} )\r
+\r
+lines = [[\r
+for k,v in pairs(t) do\r
+ if type(k) == 'number' then\r
+ print(v) -- array-like case\r
+ else\r
+ print(k,v)\r
+ end -- if\r
+end\r
+]]\r
+\r
+ls = List()\r
+for tp,val in lexer.lua(lines,{space=true,comments=true}) do\r
+ assert(tp ~= 'space' and tp ~= 'comment')\r
+ if tp == 'keyword' then ls:append(val) end\r
+end\r
+asserteq(ls,List{'for','in','do','if','then','else','end','end'})\r
+\r
+tok = lexer.scan([[\r
+ 'help' "help" "dolly you're fine" "a \"quote\" here"\r
+]],nil,{space=true,string=true})\r
+\r
+function t2() local t,v = tok(); return v end\r
+\r
+asserteq(t2(),'help')\r
+asserteq(t2(),'help')\r
+asserteq(t2(),"dolly you're fine")\r
+asserteq(t2(),"a \\\"quote\\\" here") --> NOT convinced this is correct!\r
+\r
+tok = lexer.lua('10+2.3') ---> '+' is no longer considered part of the number!\r
+asserteq(T(tok()),T('number',10))\r
+asserteq(T(tok()),T('+','+'))\r
+asserteq(T(tok()),T('number',2.3))\r
+\r
+local txt = [==[\r
+-- comment\r
+--[[\r
+block\r
+comment\r
+]][[\r
+hello dammit\r
+]][[hello]]\r
+]==]\r
+\r
+tok = lexer.lua(txt,{})\r
+asserteq(tok(),'comment')\r
+asserteq(tok(),'comment')\r
+asserteq(tok(),'string')\r
+asserteq(tok(),'string')\r
+asserteq(tok(),'space')\r
+\r
+txt = [[\r
+// comment\r
+/* a long\r
+set of words */ // more\r
+]]\r
+\r
+tok = lexer.cpp(txt,{})\r
+asserteq(tok(),'comment')\r
+asserteq(tok(),'comment')\r
+asserteq(tok(),'space')\r
+asserteq(tok(),'comment')\r
+\r
+local function teststring (s)\r
+ local tok = lexer.lua(s,{},{string=false})\r
+ local t,v = tok()\r
+ asserteq(t,"string")\r
+ asserteq(v,s)\r
+end\r
+\r
+teststring [["hello\\"]]\r
+teststring [["hello\"dolly"]]\r
+teststring [['hello\'dolly']]\r
+teststring [['']]\r
+teststring [[""]]\r
+\r
+\r
--- /dev/null
+-- testing Map functionality\r
+\r
+require 'pl'\r
+\r
+local asserteq = test.asserteq\r
+\r
+local cmp = tablex.compare_no_order\r
+\r
+local m = Map{alpha=1,beta=2,gamma=3}\r
+\r
+assert (cmp(m:values(),{1,2,3}))\r
+\r
+assert (cmp(m:keys(),{'alpha','beta','gamma'}))\r
+\r
+asserteq (m:items(),{{'alpha',1},{'beta',2},{'gamma',3}})\r
+\r
+asserteq (m:getvalues {'alpha','gamma'}, {1,3})\r
--- /dev/null
+-- tablex.move when the tables are the same\r
+-- and there are overlapping ranges\r
+T = require 'pl.tablex'\r
+asserteq = require 'pl.test'.asserteq\r
+\r
+t1 = {1,2,3,4,5,6,7,8,9,10}\r
+t2 = T.copy(t1)\r
+t3 = T.copy(t1)\r
+\r
+T.move(t1,t2,4,1,4)\r
+T.move(t3,t3,4,1,4)\r
+asserteq(t1,t3)\r
--- /dev/null
+local path = require 'pl.path'
+asserteq = require 'pl.test'.asserteq
+
+function quote(s)
+ return '"'..s..'"'
+end
+
+function print2(s1,s2)
+ print(quote(s1),quote(s2))
+end
+
+function testpath(pth,p1,p2,p3)
+ local dir,rest = path.splitpath(pth)
+ local name,ext = path.splitext(rest)
+ asserteq(dir,p1)
+ asserteq(name,p2)
+ asserteq(ext,p3)
+end
+
+testpath ([[/bonzo/dog_stuff/cat.txt]],[[/bonzo/dog_stuff]],'cat','.txt')
+testpath ([[/bonzo/dog/cat/fred.stuff]],'/bonzo/dog/cat','fred','.stuff')
+testpath ([[../../alice/jones]],'../../alice','jones','')
+testpath ([[alice]],'','alice','')
+testpath ([[/path-to/dog/]],[[/path-to/dog]],'','')
+
+asserteq( path.isdir( "../docs" ), true )
+asserteq( path.isdir( "../docs/config.ld" ), false )
+
+asserteq( path.isfile( "../docs" ), false )
+asserteq( path.isfile( "../docs/config.ld" ), true )
+
+local norm = path.normpath
+local p = norm '/a/b'
+
+asserteq(norm '/a/fred/../b',p)
+asserteq(norm '/a//b',p)
+
+function testnorm(p1,p2)
+ asserteq(p2,norm(p1):gsub('\\','/'))
+end
+
+testnorm('a/b/..','a/')
+testnorm('a/b/../..','.')
+testnorm('a/b/../c/../../d','d')
+testnorm('a/.','a/.')
+testnorm('a/./','a/')
+testnorm('a/b/.././..','.')
+
+if path.is_windows then
+ asserteq(norm [[\a\.\b]],p)
+ -- UNC paths
+ asserteq(norm [[\\bonzo\..\dog]], [[\\dog]])
+ asserteq(norm [[\\?\c:\bonzo\dog\.\]],[[\\?\c:\bonzo\dog\]])
+end
+
+asserteq(norm '1/2/../3/4/../5',norm '1/3/5')
+
+
--- /dev/null
+require 'pl'\r
+function testm(x,s)\r
+ test.asserteq(pretty.number(x,'M'),s)\r
+end\r
+\r
+testm(123,'123B')\r
+testm(1234,'1.2KiB')\r
+testm(10*1024,'10.0KiB')\r
+testm(1024*1024,'1.0MiB')\r
+\r
+function testn(x,s)\r
+ test.asserteq(pretty.number(x,'N',2),s)\r
+end\r
+\r
+testn(123,'123')\r
+testn(1234,'1.23K')\r
+testn(10*1024,'10.24K')\r
+testn(1024*1024,'1.05M')\r
+testn(1024*1024*1024,'1.07B')\r
+\r
+function testc(x,s)\r
+ test.asserteq(pretty.number(x,'T'),s)\r
+end\r
+\r
+testc(123,'123')\r
+testc(1234,'1,234')\r
+testc(12345,'12,345')\r
+testc(123456,'123,456')\r
+testc(1234567,'1,234,567')\r
+testc(12345678,'12,345,678')\r
+\r
--- /dev/null
+local pretty = require 'pl.pretty'\r
+local utils = require 'pl.utils'\r
+local test = require 'pl.test'\r
+local asserteq, assertmatch = test.asserteq, test.assertmatch\r
+\r
+t1 = {\r
+ 'one','two','three',{1,2,3},\r
+ alpha=1,beta=2,gamma=3,['&']=true,[0]=false,\r
+ _fred = {true,true},\r
+ s = [[\r
+hello dolly\r
+you're so fine\r
+]]\r
+}\r
+\r
+s = pretty.write(t1) --,' ',true)\r
+t2,err = pretty.read(s)\r
+if err then return print(err) end\r
+asserteq(t1,t2)\r
+\r
+res,err = pretty.read [[\r
+ {\r
+ ['function'] = true,\r
+ ['do'] = true,\r
+ }\r
+]]\r
+assert(res)\r
+\r
+res,err = pretty.read [[\r
+ {\r
+ ['function'] = true,\r
+ ['do'] = function() return end\r
+ }\r
+]]\r
+assertmatch(err,'cannot have functions in table definition')\r
+\r
+res,err = pretty.load([[\r
+-- comments are ok\r
+a = 2\r
+bonzo = 'dog'\r
+t = {1,2,3}\r
+]])\r
+\r
+asserteq(res,{a=2,bonzo='dog',t={1,2,3}})\r
+\r
+--- another potential problem is string functions called implicitly as methods--\r
+res,err = pretty.read [[\r
+{s = ('woo'):gsub('w','wwwwww'):gsub('w','wwwwww')}\r
+]]\r
+\r
+assertmatch(err,utils.lua51 and 'attempt to index a string value' or "attempt to index constant 'woo'")\r
+\r
+---- pretty.load has a _paranoid_ option\r
+res,err = pretty.load([[\r
+k = 0\r
+for i = 1,1e12 do k = k + 1 end\r
+]],{},true)\r
+\r
+assertmatch(err,'looping not allowed')\r
+\r
+-- Check to make sure that no spaces exist when write is told not to\r
+local tbl = { "a", 2, "c", false, 23, 453, "poot", 34 }\r
+asserteq( pretty.write( tbl, "" ), [[{"a",2,"c",false,23,453,"poot",34}]] )\r
+\r
+function testm(x,s)\r
+ asserteq(pretty.number(x,'M'),s)\r
+end\r
+\r
+testm(123,'123B')\r
+testm(1234,'1.2KiB')\r
+testm(10*1024,'10.0KiB')\r
+testm(1024*1024,'1.0MiB')\r
+\r
+function testn(x,s)\r
+ asserteq(pretty.number(x,'N',2),s)\r
+end\r
+\r
+testn(123,'123')\r
+testn(1234,'1.23K')\r
+testn(10*1024,'10.24K')\r
+testn(1024*1024,'1.05M')\r
+testn(1024*1024*1024,'1.07B')\r
+\r
+function testc(x,s)\r
+ asserteq(pretty.number(x,'T'),s)\r
+end\r
+\r
+testc(123,'123')\r
+testc(1234,'1,234')\r
+testc(12345,'12,345')\r
+testc(123456,'123,456')\r
+testc(1234567,'1,234,567')\r
+testc(12345678,'12,345,678')\r
+\r
+asserteq(pretty.number(1e12,'N'),'1000.0B')\r
+\r
--- /dev/null
+-- test-pylib.lua
+local List = require 'pl.List'
+require 'pl.stringx'.import()
+local text = require 'pl.text'
+local Template = text.Template
+local asserteq = require 'pl.test' . asserteq
+
+l = List{10,20,30,40,50}
+s = List{1,2,3,4,5}
+
+-- test using: lua pylist.lua
+local lst = List:new()
+lst:append(10)
+lst:extend{20,30,40,50}
+assert (lst == List{10,20,30,40,50})
+lst:insert(3,11)
+lst:remove_value(40)
+assert (lst == List{10,20,11,30,50})
+local q=lst:pop()
+assert( lst:index(30)==4 )
+assert( lst:count(10)==1 )
+lst:sort()
+lst:reverse()
+assert (lst == List{30,20,11,10})
+assert (lst[#lst] == 10)
+assert (lst[#lst-2] == 20)
+
+lst = List {10,20,30,40,50}
+asserteq (lst:slice(2),{20,30,40,50})
+asserteq (lst:slice(-2),{40,50})
+asserteq (lst:slice(nil,3),{10,20,30})
+asserteq (lst:slice(2,4),{20,30,40})
+asserteq (lst:slice(-4,-2),{20,30,40})
+
+lst = List.range(0,9)
+seq = List{0,1,2,3,4,5,6,7,8,9}
+asserteq(List.range(0,8,2),{0,2,4,6,8})
+asserteq(List.range(0,1,0.2),{0,0.2,0.4,0.6,0.8,1},1e-9)
+
+
+assert(lst == seq)
+asserteq (List('abcd'),List{'a','b','c','d'})
+ls = List{10,20,30,40}
+ls:slice_assign(2,3,{21,31})
+assert (ls == List{10,21,31,40})
+-- strings ---
+s = '123'
+assert (s:isdigit())
+assert (not s:isspace())
+s = 'here the dog is just a dog'
+assert (s:startswith('here'))
+assert (s:endswith('dog'))
+assert (s:count('dog') == 2)
+s = ' here we go '
+assert (s:lstrip() == 'here we go ')
+assert (s:rstrip() == ' here we go')
+assert (s:strip() == 'here we go')
+assert (('hello'):center(20,'+') == '++++++++hello+++++++')
+
+t = Template('${here} is the $answer')
+assert(t:substitute {here = 'one', answer = 'two'} == 'one is the two')
+
+assert (('hello dolly'):title() == 'Hello Dolly')
+assert (('h bk bonzo TOK fred m'):title() == 'H Bk Bonzo Tok Fred M')
--- /dev/null
+require 'pl'
+relpath = path.relpath
+
+path = '/a/b/c'
+
+function slash (p)
+ return (p:gsub('\\','/'))
+end
+
+function try (p,r)
+ test.asserteq(slash(relpath(p,path)),r)
+end
+
+try('/a/b/c/one.lua','one.lua')
+try('/a/b/c/bonzo/two.lua','bonzo/two.lua')
+try('/a/b/three.lua','../three.lua')
+try('/a/four.lua','../../four.lua')
+try('one.lua','one.lua')
+try('../two.lua','../two.lua')
+
+
--- /dev/null
+input = require 'pl.input'\r
+seq = require 'pl.seq'\r
+asserteq = require('pl.test').asserteq\r
+utils = require 'pl.utils'\r
+\r
+\r
+local L = utils.string_lambda\r
+local S = seq.list\r
+local C = seq.copy\r
+local C2 = seq.copy2\r
+\r
+\r
+asserteq (seq.sum(input.numbers '10 20 30 40 50'),150)\r
+x,y = unpack(C(input.numbers('10 20')))\r
+assert (x == 10 and y == 20)\r
+\r
+\r
+local test = {{1,10},{2,20},{3,30}}\r
+asserteq(C2(ipairs{10,20,30}),test)\r
+local res = C2(input.fields({1,2},',','1,10\n2,20\n3,30\n'))\r
+asserteq(res,test)\r
+\r
+asserteq(\r
+ seq.copy(seq.filter(seq.list{10,20,5,15},seq.greater_than(10))),\r
+ {20,15}\r
+)\r
+\r
+asserteq(seq.reduce('-',{1,2,3,4,5}),-13)\r
+\r
+asserteq(seq.count(S{10,20,30,40},L'|x| x > 20'), 2)\r
+\r
+asserteq(C2(seq.zip({1,2,3},{10,20,30})),test)\r
+\r
+asserteq(C(seq.splice({10,20},{30,40})),{10,20,30,40})\r
+\r
+asserteq(C(seq.map(L'#_',{'one','tw'})),{3,2})\r
+\r
+--for l1,l2 in seq.last{10,20,30} do print(l1,l2) end\r
+\r
+asserteq(C2(seq.last{10,20,30}),{{20,10},{30,20}} )\r
+\r
+asserteq(\r
+ seq{10,20,30}:map(L'_+1'):copy(),\r
+ {11,21,31}\r
+)\r
+\r
+asserteq(\r
+ seq{'one','two'}:upper():copy(),\r
+ {'ONE','TWO'}\r
+)\r
+\r
+asserteq(\r
+ C(seq.unique(seq.list{1,2,3,2,1})),\r
+ {1,2,3}\r
+)\r
+\r
+\r
+\r
+\r
--- /dev/null
+class = require 'pl.class'
+M = require 'pl.Map'
+S = require 'pl.Set'
+List = require 'pl.List'
+
+asserteq = require 'pl.test' . asserteq
+asserteq2 = require 'pl.test' . asserteq2
+MultiMap = require 'pl.MultiMap'
+OrderedMap = require 'pl.OrderedMap'
+
+s1 = S{1,2}
+s2 = S{1,2}
+-- equality
+asserteq(s1,s2)
+-- union
+asserteq(S{1,2} + S{2,3},S{1,2,3})
+-- intersection
+asserteq(S{1,2} * S{2,3}, S{2})
+-- symmetric_difference
+asserteq(S{1,2} ^ S{2,3}, S{1,3})
+
+m = M{one=1,two=2}
+asserteq(m,M{one=1,two=2})
+m:update {three=3,four=4}
+asserteq(m,M{one=1,two=2,three=3,four=4})
+
+class.Animal()
+
+function Animal:_init(name)
+ self.name = name
+end
+
+function Animal:__tostring()
+ return self.name..': '..self:speak()
+end
+
+class.Dog(Animal)
+
+function Dog:speak()
+ return 'bark'
+end
+
+class.Cat(Animal)
+
+function Cat:_init(name,breed)
+ self:super(name) -- must init base!
+ self.breed = breed
+end
+
+function Cat:speak()
+ return 'meow'
+end
+
+Lion = class(Cat)
+
+function Lion:speak()
+ return 'roar'
+end
+
+fido = Dog('Fido')
+felix = Cat('Felix','Tabby')
+leo = Lion('Leo','African')
+
+asserteq(tostring(fido),'Fido: bark')
+asserteq(tostring(felix),'Felix: meow')
+asserteq(tostring(leo),'Leo: roar')
+
+assert(Dog:class_of(fido))
+assert(fido:is_a(Dog))
+
+assert(leo:is_a(Animal))
+
+m = MultiMap()
+m:set('john',1)
+m:set('jane',3)
+m:set('john',2)
+
+ms = MultiMap{john={1,2},jane={3}}
+
+asserteq(m,ms)
+
+m = OrderedMap()
+m:set('one',1)
+m:set('two',2)
+m:set('three',3)
+
+asserteq(m:values(),List{1,2,3})
+
+-- usually exercized like this:
+--for k,v in m:iter() do print(k,v) end
+
+fn = m:iter()
+asserteq2 ('one',1,fn())
+asserteq2 ('two',2,fn())
+asserteq2 ('three',3,fn())
+
+o1 = OrderedMap {{z=2},{beta=1},{name='fred'}}
+asserteq(tostring(o1),'{z=2,beta=1,name="fred"}')
+
+-- order of keys is not preserved here!
+o2 = OrderedMap {z=4,beta=1.1,name='alice',extra='dolly'}
+
+o1:update(o2)
+asserteq(tostring(o1),'{z=4,beta=1.1,name="alice",extra="dolly"}')
+
+o1:set('beta',nil)
+
+asserteq(o1,OrderedMap{{z=4},{name='alice'},{extra='dolly'}})
+
+o3 = OrderedMap()
+o3:set('dog',10)
+o3:set('cat',20)
+o3:set('mouse',30)
+
+asserteq(o3:keys(),{'dog','cat','mouse'})
+
+o3:set('dog',nil)
+
+asserteq(o3:keys(),{'cat','mouse'})
+
+-- Vadim found a problem when clearing a key which did not exist already.
+-- The keys list would then contain the key, although the map would not
+o3:set('lizard',nil)
+
+asserteq(o3:keys(),{'cat','mouse'})
+asserteq(o3:values(), {20,30})
+asserteq(tostring(o3),'{cat=20,mouse=30}')
+
+-- copy constructor
+o4 = OrderedMap(o3)
+
+asserteq(o4,o3)
+
+-- constructor throws an error if the argument is bad
+-- (errors same as OrderedMap:update)
+asserteq(false,pcall(function()
+ m = OrderedMap('string')
+end))
+
+---- changing order of key/value pairs ----
+
+o3 = OrderedMap{{cat=20},{mouse=30}}
+
+o3:insert(1,'bird',5) -- adds key/value before specified position
+o3:insert(1,'mouse') -- moves key keeping old value
+asserteq(o3:keys(),{'mouse','bird','cat'})
+asserteq(tostring(o3),'{mouse=30,bird=5,cat=20}')
+o3:insert(2,'cat',21) -- moves key and sets new value
+asserteq(tostring(o3),'{mouse=30,cat=21,bird=5}')
+-- if you don't specify a value for an unknown key, nothing happens to the map
+o3:insert(3,'alligator')
+asserteq(tostring(o3),'{mouse=30,cat=21,bird=5}')
+
+
+
+
+
+
--- /dev/null
+sip = require 'pl.sip'\r
+tablex = require 'pl.tablex'\r
+utils = require 'pl.utils'\r
+\r
+local function dump(t)\r
+ if not t or type(t) ~= 'table' then print '<nada>'; return end\r
+ for k,v in pairs(t) do\r
+ print(k,v,type(v))\r
+ end\r
+end\r
+\r
+function check(pat,line,tbl)\r
+ local parms = {}\r
+ if type(pat) == 'string' then\r
+ pat = sip.compile(pat)\r
+ end\r
+ local res = pat(line,parms)\r
+ if res then\r
+ if not tablex.deepcompare(parms,tbl) then\r
+ print 'parms'\r
+ dump(parms)\r
+ print 'tbl'\r
+ dump(tbl)\r
+ utils.quit(1,'failed!')\r
+ end\r
+ else -- only should happen if we're passed a nil!\r
+ assert(tbl == nil)\r
+ end\r
+end\r
+\r
+c = sip.compile('ref=$S{file}:$d{line}')\r
+check(c,'ref=bonzo:23',{file='bonzo',line=23})\r
+check(c,'here we go ref=c:\\bonzo\\dog.txt:53',{file='c:\\bonzo\\dog.txt',line=53})\r
+check(c,'here is a line ref=xxxx:xx',nil)\r
+\r
+c = sip.compile('($i{x},$i{y},$i{z})')\r
+check(c,'(10,20,30)',{x=10,y=20,z=30})\r
+check(c,' (+233,+99,-40) ',{x=233,y=99,z=-40})\r
+\r
+local pat = '$v{name} = $q{str}'\r
+--assert(sip.create_pattern(pat) == [[([%a_][%w_]*)%s*=%s*(["'])(.-)%2]])\r
+local m = sip.compile(pat)\r
+\r
+check(m,'a = "hello"',{name='a',str='hello'})\r
+check(m,"a = 'hello'",{name='a',str='hello'})\r
+check(m,'_fred="some text"',{name='_fred',str='some text'})\r
+\r
+-- some cases broken in 0.6b release\r
+check('$v is $v','bonzo is dog for sure',{'bonzo','dog'})\r
+check('$v is $','bonzo is dog for sure',{'bonzo','dog for sure'})\r
+check('$v $d','age 23',{'age',23})\r
+\r
+\r
+months={"Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"}\r
+\r
+function adjust_and_check(res)\r
+ if res.year < 100 then\r
+ if res.year < 70 then\r
+ res.year = res.year + 2000\r
+ else\r
+ res.year = res.year + 1900\r
+ end\r
+ end\r
+end\r
+\r
+shortdate = sip.compile('$d{day}/$d{month}/$d{year}')\r
+longdate = sip.compile('$d{day} $v{mon} $d{year}')\r
+isodate = sip.compile('$d{year}-$d{month}-$d{day}')\r
+\r
+function dcheck (d1,d2)\r
+ adjust_and_check(d1)\r
+ assert(d1.day == d2.day and d1.month == d2.month and d1.year == d2.year)\r
+end\r
+\r
+function dates(str,tbl)\r
+ local res = {}\r
+ if shortdate(str,res) then\r
+ dcheck(res,tbl)\r
+ elseif isodate(str,res) then\r
+ dcheck(res,tbl)\r
+ elseif longdate(str,res) then\r
+ res.month = tablex.find(months,res.mon)\r
+ dcheck(res,tbl)\r
+ else\r
+ assert(tbl == nil)\r
+ end\r
+end\r
+\r
+dates ('10/12/2007',{year=2007,month=12,day=10})\r
+dates ('2006-03-01',{year=2006,month=3,day=1})\r
+dates ('25/07/05',{year=2005,month=7,day=25})\r
+dates ('20 Mar 1959',{year=1959,month=3,day=20})\r
+\r
+\r
+\r
+\r
+\r
+\r
+\r
--- /dev/null
+--require 'pl'\r
+require 'pl.strict'\r
+local utils = require 'pl.utils'\r
+utils.printf("that's fine!\n")\r
+local res,err = pcall(function()\r
+ print(x)\r
+ print 'ok?'\r
+end)\r
+assert(err,"variable 'x' is not declared")\r
+\r
--- /dev/null
+local stringio = require 'pl.stringio'\r
+local test = require 'pl.test'\r
+local asserteq = test.asserteq\r
+local T = test.tuple\r
+\r
+function fprintf(f,fmt,...)\r
+ f:write(fmt:format(...))\r
+end\r
+\r
+fs = stringio.create()\r
+for i = 1,100 do\r
+ fs:write('hello','\n','dolly','\n')\r
+end\r
+asserteq(#fs:value(),1200)\r
+\r
+fs = stringio.create()\r
+fs:writef("%s %d",'answer',42) -- note writef() extension method\r
+asserteq(fs:value(),"answer 42")\r
+\r
+inf = stringio.open('10 20 30')\r
+asserteq(T(inf:read('*n','*n','*n')),T(10,20,30))\r
+\r
+local txt = [[\r
+Some lines\r
+here are they\r
+not for other\r
+english?\r
+\r
+]]\r
+\r
+inf = stringio.open (txt)\r
+fs = stringio.create()\r
+for l in inf:lines() do\r
+ fs:write(l,'\n')\r
+end\r
+asserteq(txt,fs:value())\r
+\r
+inf = stringio.open '1234567890ABCDEF'\r
+asserteq(T(inf:read(3), inf:read(5), inf:read()),T('123','45678','90ABCDEF'))\r
+\r
+s = stringio.open 'one\ntwo'\r
+asserteq(s:read() , 'one')\r
+asserteq(s:read() , 'two')\r
+asserteq(s:read() , nil)\r
+s = stringio.open 'one\ntwo'\r
+iter = s:lines()\r
+asserteq(iter() , 'one')\r
+asserteq(iter() , 'two')\r
+asserteq(iter() , nil)\r
+s = stringio.open 'ABC'\r
+iter = s:lines(1)\r
+asserteq(iter() , 'A')\r
+asserteq(iter() , 'B')\r
+asserteq(iter() , 'C')\r
+asserteq(iter() , nil)\r
+\r
+s = stringio.open '20 5.2e-2 52.3'\r
+x,y,z = s:read('*n','*n','*n')\r
+out = stringio.create()\r
+fprintf(out,"%5.2f %5.2f %5.2f!",x,y,z)\r
+asserteq(out:value(),"20.00 0.05 52.30!")\r
+\r
+s = stringio.open 'one\ntwo\n\n'\r
+iter = s:lines '*L'\r
+asserteq(iter(),'one\n')\r
+asserteq(iter(),'two\n')\r
+asserteq(iter(),'\n')\r
+asserteq(iter(),nil)\r
+\r
+\r
+\r
+\r
--- /dev/null
+local stringx = require 'pl.stringx'
+local asserteq = require 'pl.test' . asserteq
+local T = require 'pl.test'.tuple
+
+local function FIX(s)
+ io.stderr:write('FIX:' .. s .. '\n')
+end
+
+
+-- isalpha
+asserteq(T(stringx.isalpha''), T(false))
+asserteq(T(stringx.isalpha' '), T(false))
+asserteq(T(stringx.isalpha'0'), T(false))
+asserteq(T(stringx.isalpha'\0'), T(false))
+asserteq(T(stringx.isalpha'azAZ'), T(true))
+asserteq(T(stringx.isalpha'az9AZ'), T(false))
+
+-- isdigit
+asserteq(T(stringx.isdigit''), T(false))
+asserteq(T(stringx.isdigit' '), T(false))
+asserteq(T(stringx.isdigit'a'), T(false))
+asserteq(T(stringx.isdigit'0123456789'), T(true))
+
+-- isalnum
+asserteq(T(stringx.isalnum''), T(false))
+asserteq(T(stringx.isalnum' '), T(false))
+asserteq(T(stringx.isalnum('azAZ01234567890')), T(true))
+
+-- isspace
+asserteq(T(stringx.isspace''), T(false))
+asserteq(T(stringx.isspace' '), T(true))
+asserteq(T(stringx.isspace' \r\n\f\t'), T(true))
+asserteq(T(stringx.isspace' \r\n-\f\t'), T(false))
+
+-- islower
+asserteq(T(stringx.islower''), T(false))
+asserteq(T(stringx.islower'az'), T(true))
+asserteq(T(stringx.islower'aMz'), T(false))
+asserteq(T(stringx.islower'a z'), T(true))
+
+-- startswith
+local startswith = stringx.startswith
+asserteq(T(startswith('', '')), T(true))
+asserteq(T(startswith('', 'a')), T(false))
+asserteq(T(startswith('a', '')), T(true))
+asserteq(T(startswith('a', 'a')), T(true))
+asserteq(T(startswith('a', 'b')), T(false))
+asserteq(T(startswith('a', 'ab')), T(false))
+asserteq(T(startswith('abc', 'ab')), T(true))
+asserteq(T(startswith('abc', 'bc')), T(false)) -- off by one
+asserteq(T(startswith('abc', '.')), T(false)) -- Lua pattern char
+asserteq(T(startswith('a\0bc', 'a\0b')), T(true)) -- '\0'
+
+
+-- endswith
+-- http://snippets.luacode.org/sputnik.lua?p=snippets/Check_string_ends_with_other_string_74
+local endswith = stringx.endswith
+asserteq(T(endswith("", "")), T(true))
+asserteq(T(endswith("", "a")), T(false))
+asserteq(T(endswith("a", "")), T(true))
+asserteq(T(endswith("a", "a")), T(true))
+asserteq(T(endswith("a", "A")), T(false)) -- case sensitive
+asserteq(T(endswith("a", "aa")), T(false))
+asserteq(T(endswith("abc", "")), T(true))
+asserteq(T(endswith("abc", "ab")), T(false)) -- off by one
+asserteq(T(endswith("abc", "c")), T(true))
+asserteq(T(endswith("abc", "bc")), T(true))
+asserteq(T(endswith("abc", "abc")), T(true))
+asserteq(T(endswith("abc", " abc")), T(false))
+asserteq(T(endswith("abc", "a")), T(false))
+asserteq(T(endswith("abc", ".")), T(false)) -- Lua pattern char
+asserteq(T(endswith("ab\0c", "b\0c")), T(true)) -- \0
+asserteq(T(endswith("ab\0c", "b\0d")), T(false)) -- \0
+
+asserteq(endswith('dollar.dot',{'.dot','.txt'}),true)
+asserteq(endswith('dollar.txt',{'.dot','.txt'}),true)
+asserteq(endswith('dollar.rtxt',{'.dot','.txt'}),false)
+
+-- splitlines
+asserteq(T(stringx.splitlines('')), T({''}))
+asserteq(stringx.splitlines('a'), {'a'})
+asserteq(stringx.splitlines('\n'), {''})
+asserteq(stringx.splitlines('\n\n'), {'', ''})
+asserteq(stringx.splitlines('\r\r'), {'', ''})
+asserteq(stringx.splitlines('ab\ncd\n'), {'ab', 'cd'})
+
+-- expandtabs
+---FIX[[raises error
+asserteq(T(stringx.expandtabs('',0)), T(''))
+asserteq(T(stringx.expandtabs('',1)), T(''))
+asserteq(T(stringx.expandtabs(' ',1)), T(' '))
+-- expandtabs now works like Python's str.expandtabs (up to next tab stop)
+asserteq(T(stringx.expandtabs(' \t ')), T((' '):rep(1+8)))
+asserteq(T(stringx.expandtabs(' \t ',2)), T(' '))
+--]]
+
+-- lfind
+asserteq(T(stringx.lfind('', '')), T(1))
+asserteq(T(stringx.lfind('a', '')), T(1))
+asserteq(T(stringx.lfind('ab', 'b')), T(2))
+asserteq(T(stringx.lfind('abc', 'cd')), T(nil))
+asserteq(T(stringx.lfind('abcbc', 'bc')), T(2))
+
+-- rfind
+asserteq(T(stringx.rfind('', '')), T(1))
+asserteq(T(stringx.rfind('ab', '')), T(3))
+asserteq(T(stringx.rfind('abcbc', 'bc')), T(4))
+asserteq(T(stringx.rfind('abcbcb', 'bc')), T(4))
+asserteq(T(stringx.rfind('ab..cd', '.')), T(4)) -- pattern char
+
+-- replace
+asserteq(T(stringx.replace('', '', '')), T(''))
+asserteq(T(stringx.replace(' ', '', '')), T(' '))
+asserteq(T(stringx.replace(' ', '', ' ')), T(' '))
+asserteq(T(stringx.replace(' ', ' ', '')), T(''))
+asserteq(T(stringx.replace('abcabcabc', 'bc', 'BC')), T('aBCaBCaBC'))
+asserteq(T(stringx.replace('abcabcabc', 'bc', 'BC', 1)), T('aBCabcabc'))
+asserteq(T(stringx.replace('abcabcabc', 'bc', 'BC', 0)), T('abcabcabc'))
+asserteq(T(stringx.replace('abc', 'd', 'e')), T('abc'))
+asserteq(T(stringx.replace('a.b', '.', '%d')), T('a%db'))
+
+-- split
+local split = stringx.split
+asserteq(split('', ''), {''})
+asserteq(split('', 'z'), {}) --FIX:intended and specified behavior?
+asserteq(split('a', ''), {'a'}) --FIX:intended and specified behavior?
+asserteq(split('a', 'a'), {''})
+-- stringx.split now follows the Python pattern, so it uses a substring, not a pattern.
+-- If you need to split on a pattern, use utils.split()
+-- asserteq(split('ab1cd23ef%d', '%d+'), {'ab', 'cd', 'ef%d'}) -- pattern chars
+-- note that leading space is ignored by the default
+asserteq(split(' 1 2 3 '),{'1','2','3'})
+asserteq(split('a*bb*c*ddd','*'),{'a','bb','c','ddd'})
+asserteq(split('dog:fred:bonzo:alice',':',3), {'dog','fred','bonzo:alice'})
+asserteq(split('///','/'),{'','','',''})
+-- capitalize
+asserteq(T(stringx.capitalize('')), T(''))
+asserteq(T(stringx.capitalize('abC deF1')), T('Abc Def1')) -- Python behaviour
+
+-- count
+asserteq(T(stringx.count('', '')), T(0)) --infinite loop]]
+asserteq(T(stringx.count(' ', '')), T(2)) --infinite loop]]
+asserteq(T(stringx.count('a..c', '.')), T(2)) -- pattern chars
+asserteq(T(stringx.count('a1c', '%d')), T(0)) -- pattern chars
+
+-- ljust
+asserteq(T(stringx.ljust('', 0)), T(''))
+asserteq(T(stringx.ljust('', 2)), T(' '))
+asserteq(T(stringx.ljust('ab', 3)), T('ab '))
+asserteq(T(stringx.ljust('ab', 3, '%')), T('ab%'))
+asserteq(T(stringx.ljust('abcd', 3)), T('abcd')) -- agrees with Python
+
+-- rjust
+asserteq(T(stringx.rjust('', 0)), T(''))
+asserteq(T(stringx.rjust('', 2)), T(' '))
+asserteq(T(stringx.rjust('ab', 3)), T(' ab'))
+asserteq(T(stringx.rjust('ab', 3, '%')), T('%ab'))
+asserteq(T(stringx.rjust('abcd', 3)), T('abcd')) -- agrees with Python
+
+-- center
+asserteq(T(stringx.center('', 0)), T(''))
+asserteq(T(stringx.center('', 1)), T(' '))
+asserteq(T(stringx.center('', 2)), T(' '))
+asserteq(T(stringx.center('a', 1)), T('a'))
+asserteq(T(stringx.center('a', 2)), T(' a'))
+asserteq(T(stringx.center('a', 3)), T(' a '))
+
+
+-- ltrim
+-- http://snippets.luacode.org/sputnik.lua?p=snippets/trim_whitespace_from_string_76
+local trim = stringx.lstrip
+asserteq(T(trim''), T'')
+asserteq(T(trim' '), T'')
+asserteq(T(trim' '), T'')
+asserteq(T(trim'a'), T'a')
+asserteq(T(trim' a'), T'a')
+asserteq(T(trim'a '), T'a ')
+asserteq(T(trim' a '), T'a ')
+asserteq(T(trim' a '), T'a ')
+asserteq(T(trim' ab cd '), T'ab cd ')
+asserteq(T(trim' \t\r\n\f\va\000b \r\t\n\f\v'), T'a\000b \r\t\n\f\v')
+-- more
+
+
+-- rtrim
+-- http://snippets.luacode.org/sputnik.lua?p=snippets/trim_whitespace_from_string_76
+local trim = stringx.rstrip
+asserteq(T(trim''), T'')
+asserteq(T(trim' '), T'')
+asserteq(T(trim' '), T'')
+asserteq(T(trim'a'), T'a')
+asserteq(T(trim' a'), T' a')
+asserteq(T(trim'a '), T'a')
+asserteq(T(trim' a '), T' a')
+asserteq(T(trim' a '), T' a')
+asserteq(T(trim' ab cd '), T' ab cd')
+asserteq(T(trim' \t\r\n\f\va\000b \r\t\n\f\v'), T' \t\r\n\f\va\000b')
+-- more
+
+
+-- trim
+-- http://snippets.luacode.org/sputnik.lua?p=snippets/trim_whitespace_from_string_76
+local trim = stringx.strip
+asserteq(T(trim''), T'')
+asserteq(T(trim' '), T'')
+asserteq(T(trim' '), T'')
+asserteq(T(trim'a'), T'a')
+asserteq(T(trim' a'), T'a')
+asserteq(T(trim'a '), T'a')
+asserteq(T(trim' a '), T'a')
+asserteq(T(trim' a '), T'a')
+asserteq(T(trim' ab cd '), T'ab cd')
+asserteq(T(trim' \t\r\n\f\va\000b \r\t\n\f\v'), T'a\000b')
+-- more
+
+
+-- partition
+-- as per str.partition in Python, delimiter must be non-empty;
+-- interpreted as a plain string
+--asserteq(T(stringx.partition('', '')), T('', '', '')) -- error]]
+--asserteq(T(stringx.partition('a', '')), T('', '', 'a')) --error]]
+asserteq(T(stringx.partition('a', 'a')), T('', 'a', ''))
+asserteq(T(stringx.partition('abc', 'b')), T('a', 'b', 'c'))
+asserteq(T(stringx.partition('abc', '.+')), T('abc','',''))
+asserteq(T(stringx.partition('a,b,c', ',')), T('a',',','b,c'))
+-- rpartition
+asserteq(T(stringx.rpartition('a/b/c', '/')), T('a/b', '/', 'c'))
+asserteq(T(stringx.rpartition('abc', 'b')), T('a', 'b', 'c'))
+
+
+-- at (works like s:sub(idx,idx), so negative indices allowed
+asserteq(T(stringx.at('a', 1)), T('a'))
+asserteq(T(stringx.at('ab', 2)), T('b'))
+asserteq(T(stringx.at('abcd', -1)), T('d'))
+
+-- lines
+local function merge(it, ...)
+ assert(select('#', ...) == 0)
+ local ts = {}
+ for val in it do ts[#ts+1] = val end
+ return ts
+end
+asserteq(merge(stringx.lines('')), {''})
+asserteq(merge(stringx.lines('ab')), {'ab'})
+asserteq(merge(stringx.lines('ab\ncd')), {'ab', 'cd'})
+
+-- shorten
+-- The returned string is always equal or less to the given size.
+asserteq(T(stringx.shorten('', 0)), T'')
+asserteq(T(stringx.shorten('a', 1)), T'a')
+asserteq(T(stringx.shorten('ab', 1)), T'.') --FIX:ok?
+asserteq(T(stringx.shorten('abc', 3)), T'abc')
+asserteq(T(stringx.shorten('abcd', 3)), T'...')
+asserteq(T(stringx.shorten('abcde', 5)), T'abcde')
+asserteq(T(stringx.shorten('abcde', 4)), T'a...')
+asserteq(T(stringx.shorten('abcde', 3)), T'...')
+asserteq(T(stringx.shorten('abcde', 2)), T'..')
+asserteq(T(stringx.shorten('abcde', 0)), T'')
+asserteq(T(stringx.shorten('', 0, true)), T'')
+asserteq(T(stringx.shorten('a', 1, true)), T'a')
+asserteq(T(stringx.shorten('ab', 1, true)), T'.')
+asserteq(T(stringx.shorten('abcde', 5, true)), T'abcde')
+asserteq(T(stringx.shorten('abcde', 4, true)), T'...e')
+asserteq(T(stringx.shorten('abcde', 3, true)), T'...')
+asserteq(T(stringx.shorten('abcde', 2, true)), T'..')
+asserteq(T(stringx.shorten('abcde', 0, true)), T'')
+
+-- strip
+asserteq(stringx.strip(' hello '),'hello')
+asserteq(stringx.strip('--[hello] -- - ','-[] '),'hello')
+asserteq(stringx.rstrip('--[hello] -- - ','-[] '),'--[hello')
+
--- /dev/null
+local subst = require 'pl.template'.substitute
+local List = require 'pl.List'
+local asserteq = require 'pl.test'.asserteq
+
+asserteq(subst([[
+# for i = 1,2 do
+<p>Hello $(tostring(i))</p>
+# end
+]],_G),[[
+<p>Hello 1</p>
+<p>Hello 2</p>
+]])
+
+asserteq(subst([[
+<ul>
+# for name in ls:iter() do
+ <li>$(name)</li>
+#end
+</ul>
+]],{ls = List{'john','alice','jane'}}),[[
+<ul>
+ <li>john</li>
+ <li>alice</li>
+ <li>jane</li>
+</ul>
+]])
+
+-- can change the default escape from '#' so we can do C/C++ output.
+-- note that the environment can have a parent field.
+asserteq(subst([[
+> for i,v in ipairs{'alpha','beta','gamma'} do
+ cout << obj.${v} << endl;
+> end
+]],{_parent=_G, _brackets='{}', _escape='>'}),[[
+ cout << obj.alpha << endl;
+ cout << obj.beta << endl;
+ cout << obj.gamma << endl;
+]])
+
+
+
--- /dev/null
+local tablex = require 'pl.tablex'\r
+local utils = require ('pl.utils')\r
+local L = utils.string_lambda\r
+-- bring tablex funtions into global namespace\r
+utils.import(tablex)\r
+local asserteq = require('pl.test').asserteq\r
+\r
+local cmp = deepcompare\r
+\r
+asserteq(\r
+ copy {10,20,30},\r
+ {10,20,30}\r
+)\r
+\r
+asserteq(\r
+ deepcopy {10,20,{30,40}},\r
+ {10,20,{30,40}}\r
+)\r
+\r
+asserteq(\r
+ pairmap(function(i,v) return v end,{10,20,30}),\r
+ {10,20,30}\r
+)\r
+\r
+asserteq(\r
+ pairmap(L'_',{fred=10,bonzo=20}),\r
+ {'fred','bonzo'}\r
+)\r
+\r
+asserteq(\r
+ pairmap(function(k,v) return v end,{fred=10,bonzo=20}),\r
+ {10,20}\r
+)\r
+\r
+asserteq(\r
+ pairmap(function(i,v) return v,i end,{10,20,30}),\r
+ {10,20,30}\r
+)\r
+\r
+asserteq(\r
+ pairmap(function(k,v) return {k,v},k end,{one=1,two=2}),\r
+ {one={'one',1},two={'two',2}}\r
+)\r
+-- same as above, using string lambdas\r
+asserteq(\r
+ pairmap(L'|k,v|{k,v},k',{one=1,two=2}),\r
+ {one={'one',1},two={'two',2}}\r
+)\r
+\r
+\r
+asserteq(\r
+ map(function(v) return v*v end,{10,20,30}),\r
+ {100,400,900}\r
+)\r
+\r
+-- extra arguments to map() are passed to the function; can use\r
+-- the abbreviations provided by pl.operator\r
+asserteq(\r
+ map('+',{10,20,30},1),\r
+ {11,21,31}\r
+)\r
+\r
+asserteq(\r
+ map(L'_+1',{10,20,30}),\r
+ {11,21,31}\r
+)\r
+\r
+-- map2 generalizes for operations on two tables\r
+asserteq(\r
+ map2(math.max,{1,2,3},{0,4,2}),\r
+ {1,4,3}\r
+)\r
+\r
+-- mapn operates over an arbitrary number of input tables (but use map2 for n=2)\r
+asserteq(\r
+ mapn(function(x,y,z) return x+y+z end, {1,2,3},{10,20,30},{100,200,300}),\r
+ {111,222,333}\r
+)\r
+\r
+asserteq(\r
+ mapn(math.max, {1,20,300},{10,2,3},{100,200,100}),\r
+ {100,200,300}\r
+)\r
+\r
+asserteq(\r
+ zip({10,20,30},{100,200,300}),\r
+ {{10,100},{20,200},{30,300}}\r
+)\r
+\r
+assert(compare_no_order({1,2,3,4},{2,1,4,3})==true)\r
+assert(compare_no_order({1,2,3,4},{2,1,4,4})==false)\r
+\r
+asserteq(range(10,9),{})\r
+asserteq(range(10,10),{10})\r
+asserteq(range(10,11),{10,11})\r
+\r
+-- update inserts key-value pairs from the second table\r
+t1 = {one=1,two=2}\r
+t2 = {three=3,two=20,four=4}\r
+asserteq(update(t1,t2),{one=1,three=3,two=20,four=4})\r
+\r
+-- the difference between move and icopy is that the second removes\r
+-- any extra elements in the destination after end of copy\r
+-- 3rd arg is the index to start in the destination, defaults to 1\r
+asserteq(move({1,2,3,4,5,6},{20,30}),{20,30,3,4,5,6})\r
+asserteq(move({1,2,3,4,5,6},{20,30},2),{1,20,30,4,5,6})\r
+asserteq(icopy({1,2,3,4,5,6},{20,30},2),{1,20,30})\r
+-- 5th arg determines how many elements to copy (default size of source)\r
+asserteq(icopy({1,2,3,4,5,6},{20,30},2,1,1),{1,20})\r
+-- 4th arg is where to stop copying from the source (default s to 1)\r
+asserteq(icopy({1,2,3,4,5,6},{20,30},2,2,1),{1,30})\r
+\r
+-- whereas insertvalues works like table.insert, but inserts a range of values\r
+-- from the given table.\r
+asserteq(insertvalues({1,2,3,4,5,6},2,{20,30}),{1,20,30,2,3,4,5,6})\r
+asserteq(insertvalues({1,2},{3,4}),{1,2,3,4})\r
+\r
+-- the 4th arg of move and icopy gives the start index in the source table\r
+asserteq(move({1,2,3,4,5,6},{20,30},2,2),{1,30,3,4,5,6})\r
+asserteq(icopy({1,2,3,4,5,6},{20,30},2,2),{1,30})\r
+\r
+t = {1,2,3,4,5,6}\r
+move(t,{20,30},2,1,1)\r
+asserteq(t,{1,20,3,4,5,6})\r
+set(t,0,2,3)\r
+asserteq(t,{1,0,0,4,5,6})\r
+insertvalues(t,1,{10,20})\r
+asserteq(t,{10,20,1,0,0,4,5,6})\r
+\r
+\r
--- /dev/null
+local T = require 'pl.text'\r
+local Template = T.Template\r
+local asserteq = require 'pl.test'.asserteq\r
+\r
+local t1 = Template [[\r
+while true do\r
+ $contents\r
+end\r
+]]\r
+\r
+assert(t1:substitute {contents = 'print "hello"'},[[\r
+while true do\r
+ print "hello"\r
+end\r
+]])\r
+\r
+assert(t1:indent_substitute {contents = [[\r
+for i = 1,10 do\r
+ gotcha(i)\r
+end\r
+]]},[[\r
+while true do\r
+ for i = 1,10 do\r
+ gotcha(i)\r
+ end\r
+end\r
+]])\r
+\r
+asserteq(T.dedent [[\r
+ one\r
+ two\r
+ three\r
+]],[[\r
+one\r
+two\r
+three\r
+]])\r
+asserteq(T.fill ([[\r
+It is often said of Lua that it does not include batteries. That is because the goal of Lua is to produce a lean expressive language that will be used on all sorts of machines, (some of which don't even have hierarchical filesystems). The Lua language is the equivalent of an operating system kernel; the creators of Lua do not see it as their responsibility to create a full software ecosystem around the language. That is the role of the community.\r
+]],20),[[\r
+It is often said of Lua\r
+that it does not include\r
+batteries. That is because\r
+the goal of Lua is to\r
+produce a lean expressive\r
+language that will be\r
+used on all sorts of machines,\r
+(some of which don't\r
+even have hierarchical\r
+filesystems). The Lua\r
+language is the equivalent\r
+of an operating system\r
+kernel; the creators of\r
+Lua do not see it as their\r
+responsibility to create\r
+a full software ecosystem\r
+around the language. That\r
+is the role of the community.\r
+]])\r
+\r
+local template = require 'pl.template'\r
+\r
+local t = [[\r
+# for i = 1,3 do\r
+ print($(i+1))\r
+# end\r
+]]\r
+\r
+asserteq(template.substitute(t),[[\r
+ print(2)\r
+ print(3)\r
+ print(4)\r
+]])\r
+\r
+t = [[\r
+> for i = 1,3 do\r
+ print(${i+1})\r
+> end\r
+]]\r
+\r
+asserteq(template.substitute(t,{_brackets='{}',_escape='>'}),[[\r
+ print(2)\r
+ print(3)\r
+ print(4)\r
+]])\r
+\r
+t = [[\r
+# for k,v in pairs(T) do\r
+ "$(k)", -- $(v)\r
+# end\r
+]]\r
+\r
+local Tee = {Dog = 'Bonzo', Cat = 'Felix', Lion = 'Leo'}\r
+\r
+asserteq(template.substitute(t,{T=Tee,_parent=_G}),[[\r
+ "Dog", -- Bonzo\r
+ "Cat", -- Felix\r
+ "Lion", -- Leo\r
+]])\r
+\r
+-- for those with a fondness for Python-style % formatting...\r
+T.format_operator()\r
+asserteq('[%s]' % 'home', '[home]')\r
+asserteq('%s = %d' % {'fred',42},'fred = 42')\r
+\r
+-- mostly works like string.format, except that %s forces use of tostring()\r
+-- rather than throwing an error\r
+local List = require 'pl.List'\r
+asserteq('TBL:%s' % List{1,2,3},'TBL:{1,2,3}')\r
+\r
+-- table with keys and format with $\r
+asserteq('<$one>' % {one=1}, '<1>')\r
+-- (second arg may also be a function, like os.getenv)\r
+function subst(k)\r
+ if k == 'A' then return 'ay'\r
+ elseif k == 'B' then return 'bee'\r
+ else return '?'\r
+ end\r
+end\r
+asserteq(\r
+ '$A & $B' % subst,'ay & bee'\r
+)\r
+\r
+\r
+\r
--- /dev/null
+---- deriving specialized classes from List
+-- illustrating covariance of List methods
+require 'pl'
+local L = utils.string_lambda
+local asserteq = test.asserteq
+
+class.Vector(List)
+
+
+function Vector.range (x1,x2,delta)
+ return Vector(List.range(x1,x2,delta))
+end
+
+local function vbinop (op,v1,v2,scalar)
+ if not Vector:class_of(v1) then
+ v2, v1 = v1, v2
+ end
+ if type(v2) ~= 'table' then
+ return v1:map(op,v2)
+ else
+ if scalar then error("operation not permitted on two vectors",3) end
+ if #v1 ~= #v2 then error("vectors have different lengths",3) end
+ return v1:map2(op,v2)
+ end
+end
+
+function Vector.__add (v1,v2)
+ return vbinop(operator.add,v1,v2)
+end
+
+function Vector.__sub (v1,v2)
+ return vbinop(operator.sub,v1,v2)
+end
+
+function Vector.__mul (v1,v2)
+ return vbinop(operator.mul,v1,v2,true)
+end
+
+function Vector.__div (v1,v2)
+ return vbinop(operator.div,v1,v2,true)
+end
+
+function Vector.__unm (v)
+ return v:map(operator.unm)
+end
+
+Vector.catch(List.default_map_with(math))
+
+v = Vector()
+
+assert(v:is_a(Vector))
+assert(Vector:class_of(v))
+
+v:append(10)
+v:append(20)
+asserteq(1+v,v+1)
+
+-- covariance: the inherited Vector.map returns a Vector
+asserteq(List{1,2} + v:map (L'2*_'),{21,42})
+
+u = Vector{1,2}
+
+asserteq(v + u,{11,22})
+asserteq(v - u,{9,18})
+asserteq (v - 1, {9,19})
+asserteq(2 * v, {20,40})
+-- print(v * v) -- throws error: not permitted
+-- print(v + Vector{1,2,3}) -- throws error: different lengths
+asserteq(2*v + u, {21,42})
+asserteq(-v, {-10,-20})
+
+-- Vector.slice returns the Right Thing due to covariance
+asserteq(
+ Vector.range(0,1,0.1):slice(1,3)+1,
+ {1,1.1,1.2},
+ 1e-8)
+
+u:transform '_+1'
+asserteq(u,{2,3})
+
+u = Vector.range(0,1,0.1)
+asserteq(
+ u:map(math.sin),
+ {0,0.0998,0.1986,0.2955,0.3894,0.4794,0.5646,0.6442,0.7173,0.7833,0.8414},
+0.001)
+
+-- unknown Vector methods are assumed to be math.* functions
+asserteq(Vector{-1,2,3,-4}:abs(),Vector.range(1,4))
+
+local R = Vector.range
+
+-- concatenating two Vectors returns another vector (covariance again)
+-- note the operator precedence here...
+asserteq((
+ R(0,1,0.1)..R(1.2,2,0.2)) + 1,
+ {1,1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,2,2.2,2.4,2.6,2.8,3},
+ 1e-8)
+
+
+class.Strings(List)
+
+Strings.catch(List.default_map_with(string))
+
+ls = Strings{'one','two','three'}
+asserteq(ls:upper(),{'ONE','TWO','THREE'})
+asserteq(ls:sub(1,2),{'on','tw','th'})
+
+-- an issue with covariance: not all map operations on specialized lists
+-- results in another list of that type!
+local sizes = ls:map '#'
+asserteq(sizes, {3,3,5})
+asserteq(utils.type(sizes),'Strings')
+
+
+
--- /dev/null
+local xml = require 'pl.xml'
+local asserteq = require 'pl.test'.asserteq
+local dump = require 'pl.pretty'.dump
+
+-- Prosody stanza.lua style XML building
+
+d = xml.new 'top' : addtag 'child' : text 'alice' : up() : addtag 'child' : text 'bob'
+
+d = xml.new 'children' :
+ addtag 'child' :
+ addtag 'name' : text 'alice' : up() : addtag 'age' : text '5' : up() : addtag('toy',{type='fluffy'}) : up() :
+ up() :
+ addtag 'child':
+ addtag 'name' : text 'bob' : up() : addtag 'age' : text '6' : up() : addtag('toy',{type='squeaky'})
+
+asserteq(
+xml.tostring(d,'',' '),
+[[
+
+<children>
+ <child>
+ <name>alice</name>
+ <age>5</age>
+ <toy type='fluffy'/>
+ </child>
+ <child>
+ <name>bob</name>
+ <age>6</age>
+ <toy type='squeaky'/>
+ </child>
+</children>]])
+
+-- Orbit-style 'xmlification'
+
+local children,child,toy,name,age = xml.tags 'children, child, toy, name, age'
+
+d1 = children {
+ child {name 'alice', age '5', toy {type='fluffy'}},
+ child {name 'bob', age '6', toy {type='squeaky'}}
+}
+
+assert(xml.compare(d,d1))
+
+-- or we can use a template document to convert Lua data to LOM
+
+templ = child {name '$name', age '$age', toy{type='$toy'}}
+
+d2 = children(templ:subst{
+ {name='alice',age='5',toy='fluffy'},
+ {name='bob',age='6',toy='squeaky'}
+})
+
+assert(xml.compare(d1,d2))
+
+-- Parsing Google Weather service results --
+
+local joburg = [[
+<xml_api_reply version='1'>
+ <weather module_id='0' tab_id='0' mobile_zipped='1' section='0' row='0' mobile_row='0'>
+ <forecast_information>
+ <city data='Johannesburg, Gauteng'/>
+ <postal_code data='Johannesburg,ZA'/>
+ <latitude_e6 data=''/>
+ <longitude_e6 data=''/>
+ <forecast_date data='2010-10-02'/>
+ <current_date_time data='2010-10-02 18:30:00 +0000'/>
+ <unit_system data='US'/>
+ </forecast_information>
+ <current_conditions>
+ <condition data='Clear'/>
+ <temp_f data='75'/>
+ <temp_c data='24'/>
+ <humidity data='Humidity: 19%'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <wind_condition data='Wind: NW at 7 mph'/>
+ </current_conditions>
+ <forecast_conditions>
+ <day_of_week data='Sat'/>
+ <low data='60'/>
+ <high data='89'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <condition data='Clear'/>
+ </forecast_conditions>
+ <forecast_conditions>
+ <day_of_week data='Sun'/>
+ <low data='53'/>
+ <high data='86'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <condition data='Clear'/>
+ </forecast_conditions>
+ <forecast_conditions>
+ <day_of_week data='Mon'/>
+ <low data='57'/>
+ <high data='87'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <condition data='Clear'/>
+ </forecast_conditions>
+ <forecast_conditions>
+ <day_of_week data='Tue'/>
+ <low data='60'/>
+ <high data='84'/>
+ <icon data='/ig/images/weather/sunny.gif'/>
+ <condition data='Clear'/>
+ </forecast_conditions>
+ </weather>
+</xml_api_reply>
+
+]]
+
+local d = xml.parse(joburg)
+
+function match(t,xpect)
+ local res,ret = d:match(t)
+ asserteq(res,xpect)
+end
+
+t1 = [[
+ <weather>
+ <current_conditions>
+ <condition data='$condition'/>
+ <temp_c data='$temp'/>
+ </current_conditions>
+ </weather>
+]]
+
+match(t1,{
+ condition = "Clear",
+ temp = "24",
+} )
+
+t2 = [[
+ <weather>
+ {{<forecast_conditions>
+ <day_of_week data='$day'/>
+ <low data='$low'/>
+ <high data='$high'/>
+ <condition data='$condition'/>
+ </forecast_conditions>}}
+ </weather>
+]]
+
+match(t2,{
+ {
+ low = "60",
+ high = "89",
+ day = "Sat",
+ condition = "Clear",
+ },
+ {
+ low = "53",
+ high = "86",
+ day = "Sun",
+ condition = "Clear",
+ },
+ {
+ low = "57",
+ high = "87",
+ day = "Mon",
+ condition = "Clear",
+ },
+ {
+ low = "60",
+ high = "84",
+ day = "Tue",
+ condition = "Clear",
+ }
+})
+
+config = [[
+<config>
+ <alpha>1.3</alpha>
+ <beta>10</beta>
+ <name>bozo</name>
+</config>
+]]
+d,err = xml.parse(config)
+if not d then print(err); os.exit(1) end
+
+
+-- can match against wildcard tag names (end with -)
+-- can be names
+match([[
+<config>
+ {{<key->$value</key->}}
+</config>
+]],{
+ {key="alpha", value = "1.3"},
+ {key="beta", value = "10"},
+ {key="name",value = "bozo"},
+})
+
+-- can be numerical indices
+match([[
+<config>
+ {{<1->$2</1->}}
+</config>
+]],{
+ {"alpha","1.3"},
+ {"beta","10"},
+ {"name","bozo"},
+})
+
+-- _ is special; means 'this value is key of captured table'
+match([[
+<config>
+ {{<_->$1</_->}}
+</config>
+]],{
+ alpha = {"1.3"},
+ beta = {"10"},
+ name = {"bozo"},
+})
+
+-- the numerical index 0 is special: a capture of {[0]=val} becomes simply the value val
+match([[
+<config>
+ {{<_->$0</_->}}
+</config>
+]],{
+ alpha = "1.3",
+ name = "bozo",
+ beta = "10"
+})
+
+-- this can of course also work with attributes, but then we don't want to collapse!
+
+config = [[
+<config>
+ <alpha type='number'>1.3</alpha>
+ <beta type='number'>10</beta>
+ <name type='string'>bozo</name>
+</config>
+]]
+d,err = xml.parse(config)
+if not d then print(err); os.exit(1) end
+
+match([[
+<config>
+ {{<_- type='$1'>$2</_->}}
+</config>
+]],{
+ alpha = {"number","1.3"},
+ beta = {"number","10"},
+ name = {"string","bozo"},
+})
+
+d,err = xml.parse [[
+
+<configuremap>
+ <configure name="NAME" value="ImageMagick"/>
+ <configure name="LIB_VERSION" value="0x651"/>
+ <configure name="LIB_VERSION_NUMBER" value="6,5,1,3"/>
+ <configure name="RELEASE_DATE" value="2009-05-01"/>
+ <configure name="VERSION" value="6.5.1"/>
+ <configure name="CC" value="vs7"/>
+ <configure name="HOST" value="windows-unknown-linux-gnu"/>
+ <configure name="DELEGATES" value="bzlib freetype jpeg jp2 lcms png tiff x11 xml wmf zlib"/>
+ <configure name="COPYRIGHT" value="Copyright (C) 1999-2009 ImageMagick Studio LLC"/>
+ <configure name="WEBSITE" value="http://www.imagemagick.org"/>
+
+</configuremap>
+]]
+if not d then print(err); os.exit(1) end
+--xml.debug = true
+
+res,err = d:match [[
+<configuremap>
+ {{<configure name="$_" value="$0"/>}}
+</configuremap>
+]]
+
+asserteq(res,{
+ HOST = "windows-unknown-linux-gnu",
+ COPYRIGHT = "Copyright (C) 1999-2009 ImageMagick Studio LLC",
+ NAME = "ImageMagick",
+ LIB_VERSION = "0x651",
+ VERSION = "6.5.1",
+ RELEASE_DATE = "2009-05-01",
+ WEBSITE = "http://www.imagemagick.org",
+ LIB_VERSION_NUMBER = "6,5,1,3",
+ CC = "vs7",
+ DELEGATES = "bzlib freetype jpeg jp2 lcms png tiff x11 xml wmf zlib"
+})
+
+-- short excerpt from
+-- /usr/share/mobile-broadband-provider-info/serviceproviders.xml
+
+d = xml.parse [[
+<serviceproviders format="2.0">
+<country code="za">
+ <provider>
+ <name>Cell-c</name>
+ <gsm>
+ <network-id mcc="655" mnc="07"/>
+ <apn value="internet">
+ <username>Cellcis</username>
+ <dns>196.7.0.138</dns>
+ <dns>196.7.142.132</dns>
+ </apn>
+ </gsm>
+ </provider>
+ <provider>
+ <name>MTN</name>
+ <gsm>
+ <network-id mcc="655" mnc="10"/>
+ <apn value="internet">
+ <dns>196.11.240.241</dns>
+ <dns>209.212.97.1</dns>
+ </apn>
+ </gsm>
+ </provider>
+ <provider>
+ <name>Vodacom</name>
+ <gsm>
+ <network-id mcc="655" mnc="01"/>
+ <apn value="internet">
+ <dns>196.207.40.165</dns>
+ <dns>196.43.46.190</dns>
+ </apn>
+ <apn value="unrestricted">
+ <name>Unrestricted</name>
+ <dns>196.207.32.69</dns>
+ <dns>196.43.45.190</dns>
+ </apn>
+ </gsm>
+ </provider>
+ <provider>
+ <name>Virgin Mobile</name>
+ <gsm>
+ <apn value="vdata">
+ <dns>196.7.0.138</dns>
+ <dns>196.7.142.132</dns>
+ </apn>
+ </gsm>
+ </provider>
+</country>
+
+</serviceproviders>
+]]
+
+res = d:match [[
+ <serviceproviders>
+ {{<country code="$_">
+ {{<provider>
+ <name>$0</name>
+ </provider>}}
+ </country>}}
+ </serviceproviders>
+]]
+
+asserteq(res,{
+ za = {
+ "Cell-c",
+ "MTN",
+ "Vodacom",
+ "Virgin Mobile"
+ }
+})
+
+res = d:match [[
+<serviceproviders>
+ <country code="$country">
+ <provider>
+ <name>$name</name>
+ <gsm>
+ <apn value="$apn">
+ <dns>196.43.46.190</dns>
+ </apn>
+ </gsm>
+ </provider>
+ </country>
+</serviceproviders>
+]]
+
+asserteq(res,{
+ name = "Vodacom",
+ country = "za",
+ apn = "internet"
+})
+
+-- can always use xmlification to generate your templates...
+
+local SP, country, provider, gsm, apn, dns = xml.tags 'serviceprovider, country, provider, gsm, apn, dns'
+
+t = SP{country{code="$country",provider{
+ name '$name', gsm{apn {value="$apn",dns '196.43.46.190'}}
+ }}}
+
+print(xml.tostring(t,' ',' '))
--- /dev/null
+require 'pl'\r
+\r
+norm = path.normpath\r
+\r
+p = norm '/a/b'\r
+\r
+assert(norm '/a/fred/../b' == p)\r
+assert(norm '/a//b' == p)\r
+\r
+if path.is_windows then\r
+ assert(norm [[\a\.\b]] == p)\r
+end\r
+\r
projects / e2factory.git / commitdiff