mirror of
https://github.com/FlightControl-Master/MOOSE.git
synced 2025-08-15 10:47:21 +00:00
Updates to lauch files, add Utils with 7-Zip and lua
Known bug : .lauch file Generate doc doesn't work, it needs an update to luadocumentor.bat
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
Metalua Compiler
|
||||
================
|
||||
|
||||
## Metalua compiler
|
||||
|
||||
This module `metalua-compiler` depends on `metalua-parser`. Its main
|
||||
feature is to compile ASTs into Lua 5.1 bytecode, allowing to convert
|
||||
them into bytecode files and executable functions. This opens the
|
||||
following possibilities:
|
||||
|
||||
* compiler objects generated with `require 'metalua.compiler'.new()`
|
||||
support methods `:xxx_to_function()` and `:xxx_to_bytecode()`;
|
||||
|
||||
* Compile-time meta-programming: use of `-{...}` splices in source
|
||||
code, to generate code during compilation;
|
||||
|
||||
* Some syntax extensions, such as structural pattern matching and
|
||||
lists by comprehension;
|
||||
|
||||
* Some AST manipulation facilities such as `treequery`, which are
|
||||
implemented with Metalua syntax extensions.
|
||||
|
||||
## What's new in Metalua 0.7
|
||||
|
||||
This is a major overhaul of the compiler's architecture. Some of the
|
||||
most noteworthy changes are:
|
||||
|
||||
* No more installation or bootstrap script. Some Metalua source files
|
||||
have been rewritten in plain Lua, and module sources have been
|
||||
refactored, so that if you just drop the `metalua` folder somewhere
|
||||
in your `LUA_PATH`, it works.
|
||||
|
||||
* The compiler can be cut in two parts:
|
||||
|
||||
* a parser which generates ASTs out of Lua sources, and should be
|
||||
either portable or easily ported to Lua 5.2;
|
||||
|
||||
* a compiler, which can turn sources and AST into executable
|
||||
Lua 5.1 bytecode and run it. It also supports compile-time
|
||||
meta-programming, i.e. code included between `-{ ... }` is
|
||||
executed during compilation, and the ASTs it produces are
|
||||
included in the resulting bytecode.
|
||||
|
||||
* Both parts are packaged as separate LuaRocks, `metalua-parser` and
|
||||
`metalua-compiler` respectively, so that you can install the former
|
||||
without the latter.
|
||||
|
||||
* The parser is not a unique object anymore. Instead,
|
||||
`require "metalua.compiler".new()` returns a different compiler
|
||||
instance every time it's called. Compiler instances can be reused on
|
||||
as many source files as wanted, but extending one instance's grammar
|
||||
doesn't affect other compiler instances.
|
||||
|
||||
* Included standard library has been shed. There are too many standard
|
||||
libs in Lua, and none of them is standard enough, offering
|
||||
yet-another-one, coupled with a specific compiler can only add to
|
||||
confusion.
|
||||
|
||||
* Many syntax extensions, which either were arguably more code samples
|
||||
than actual production-ready tools, or relied too heavily on the
|
||||
removed runtime standard libraries, have been removed.
|
||||
|
||||
* The remaining libraries and samples are:
|
||||
|
||||
* `metalua.compiler` converts sources into ASTs, bytecode,
|
||||
functions, and ASTs back into sources.
|
||||
|
||||
* `metalua` compiles and/or executes files from the command line,
|
||||
can start an interactive REPL session.
|
||||
|
||||
* `metalua.loader` adds a package loader which allows to use modules
|
||||
written in Metalua, even from a plain Lua program.
|
||||
|
||||
* `metalua.treequery` is an advanced DSL allowing to search ASTs in
|
||||
a smart way, e.g. "_search `return` statements which return a
|
||||
`local` variable but aren't in a nested `function`_".
|
||||
|
||||
* `metalua.extension.comprehension` is a language extension which
|
||||
supports lists by comprehension
|
||||
(`even = { i for i=1, 100 if i%2==0 }`) and improved loops
|
||||
(`for i=1, 10 for j=1,10 if i~=j do print(i,j) end`).
|
||||
|
||||
* `metalua.extension.match` is a language extension which offers
|
||||
Haskell/ML structural pattern matching
|
||||
(``match AST with `Function{ args, body } -> ... | `Number{ 0 } -> ...end``)
|
||||
|
||||
* **TODO Move basic extensions in a separate module.**
|
||||
|
||||
* To remove the compilation speed penalty associated with
|
||||
metaprogramming, when environment variable `LUA_MCACHE` or Lua
|
||||
variable `package.mcache` is defined and LuaFileSystem is available,
|
||||
the results of Metalua source compilations is cached. Unless the
|
||||
source file is more recent than the latest cached bytecode file, the
|
||||
latter is loaded instead of the former.
|
||||
|
||||
* The Luarock install for the full compiler lists dependencies towards
|
||||
Readline, LuaFileSytem, and Alt-Getopts. Those projects are
|
||||
optional, but having them automatically installed by LuaRocks offers
|
||||
a better user experience.
|
||||
|
||||
* The license has changed from MIT to double license MIT + EPL. This
|
||||
has been done in order to provide the IP guarantees expected by the
|
||||
Eclipse Foundation, to include Metalua in Eclipse's
|
||||
[Lua Development Tools](http://www.eclipse.org/koneki/ldt/).
|
||||
@@ -0,0 +1,177 @@
|
||||
Metalua Parser
|
||||
==============
|
||||
|
||||
`metalua-parser` is a subset of the Metalua compiler, which turns
|
||||
valid Lua source files and strings into abstract syntax trees
|
||||
(AST). This README includes a description of this AST format. People
|
||||
interested by Lua code analysis and generation are encouraged to
|
||||
produce and/or consume this format to represent ASTs.
|
||||
|
||||
It has been designed for Lua 5.1. It hasn't been tested against
|
||||
Lua 5.2, but should be easily ported.
|
||||
|
||||
## Usage
|
||||
|
||||
Module `metalua.compiler` has a `new()` function, which returns a
|
||||
compiler instance. This instance has a set of methods of the form
|
||||
`:xxx_to_yyy(input)`, where `xxx` and `yyy` must be one of the
|
||||
following:
|
||||
|
||||
* `srcfile` the name of a Lua source file;
|
||||
* `src` a string containing the Lua sources of a list of statements;
|
||||
* `lexstream` a lexical tokens stream;
|
||||
* `ast` an abstract syntax tree;
|
||||
* `bytecode` a chunk of Lua bytecode that can be loaded in a Lua 5.1
|
||||
VM (not available if you only installed the parser);
|
||||
* `function` an executable Lua function.
|
||||
|
||||
Compiling into bytecode or executable functions requires the whole
|
||||
Metalua compiler, not only the parser. The most frequently used
|
||||
functions are `:src_to_ast(source_string)` and
|
||||
`:srcfile_to_ast("path/to/source/file.lua")`.
|
||||
|
||||
mlc = require 'metalua.compiler'.new()
|
||||
ast = mlc :src_to_ast[[ return 123 ]]
|
||||
|
||||
A compiler instance can be reused as much as you want; it's only
|
||||
interesting to work with more than one compiler instance when you
|
||||
start extending their grammars.
|
||||
|
||||
## Abstract Syntax Trees definition
|
||||
|
||||
### Notation
|
||||
|
||||
Trees are written below with some Metalua syntax sugar, which
|
||||
increases their readability. the backquote symbol introduces a `tag`,
|
||||
i.e. a string stored in the `"tag"` field of a table:
|
||||
|
||||
* `` `Foo{ 1, 2, 3 }`` is a shortcut for `{tag="Foo", 1, 2, 3}`;
|
||||
* `` `Foo`` is a shortcut for `{tag="Foo"}`;
|
||||
* `` `Foo 123`` is a shortcut for `` `Foo{ 123 }``, and therefore
|
||||
`{tag="Foo", 123 }`; the expression after the tag must be a literal
|
||||
number or string.
|
||||
|
||||
When using a Metalua interpreter or compiler, the backtick syntax is
|
||||
supported and can be used directly. Metalua's pretty-printing helpers
|
||||
also try to use backtick syntax whenever applicable.
|
||||
|
||||
### Tree elements
|
||||
|
||||
Tree elements are mainly categorized into statements `stat`,
|
||||
expressions `expr` and lists of statements `block`. Auxiliary
|
||||
definitions include function applications/method invocation `apply`,
|
||||
are both valid statements and expressions, expressions admissible on
|
||||
the left-hand-side of an assignment statement `lhs`.
|
||||
|
||||
block: { stat* }
|
||||
|
||||
stat:
|
||||
`Do{ stat* }
|
||||
| `Set{ {lhs+} {expr+} } -- lhs1, lhs2... = e1, e2...
|
||||
| `While{ expr block } -- while e do b end
|
||||
| `Repeat{ block expr } -- repeat b until e
|
||||
| `If{ (expr block)+ block? } -- if e1 then b1 [elseif e2 then b2] ... [else bn] end
|
||||
| `Fornum{ ident expr expr expr? block } -- for ident = e, e[, e] do b end
|
||||
| `Forin{ {ident+} {expr+} block } -- for i1, i2... in e1, e2... do b end
|
||||
| `Local{ {ident+} {expr+}? } -- local i1, i2... = e1, e2...
|
||||
| `Localrec{ ident expr } -- only used for 'local function'
|
||||
| `Goto{ <string> } -- goto str
|
||||
| `Label{ <string> } -- ::str::
|
||||
| `Return{ <expr*> } -- return e1, e2...
|
||||
| `Break -- break
|
||||
| apply
|
||||
|
||||
expr:
|
||||
`Nil | `Dots | `True | `False
|
||||
| `Number{ <number> }
|
||||
| `String{ <string> }
|
||||
| `Function{ { ident* `Dots? } block }
|
||||
| `Table{ ( `Pair{ expr expr } | expr )* }
|
||||
| `Op{ opid expr expr? }
|
||||
| `Paren{ expr } -- significant to cut multiple values returns
|
||||
| apply
|
||||
| lhs
|
||||
|
||||
apply:
|
||||
`Call{ expr expr* }
|
||||
| `Invoke{ expr `String{ <string> } expr* }
|
||||
|
||||
ident: `Id{ <string> }
|
||||
|
||||
lhs: ident | `Index{ expr expr }
|
||||
|
||||
opid: 'add' | 'sub' | 'mul' | 'div'
|
||||
| 'mod' | 'pow' | 'concat'| 'eq'
|
||||
| 'lt' | 'le' | 'and' | 'or'
|
||||
| 'not' | 'len'
|
||||
|
||||
### Meta-data (lineinfo)
|
||||
|
||||
|
||||
ASTs also embed some metadata, allowing to map them to their source
|
||||
representation. Those informations are stored in a `"lineinfo"` field
|
||||
in each tree node, which points to the range of characters in the
|
||||
source string which represents it, and to the content of any comment
|
||||
that would appear immediately before or after that node.
|
||||
|
||||
Lineinfo objects have two fields, `"first"` and `"last"`, describing
|
||||
respectively the beginning and the end of the subtree in the
|
||||
sources. For instance, the sub-node ``Number{123}` produced by parsing
|
||||
`[[return 123]]` will have `lineinfo.first` describing offset 8, and
|
||||
`lineinfo.last` describing offset 10:
|
||||
|
||||
|
||||
> mlc = require 'metalua.compiler'.new()
|
||||
> ast = mlc :src_to_ast "return 123 -- comment"
|
||||
> print(ast[1][1].lineinfo)
|
||||
<?|L1|C8-10|K8-10|C>
|
||||
>
|
||||
|
||||
A lineinfo keeps track of character offsets relative to the beginning
|
||||
of the source string/file ("K8-10" above), line numbers (L1 above; a
|
||||
lineinfo spanning on several lines would read something like "L1-10"),
|
||||
columns i.e. offset within the line ("C8-10" above), and a filename if
|
||||
available (the "?" mark above indicating that we have no file name, as
|
||||
the AST comes from a string). The final "|C>" indicates that there's a
|
||||
comment immediately after the node; an initial "<C|" would have meant
|
||||
that there was a comment immediately before the node.
|
||||
|
||||
Positions represent either the end of a token and the beginning of an
|
||||
inter-token space (`"last"` fields) or the beginning of a token, and
|
||||
the end of an inter-token space (`"first"` fields). Inter-token spaces
|
||||
might be empty. They can also contain comments, which might be useful
|
||||
to link with surrounding tokens and AST subtrees.
|
||||
|
||||
Positions are chained with their "dual" one: a position at the
|
||||
beginning of and inter-token space keeps a refernce to the position at
|
||||
the end of that inter-token space in its `"facing"` field, and
|
||||
conversly, end-of-inter-token positions keep track of the inter-token
|
||||
space beginning, also in `"facing"`. An inter-token space can be
|
||||
empty, e.g. in `"2+2"`, in which case `lineinfo==lineinfo.facing`.
|
||||
|
||||
Comments are also kept in the `"comments"` field. If present, this
|
||||
field contains a list of comments, with a `"lineinfo"` field
|
||||
describing the span between the first and last comment. Each comment
|
||||
is represented by a list of one string, with a `"lineinfo"` describing
|
||||
the span of this comment only. Consecutive lines of `--` comments are
|
||||
considered as one comment: `"-- foo\n-- bar\n"` parses as one comment
|
||||
whose text is `"foo\nbar"`, whereas `"-- foo\n\n-- bar\n"` parses as
|
||||
two comments `"foo"` and `"bar"`.
|
||||
|
||||
So for instance, if `f` is the AST of a function and I want to
|
||||
retrieve the comment before the function, I'd do:
|
||||
|
||||
f_comment = f.lineinfo.first.comments[1][1]
|
||||
|
||||
The informations in lineinfo positions, i.e. in each `"first"` and
|
||||
`"last"` field, are held in the following fields:
|
||||
|
||||
* `"source"` the filename (optional);
|
||||
* `"offset"` the 1-based offset relative to the beginning of the string/file;
|
||||
* `"line"` the 1-based line number;
|
||||
* `"column"` the 1-based offset within the line;
|
||||
* `"facing"` the position at the opposite end of the inter-token space.
|
||||
* `"comments"` the comments in the associated inter-token space (optional).
|
||||
* `"id"` an arbitrary number, which uniquely identifies an inter-token
|
||||
space within a given tokens stream.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
Metalua
|
||||
=======
|
||||
|
||||
Metalua is a Lua code analysis tool, as well as a compiler for a
|
||||
superset of Lua 5.1 supporting Compile-Time Meta-Programming. It's
|
||||
separated into two LuaRocks, `metalua-parser` and
|
||||
`metalua-compiler`. The documentation of each rock can be found in
|
||||
`README-parser.md` and `README-compiler.md`.
|
||||
|
||||
All the code in Metalue is released under dual lincenses:
|
||||
|
||||
* MIT public license (same as Lua);
|
||||
* EPL public license (same as Eclipse).
|
||||
@@ -0,0 +1,38 @@
|
||||
--*-lua-*--
|
||||
package = "metalua-parser"
|
||||
version = "0.7.3-2"
|
||||
source = {
|
||||
url = "http://git.eclipse.org/c/koneki/org.eclipse.koneki.metalua.git/snapshot/org.eclipse.koneki.metalua-v0.7.3.tar.gz"
|
||||
}
|
||||
description = {
|
||||
summary = "Metalua's parser: converting Lua source strings and files into AST",
|
||||
detailed = [[
|
||||
This is a subset of the full Metalua compiler. It defines and generates an AST
|
||||
format for Lua programs, which offers a nice level of abstraction to reason about
|
||||
and manipulate Lua programs.
|
||||
]],
|
||||
homepage = "http://git.eclipse.org/c/koneki/org.eclipse.koneki.metalua.git",
|
||||
license = "EPL + MIT"
|
||||
}
|
||||
dependencies = {
|
||||
"lua >= 5.1"
|
||||
}
|
||||
build = {
|
||||
type="builtin",
|
||||
modules={
|
||||
["metalua.grammar.generator"] = "metalua/grammar/generator.lua",
|
||||
["metalua.grammar.lexer"] = "metalua/grammar/lexer.lua",
|
||||
["metalua.compiler.parser"] = "metalua/compiler/parser.lua",
|
||||
["metalua.compiler.parser.table"] = "metalua/compiler/parser/table.lua",
|
||||
["metalua.compiler.parser.ext"] = "metalua/compiler/parser/ext.lua",
|
||||
["metalua.compiler.parser.annot.generator"] = "metalua/compiler/parser/annot/generator.lua",
|
||||
["metalua.compiler.parser.annot.grammar"] = "metalua/compiler/parser/annot/grammar.lua",
|
||||
["metalua.compiler.parser.stat"] = "metalua/compiler/parser/stat.lua",
|
||||
["metalua.compiler.parser.misc"] = "metalua/compiler/parser/misc.lua",
|
||||
["metalua.compiler.parser.lexer"] = "metalua/compiler/parser/lexer.lua",
|
||||
["metalua.compiler.parser.meta"] = "metalua/compiler/parser/meta.lua",
|
||||
["metalua.compiler.parser.expr"] = "metalua/compiler/parser/expr.lua",
|
||||
["metalua.compiler"] = "metalua/compiler.lua",
|
||||
["metalua.pprint"] = "metalua/pprint.lua",
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
rock_manifest = {
|
||||
doc = {
|
||||
["README-compiler.md"] = "292523d759247d210d32fb2f6153e0f4",
|
||||
["README-parser.md"] = "b44e3673d96dd296f2c0e92a6c87ee18",
|
||||
["README.md"] = "20bfb490cddef9e101e44688791abcda"
|
||||
},
|
||||
lua = {
|
||||
metalua = {
|
||||
compiler = {
|
||||
parser = {
|
||||
annot = {
|
||||
["generator.lua"] = "d86f7507d66ba6a3692a6f8611e9939b",
|
||||
["grammar.lua"] = "7d195bde7992efd9923771751b67b18f"
|
||||
},
|
||||
["expr.lua"] = "3a0b1984a6f92280e2e63b074fdcec10",
|
||||
["ext.lua"] = "a99e31a07bc390b826f6653bcc47d89b",
|
||||
["lexer.lua"] = "eac0f9d475d9dae4ea5a2724014cebec",
|
||||
["meta.lua"] = "12870bceda6395695020b739196e2a92",
|
||||
["misc.lua"] = "49d59f4fc1bfb77b36f78d4f87ae258f",
|
||||
["stat.lua"] = "83f10ac899be12ca4df58bbe8645299f",
|
||||
["table.lua"] = "5d2389e89603b7f78c731e6918aa1a9b"
|
||||
},
|
||||
["parser.lua"] = "e6ae68ce200de8071bb0fefad97f9b79"
|
||||
},
|
||||
["compiler.lua"] = "ca65ee9a3053581f4315821a31d0c1fd",
|
||||
grammar = {
|
||||
["generator.lua"] = "b8a29e817d6798c12f40a230a0f6d0af",
|
||||
["lexer.lua"] = "7cb7c835479a9be884130eaacb9be60a"
|
||||
},
|
||||
["pprint.lua"] = "0b9bd8757b45c2d4be30106abcbd45b2"
|
||||
}
|
||||
},
|
||||
["metalua-parser-0.7.3-2.rockspec"] = "a56680900b0b51701db7cd7abf49af92"
|
||||
}
|
||||
Reference in New Issue
Block a user