mirror of
https://github.com/FlightControl-Master/MOOSE.git
synced 2025-10-29 16:58:06 +00:00
Updates to lauch files, add Utils with 7-Zip and lua
Known bug : .lauch file Generate doc doesn't work, it needs an update to luadocumentor.bat
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
Metalua Compiler
|
||||
================
|
||||
|
||||
## Metalua compiler
|
||||
|
||||
This module `metalua-compiler` depends on `metalua-parser`. Its main
|
||||
feature is to compile ASTs into Lua 5.1 bytecode, allowing to convert
|
||||
them into bytecode files and executable functions. This opens the
|
||||
following possibilities:
|
||||
|
||||
* compiler objects generated with `require 'metalua.compiler'.new()`
|
||||
support methods `:xxx_to_function()` and `:xxx_to_bytecode()`;
|
||||
|
||||
* Compile-time meta-programming: use of `-{...}` splices in source
|
||||
code, to generate code during compilation;
|
||||
|
||||
* Some syntax extensions, such as structural pattern matching and
|
||||
lists by comprehension;
|
||||
|
||||
* Some AST manipulation facilities such as `treequery`, which are
|
||||
implemented with Metalua syntax extensions.
|
||||
|
||||
## What's new in Metalua 0.7
|
||||
|
||||
This is a major overhaul of the compiler's architecture. Some of the
|
||||
most noteworthy changes are:
|
||||
|
||||
* No more installation or bootstrap script. Some Metalua source files
|
||||
have been rewritten in plain Lua, and module sources have been
|
||||
refactored, so that if you just drop the `metalua` folder somewhere
|
||||
in your `LUA_PATH`, it works.
|
||||
|
||||
* The compiler can be cut in two parts:
|
||||
|
||||
* a parser which generates ASTs out of Lua sources, and should be
|
||||
either portable or easily ported to Lua 5.2;
|
||||
|
||||
* a compiler, which can turn sources and AST into executable
|
||||
Lua 5.1 bytecode and run it. It also supports compile-time
|
||||
meta-programming, i.e. code included between `-{ ... }` is
|
||||
executed during compilation, and the ASTs it produces are
|
||||
included in the resulting bytecode.
|
||||
|
||||
* Both parts are packaged as separate LuaRocks, `metalua-parser` and
|
||||
`metalua-compiler` respectively, so that you can install the former
|
||||
without the latter.
|
||||
|
||||
* The parser is not a unique object anymore. Instead,
|
||||
`require "metalua.compiler".new()` returns a different compiler
|
||||
instance every time it's called. Compiler instances can be reused on
|
||||
as many source files as wanted, but extending one instance's grammar
|
||||
doesn't affect other compiler instances.
|
||||
|
||||
* Included standard library has been shed. There are too many standard
|
||||
libs in Lua, and none of them is standard enough, offering
|
||||
yet-another-one, coupled with a specific compiler can only add to
|
||||
confusion.
|
||||
|
||||
* Many syntax extensions, which either were arguably more code samples
|
||||
than actual production-ready tools, or relied too heavily on the
|
||||
removed runtime standard libraries, have been removed.
|
||||
|
||||
* The remaining libraries and samples are:
|
||||
|
||||
* `metalua.compiler` converts sources into ASTs, bytecode,
|
||||
functions, and ASTs back into sources.
|
||||
|
||||
* `metalua` compiles and/or executes files from the command line,
|
||||
can start an interactive REPL session.
|
||||
|
||||
* `metalua.loader` adds a package loader which allows to use modules
|
||||
written in Metalua, even from a plain Lua program.
|
||||
|
||||
* `metalua.treequery` is an advanced DSL allowing to search ASTs in
|
||||
a smart way, e.g. "_search `return` statements which return a
|
||||
`local` variable but aren't in a nested `function`_".
|
||||
|
||||
* `metalua.extension.comprehension` is a language extension which
|
||||
supports lists by comprehension
|
||||
(`even = { i for i=1, 100 if i%2==0 }`) and improved loops
|
||||
(`for i=1, 10 for j=1,10 if i~=j do print(i,j) end`).
|
||||
|
||||
* `metalua.extension.match` is a language extension which offers
|
||||
Haskell/ML structural pattern matching
|
||||
(``match AST with `Function{ args, body } -> ... | `Number{ 0 } -> ...end``)
|
||||
|
||||
* **TODO Move basic extensions in a separate module.**
|
||||
|
||||
* To remove the compilation speed penalty associated with
|
||||
metaprogramming, when environment variable `LUA_MCACHE` or Lua
|
||||
variable `package.mcache` is defined and LuaFileSystem is available,
|
||||
the results of Metalua source compilations is cached. Unless the
|
||||
source file is more recent than the latest cached bytecode file, the
|
||||
latter is loaded instead of the former.
|
||||
|
||||
* The Luarock install for the full compiler lists dependencies towards
|
||||
Readline, LuaFileSytem, and Alt-Getopts. Those projects are
|
||||
optional, but having them automatically installed by LuaRocks offers
|
||||
a better user experience.
|
||||
|
||||
* The license has changed from MIT to double license MIT + EPL. This
|
||||
has been done in order to provide the IP guarantees expected by the
|
||||
Eclipse Foundation, to include Metalua in Eclipse's
|
||||
[Lua Development Tools](http://www.eclipse.org/koneki/ldt/).
|
||||
@@ -0,0 +1,177 @@
|
||||
Metalua Parser
|
||||
==============
|
||||
|
||||
`metalua-parser` is a subset of the Metalua compiler, which turns
|
||||
valid Lua source files and strings into abstract syntax trees
|
||||
(AST). This README includes a description of this AST format. People
|
||||
interested by Lua code analysis and generation are encouraged to
|
||||
produce and/or consume this format to represent ASTs.
|
||||
|
||||
It has been designed for Lua 5.1. It hasn't been tested against
|
||||
Lua 5.2, but should be easily ported.
|
||||
|
||||
## Usage
|
||||
|
||||
Module `metalua.compiler` has a `new()` function, which returns a
|
||||
compiler instance. This instance has a set of methods of the form
|
||||
`:xxx_to_yyy(input)`, where `xxx` and `yyy` must be one of the
|
||||
following:
|
||||
|
||||
* `srcfile` the name of a Lua source file;
|
||||
* `src` a string containing the Lua sources of a list of statements;
|
||||
* `lexstream` a lexical tokens stream;
|
||||
* `ast` an abstract syntax tree;
|
||||
* `bytecode` a chunk of Lua bytecode that can be loaded in a Lua 5.1
|
||||
VM (not available if you only installed the parser);
|
||||
* `function` an executable Lua function.
|
||||
|
||||
Compiling into bytecode or executable functions requires the whole
|
||||
Metalua compiler, not only the parser. The most frequently used
|
||||
functions are `:src_to_ast(source_string)` and
|
||||
`:srcfile_to_ast("path/to/source/file.lua")`.
|
||||
|
||||
mlc = require 'metalua.compiler'.new()
|
||||
ast = mlc :src_to_ast[[ return 123 ]]
|
||||
|
||||
A compiler instance can be reused as much as you want; it's only
|
||||
interesting to work with more than one compiler instance when you
|
||||
start extending their grammars.
|
||||
|
||||
## Abstract Syntax Trees definition
|
||||
|
||||
### Notation
|
||||
|
||||
Trees are written below with some Metalua syntax sugar, which
|
||||
increases their readability. the backquote symbol introduces a `tag`,
|
||||
i.e. a string stored in the `"tag"` field of a table:
|
||||
|
||||
* `` `Foo{ 1, 2, 3 }`` is a shortcut for `{tag="Foo", 1, 2, 3}`;
|
||||
* `` `Foo`` is a shortcut for `{tag="Foo"}`;
|
||||
* `` `Foo 123`` is a shortcut for `` `Foo{ 123 }``, and therefore
|
||||
`{tag="Foo", 123 }`; the expression after the tag must be a literal
|
||||
number or string.
|
||||
|
||||
When using a Metalua interpreter or compiler, the backtick syntax is
|
||||
supported and can be used directly. Metalua's pretty-printing helpers
|
||||
also try to use backtick syntax whenever applicable.
|
||||
|
||||
### Tree elements
|
||||
|
||||
Tree elements are mainly categorized into statements `stat`,
|
||||
expressions `expr` and lists of statements `block`. Auxiliary
|
||||
definitions include function applications/method invocation `apply`,
|
||||
are both valid statements and expressions, expressions admissible on
|
||||
the left-hand-side of an assignment statement `lhs`.
|
||||
|
||||
block: { stat* }
|
||||
|
||||
stat:
|
||||
`Do{ stat* }
|
||||
| `Set{ {lhs+} {expr+} } -- lhs1, lhs2... = e1, e2...
|
||||
| `While{ expr block } -- while e do b end
|
||||
| `Repeat{ block expr } -- repeat b until e
|
||||
| `If{ (expr block)+ block? } -- if e1 then b1 [elseif e2 then b2] ... [else bn] end
|
||||
| `Fornum{ ident expr expr expr? block } -- for ident = e, e[, e] do b end
|
||||
| `Forin{ {ident+} {expr+} block } -- for i1, i2... in e1, e2... do b end
|
||||
| `Local{ {ident+} {expr+}? } -- local i1, i2... = e1, e2...
|
||||
| `Localrec{ ident expr } -- only used for 'local function'
|
||||
| `Goto{ <string> } -- goto str
|
||||
| `Label{ <string> } -- ::str::
|
||||
| `Return{ <expr*> } -- return e1, e2...
|
||||
| `Break -- break
|
||||
| apply
|
||||
|
||||
expr:
|
||||
`Nil | `Dots | `True | `False
|
||||
| `Number{ <number> }
|
||||
| `String{ <string> }
|
||||
| `Function{ { ident* `Dots? } block }
|
||||
| `Table{ ( `Pair{ expr expr } | expr )* }
|
||||
| `Op{ opid expr expr? }
|
||||
| `Paren{ expr } -- significant to cut multiple values returns
|
||||
| apply
|
||||
| lhs
|
||||
|
||||
apply:
|
||||
`Call{ expr expr* }
|
||||
| `Invoke{ expr `String{ <string> } expr* }
|
||||
|
||||
ident: `Id{ <string> }
|
||||
|
||||
lhs: ident | `Index{ expr expr }
|
||||
|
||||
opid: 'add' | 'sub' | 'mul' | 'div'
|
||||
| 'mod' | 'pow' | 'concat'| 'eq'
|
||||
| 'lt' | 'le' | 'and' | 'or'
|
||||
| 'not' | 'len'
|
||||
|
||||
### Meta-data (lineinfo)
|
||||
|
||||
|
||||
ASTs also embed some metadata, allowing to map them to their source
|
||||
representation. Those informations are stored in a `"lineinfo"` field
|
||||
in each tree node, which points to the range of characters in the
|
||||
source string which represents it, and to the content of any comment
|
||||
that would appear immediately before or after that node.
|
||||
|
||||
Lineinfo objects have two fields, `"first"` and `"last"`, describing
|
||||
respectively the beginning and the end of the subtree in the
|
||||
sources. For instance, the sub-node ``Number{123}` produced by parsing
|
||||
`[[return 123]]` will have `lineinfo.first` describing offset 8, and
|
||||
`lineinfo.last` describing offset 10:
|
||||
|
||||
|
||||
> mlc = require 'metalua.compiler'.new()
|
||||
> ast = mlc :src_to_ast "return 123 -- comment"
|
||||
> print(ast[1][1].lineinfo)
|
||||
<?|L1|C8-10|K8-10|C>
|
||||
>
|
||||
|
||||
A lineinfo keeps track of character offsets relative to the beginning
|
||||
of the source string/file ("K8-10" above), line numbers (L1 above; a
|
||||
lineinfo spanning on several lines would read something like "L1-10"),
|
||||
columns i.e. offset within the line ("C8-10" above), and a filename if
|
||||
available (the "?" mark above indicating that we have no file name, as
|
||||
the AST comes from a string). The final "|C>" indicates that there's a
|
||||
comment immediately after the node; an initial "<C|" would have meant
|
||||
that there was a comment immediately before the node.
|
||||
|
||||
Positions represent either the end of a token and the beginning of an
|
||||
inter-token space (`"last"` fields) or the beginning of a token, and
|
||||
the end of an inter-token space (`"first"` fields). Inter-token spaces
|
||||
might be empty. They can also contain comments, which might be useful
|
||||
to link with surrounding tokens and AST subtrees.
|
||||
|
||||
Positions are chained with their "dual" one: a position at the
|
||||
beginning of and inter-token space keeps a refernce to the position at
|
||||
the end of that inter-token space in its `"facing"` field, and
|
||||
conversly, end-of-inter-token positions keep track of the inter-token
|
||||
space beginning, also in `"facing"`. An inter-token space can be
|
||||
empty, e.g. in `"2+2"`, in which case `lineinfo==lineinfo.facing`.
|
||||
|
||||
Comments are also kept in the `"comments"` field. If present, this
|
||||
field contains a list of comments, with a `"lineinfo"` field
|
||||
describing the span between the first and last comment. Each comment
|
||||
is represented by a list of one string, with a `"lineinfo"` describing
|
||||
the span of this comment only. Consecutive lines of `--` comments are
|
||||
considered as one comment: `"-- foo\n-- bar\n"` parses as one comment
|
||||
whose text is `"foo\nbar"`, whereas `"-- foo\n\n-- bar\n"` parses as
|
||||
two comments `"foo"` and `"bar"`.
|
||||
|
||||
So for instance, if `f` is the AST of a function and I want to
|
||||
retrieve the comment before the function, I'd do:
|
||||
|
||||
f_comment = f.lineinfo.first.comments[1][1]
|
||||
|
||||
The informations in lineinfo positions, i.e. in each `"first"` and
|
||||
`"last"` field, are held in the following fields:
|
||||
|
||||
* `"source"` the filename (optional);
|
||||
* `"offset"` the 1-based offset relative to the beginning of the string/file;
|
||||
* `"line"` the 1-based line number;
|
||||
* `"column"` the 1-based offset within the line;
|
||||
* `"facing"` the position at the opposite end of the inter-token space.
|
||||
* `"comments"` the comments in the associated inter-token space (optional).
|
||||
* `"id"` an arbitrary number, which uniquely identifies an inter-token
|
||||
space within a given tokens stream.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
Metalua
|
||||
=======
|
||||
|
||||
Metalua is a Lua code analysis tool, as well as a compiler for a
|
||||
superset of Lua 5.1 supporting Compile-Time Meta-Programming. It's
|
||||
separated into two LuaRocks, `metalua-parser` and
|
||||
`metalua-compiler`. The documentation of each rock can be found in
|
||||
`README-parser.md` and `README-compiler.md`.
|
||||
|
||||
All the code in Metalue is released under dual lincenses:
|
||||
|
||||
* MIT public license (same as Lua);
|
||||
* EPL public license (same as Eclipse).
|
||||
@@ -0,0 +1,47 @@
|
||||
--*-lua-*--
|
||||
package = "metalua-compiler"
|
||||
version = "0.7.3-1"
|
||||
source = {
|
||||
url = "http://git.eclipse.org/c/koneki/org.eclipse.koneki.metalua.git/snapshot/org.eclipse.koneki.metalua-v0.7.3.tar.gz"
|
||||
}
|
||||
|
||||
description = {
|
||||
summary = "Metalua's compiler: converting (Meta)lua source strings and files into executable Lua 5.1 bytecode",
|
||||
detailed = [[
|
||||
This is the Metalua copmiler, packaged as a rock, depending
|
||||
on the spearate metalua-parser AST generating library. It
|
||||
compiles a superset of Lua 5.1 into bytecode, which can
|
||||
then be loaded and executed by a Lua 5.1 VM. It also allows
|
||||
to dump ASTs back into Lua source files.
|
||||
]],
|
||||
homepage = "http://git.eclipse.org/c/koneki/org.eclipse.koneki.metalua.git",
|
||||
license = "EPL + MIT"
|
||||
}
|
||||
dependencies = {
|
||||
"lua ~> 5.1", -- Lua 5.2 bytecode not supported
|
||||
"luafilesystem ~> 1.6", -- Cached compilation based on file timestamps
|
||||
"metalua-parser >= 0.7.3", -- AST production
|
||||
}
|
||||
|
||||
build = {
|
||||
type="builtin",
|
||||
modules={
|
||||
["metalua.compiler.bytecode"] = "metalua/compiler/bytecode.lua",
|
||||
["metalua.compiler.globals"] = "metalua/compiler/globals.lua",
|
||||
["metalua.compiler.bytecode.compile"] = "metalua/compiler/bytecode/compile.lua",
|
||||
["metalua.compiler.bytecode.lcode"] = "metalua/compiler/bytecode/lcode.lua",
|
||||
["metalua.compiler.bytecode.lopcodes"] = "metalua/compiler/bytecode/lopcodes.lua",
|
||||
["metalua.compiler.bytecode.ldump"] = "metalua/compiler/bytecode/ldump.lua",
|
||||
["metalua.loader"] = "metalua/loader.lua",
|
||||
},
|
||||
install={
|
||||
lua={
|
||||
["metalua.treequery"] = "metalua/treequery.mlua",
|
||||
["metalua.compiler.ast_to_src"] = "metalua/compiler/ast_to_src.mlua",
|
||||
["metalua.treequery.walk"] = "metalua/treequery/walk.mlua",
|
||||
["metalua.extension.match"] = "metalua/extension/match.mlua",
|
||||
["metalua.extension.comprehension"] = "metalua/extension/comprehension.mlua",
|
||||
["metalua.repl"] = "metalua/repl.mlua",
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
rock_manifest = {
|
||||
doc = {
|
||||
["README-compiler.md"] = "292523d759247d210d32fb2f6153e0f4",
|
||||
["README-parser.md"] = "b44e3673d96dd296f2c0e92a6c87ee18",
|
||||
["README.md"] = "20bfb490cddef9e101e44688791abcda"
|
||||
},
|
||||
lua = {
|
||||
metalua = {
|
||||
compiler = {
|
||||
["ast_to_src.mlua"] = "1309f76df37585ef8e1f67f748b07b22",
|
||||
bytecode = {
|
||||
["compile.lua"] = "430e4a6fac8b64b5ebb3ae585ebae75a",
|
||||
["lcode.lua"] = "3ad8755ebe8ea8eca6b1d2846eec92c4",
|
||||
["ldump.lua"] = "295e1d9657fb0126ce3471b3366da694",
|
||||
["lopcodes.lua"] = "a0f15cfc93b026b0a868466d066f1d21"
|
||||
},
|
||||
["bytecode.lua"] = "1032e5233455fd4e504daf5d2893527b",
|
||||
["globals.lua"] = "80ae19c6e640de0746348c91633c4c55"
|
||||
},
|
||||
extension = {
|
||||
["comprehension.mlua"] = "426f5856896bda4c3763bd5f61410685",
|
||||
["match.mlua"] = "79960265331e8b2f46199c2411a103de"
|
||||
},
|
||||
["loader.lua"] = "1cdbf6cdf6ca97c55540d068474f1d8a",
|
||||
["repl.mlua"] = "729456f3a8cc073788acee564a0495f0",
|
||||
treequery = {
|
||||
["walk.mlua"] = "5159aaddbec55936f91ea4236f6451d3"
|
||||
},
|
||||
["treequery.mlua"] = "97ffcee0825ac3bc776d01566767b2e8"
|
||||
}
|
||||
},
|
||||
["metalua-compiler-0.7.3-1.rockspec"] = "b3883b25641d862db6828300bb755d51"
|
||||
}
|
||||
Reference in New Issue
Block a user