From 3fbc76e37fea27926148c80d0a858d84784d526d Mon Sep 17 00:00:00 2001 From: kaltokri Date: Tue, 24 Oct 2023 16:09:09 +0200 Subject: [PATCH] Documentation of the build system added --- .gitignore | 2 + docs/buildsystem/build-docs.md | 73 ++++++++++++++++++++++++++++++ docs/buildsystem/build-includes.md | 45 ++++++++++++++++++ docs/buildsystem/gh-pages.md | 51 +++++++++++++++++++++ docs/buildsystem/index.md | 58 ++++++++++++++++++++++++ docs/buildsystem/local-test.md | 49 ++++++++++++++++++++ 6 files changed, 278 insertions(+) create mode 100644 docs/buildsystem/build-docs.md create mode 100644 docs/buildsystem/build-includes.md create mode 100644 docs/buildsystem/gh-pages.md create mode 100644 docs/buildsystem/index.md create mode 100644 docs/buildsystem/local-test.md diff --git a/.gitignore b/.gitignore index 0578aa593..05489aa14 100644 --- a/.gitignore +++ b/.gitignore @@ -227,3 +227,5 @@ MooseCodeWS.code-workspace # Excludes for act (https://github.com/nektos/act) .secrets .env +.actrc +.vars diff --git a/docs/buildsystem/build-docs.md b/docs/buildsystem/build-docs.md new file mode 100644 index 000000000..8247e30bb --- /dev/null +++ b/docs/buildsystem/build-docs.md @@ -0,0 +1,73 @@ +--- +parent: Build system +nav_order: 2 +--- + +# Build class documentation + +The documentation of all classes are included in the code as comments. +This kind of documentation process is called [LuaDoc]. But we build the +html files for the documentation with [LuaDocumentor]. It is a tool +similar to [LuaDoc], but with some additional features the Moose team +decided to use. + +{: .important } +> The team created also some modifications, which are not part of the +> official [LuaDocumentor] tool. So we use the code in the git repository +> [Applevangelist/luadocumentor] in the branch `patch-1`. + +There are two git repositories which are used to save the generated +documentation: + +- [MOOSE_DOCS] is the repository for the `master` branch of [MOOSE] + - A configured GitHub Pages job will deploy the result to:
+ +- [MOOSE_DOCS_DEVELOP] is the repository for the `develop` branch of [MOOSE] + - A configured GitHub Pages job will deploy the result to:
+ + +Main build steps to create the class documentation are defined in [.github/workflows/build-docs.yml]: + +- Checkout of the git repository [MOOSE]. +- Create output folders. +- Checkout of the git repository [Applevangelist/luadocumentor] with + branch `patch-1` into a subdirectory. +- Update the Linux system software. +- Install needed tools: + - [tree] - A tool to output a tree view of a folder structure. + - [lua] - Package to run [Lua] scripts. This time [Lua] 5.1, + because it matches the DCS environment. + - [LuaRocks] - This is the package manager for Lua modules. + - [markdown] - Dependency for [LuaDocumentor] + - [penlight] - Dependency for [LuaDocumentor] + - [metalua-compiler] - Dependency for [LuaDocumentor] + - [metalua-parser] - Dependency for [metalua-compiler] + - [checks] - Dependency for [metalua-parser] + +- Run the build steps: + - Run `luadocumentor.lua` to create the html files. + +- Deploy build results: + - Checkout [MOOSE_DOCS] or [MOOSE_DOCS_DEVELOP] git repository in a subdirectory. + - Use the matching git repository for the branch of [MOOSE]. + - `master` -> [MOOSE_DOCS]. + - `develop` -> [MOOSE_DOCS_DEVELOP]. + - Use a `TOKEN` for checkout, so a `push` is possible later on. + - Copy build result to `MOOSE_DOCS` folder. + - Push results to the target repository. + +[tree]: https://www.cyberciti.biz/faq/linux-show-directory-structure-command-line/ +[LuaDoc]: https://keplerproject.github.io/luadoc/ +[LuaDocumentor]: https://luarocks.org/modules/luarocks/luadocumentor +[Applevangelist/luadocumentor]: https://github.com/Applevangelist/luadocumentor/tree/patch-1 +[markdown]: https://luarocks.org/modules/mpeterv/markdown +[penlight]: https://luarocks.org/modules/tieske/penlight +[metalua-compiler]: https://luarocks.org/modules/luarocks/metalua-compiler +[metalua-parser]: https://luarocks.org/modules/luarocks/metalua-parser +[checks]: https://luarocks.org/modules/fab13n/checks +[MOOSE]: https://github.com/FlightControl-Master/MOOSE +[MOOSE_DOCS]: https://github.com/FlightControl-Master/MOOSE_DOCS +[MOOSE_DOCS_DEVELOP]: https://github.com/FlightControl-Master/MOOSE_DOCS_DEVELOP +[Lua]: https://www.lua.org/ +[LuaRocks]: https://luarocks.org/ +[.github/workflows/build-docs.yml]: https://github.com/FlightControl-Master/MOOSE/blob/master/.github/workflows/build-docs.yml diff --git a/docs/buildsystem/build-includes.md b/docs/buildsystem/build-includes.md new file mode 100644 index 000000000..a3e84c38b --- /dev/null +++ b/docs/buildsystem/build-includes.md @@ -0,0 +1,45 @@ +--- +parent: Build system +nav_order: 1 +--- + +# Build include files + +Main build steps to create the include files are defined in [.github/workflows/build-includes.yml]: + +- Checkout of the git repository [MOOSE]. +- Create output folders. +- Update the Linux system software. +- Install needed tools: + - [tree] - A tool to output a tree view of a folder structure. + - [lua5.3] - Package to run [Lua] scripts. Version 5.3 is needed, because we + need liblua5.3-dev for [LuaSrcDiet]. + - [LuaRocks] - LuaRocks is the package manager for Lua modules. + - liblua5.3-dev - Header file of [Lua] needed for [LuaSrcDiet] to work. + - [LuaSrcDiet] - To compress the [Lua] code and create `Moose_.lua`. + - [LuaCheck] - This is a static code analyzer and a linter for [Lua]. + +- Run the build steps: + - Run `./Moose Setup/Moose_Create.lua` to create `Moose.lua`. + - Run `./Moose Setup/Moose_Create.lua` to create dynamic `Moose.lua` to + load individual Lua class files used by Moose developers. + - Run [LuaSrcDiet] to compress the [Lua] code and create `Moose_.lua` + - Run [LuaCheck] to find errors in the code. Warnings are ignored, because + there are a lot of warnings, which cannot be resolved by the Moose team. + +- Deploy build results: + - Checkout [MOOSE_INCLUDE] git repository in a subdirectory. + - Use the same branch used to checkout [MOOSE] git repository. + - Use a `TOKEN` for checkout, so a `push` is possible later on. + - Copy build result to `MOOSE_INCLUDE` folder + - Push results to [MOOSE_INCLUDE] repository + +[.github/workflows/build-includes.yml]: https://github.com/FlightControl-Master/MOOSE/blob/master/.github/workflows/build-includes.yml +[tree]: https://www.cyberciti.biz/faq/linux-show-directory-structure-command-line/ +[lua5.3]: https://www.lua.org/manual/5.3/ +[LuaRocks]: https://luarocks.org/ +[LuaCheck]: https://github.com/mpeterv/luacheck +[MOOSE]: https://github.com/FlightControl-Master/MOOSE +[MOOSE_INCLUDE]: https://github.com/FlightControl-Master/MOOSE_INCLUDE +[LuaSrcDiet]: https://github.com/jirutka/luasrcdiet +[Lua]: https://www.lua.org/ diff --git a/docs/buildsystem/gh-pages.md b/docs/buildsystem/gh-pages.md new file mode 100644 index 000000000..99277ed20 --- /dev/null +++ b/docs/buildsystem/gh-pages.md @@ -0,0 +1,51 @@ +--- +parent: Build system +nav_order: 3 +--- + +# Build GitHub Pages + +This documentation is created by [GitHub Pages]. The source files are +stored in the repository [MOOSE] in the subfolder `docs`. +We use [Just the Docs], which is a modern, highly customizable, and responsive +[Jekyll] theme for documentation. + +{: .note } +> The class documentation is created by its own [build] and is not the scope for this page! + +The build steps to create this documentation are defined in [.github/workflows/gh-pages.yml]. + +It is divided into two jobs: +- build: + - Only changes to in the subfolder `docs` or `gh-pages.yml` will trigger a build. + - Checkout of the git repository [MOOSE]. + - Setup [Ruby] version 3.1, which is needed by [Jekyll]. + - Run action [configure-pages]. + - Build with [Jekyll]. + - Run action [upload-pages-artifact]. +- deploy: + - Run action [deploy-pages]. + +# Preview of this documentation + +When enhancing this documentation it is very useful to see a 1on1 preview of the pages. +This can be displayed as follows: + +- You need a working installation of [Docker]. +- Go to the `docs` subfolder. +- Run `docker compose up`. +- Open a browser with the following URL: . +- After a change of the [Markdown] files, wait some seconds and press F5 in the browser. + +[GitHub Pages]: https://pages.github.com/ +[MOOSE]: https://github.com/FlightControl-Master/MOOSE +[Just the Docs]: https://github.com/just-the-docs/just-the-docs +[Jekyll]: https://jekyllrb.com/ +[Ruby]: https://www.ruby-lang.org/en/ +[build]: build-docs.md +[.github/workflows/gh-pages.yml]: https://github.com/FlightControl-Master/MOOSE/blob/master/.github/workflows/gh-pages.yml +[configure-pages]: https://github.com/actions/configure-pages/ +[upload-pages-artifact]: https://github.com/actions/upload-pages-artifact +[deploy-pages]: https://github.com/actions/deploy-pages/ +[Docker]: https://www.docker.com/ +[Markdown]: https://www.markdownguide.org/ diff --git a/docs/buildsystem/index.md b/docs/buildsystem/index.md new file mode 100644 index 000000000..1dec95fed --- /dev/null +++ b/docs/buildsystem/index.md @@ -0,0 +1,58 @@ +--- +has_children: true +nav_order: 3 +--- + +# Build system + +{: .note } +> This documentation is not needed for end users. Only the people of the +> development team, who must maintain the build system need to read this. + +In this document we want to describe our build system for MOOSE. +MOOSE consists of multiple [Lua] files. Each class is stored in its own file. +This is needed for MOOSE developers to maintain clarity. +For users this is not practical, because they want to include the whole framework +as a single file into their missions. + +Because of this the build will collect all needed files and merge them together +in one file with the name `Moose.lua`. It includes also all comments and the +class documentation. Because of this its size is about 6-7 MB. + +To reduce the size of the file and make mission files smaller, the Moose team +decided to create a version without all comments and documentation. This file +is named `Moose_.lua`. It is created by a tool with the name [LuaSrcDiet]. + +Both files will be called static includes. In other programming languages includes +are dependencies. For Moose it is easier to memorize, that these files must be +included in your mission to use Moose. It is an static approach because you need +to add it once and it is only read from inside of the mission file after that. +A dynamic approach is to load all the single class files on each mission start +from the hard disk. But this is more for advanced Moose users and Moose developers. + +## Details + +In the past [AppVeyor] was used to run the build on a Windows system. +We decided to migrate this build to [GitHub Actions]. Installation of +dependencies was not stable on Windows with [GitHub Actions]. So we switched +to Ubuntu Linux. + +### GitHub Actions yml files + +The build configuration is stored in the folder `.github/workflows`. You will find +multiple files in this directory: + +- [build-docs.yml] - Job definition to generate the class documentation +- [build-includes.yml] - Job definition to build the static includes +- [gh-pages.yml] - Job to build this documentation page + +We decided to use different files for each job for separation of duties and easier +maintenance. + +[Lua]: https://www.lua.org/ +[LuaSrcDiet]: https://github.com/jirutka/luasrcdiet +[AppVeyor]: https://www.appveyor.com/ +[GitHub Actions]: https://docs.github.com/en/actions +[build-docs.yml]: build-docs.md +[build-includes.yml]: build-includes.md +[gh-pages.yml]: gh-pages.md diff --git a/docs/buildsystem/local-test.md b/docs/buildsystem/local-test.md new file mode 100644 index 000000000..98d39fc5a --- /dev/null +++ b/docs/buildsystem/local-test.md @@ -0,0 +1,49 @@ +--- +parent: Build system +nav_order: 4 +--- + +# Run builds locally + +When creating or enhancing [GitHub Actions] builds it is a problem to test the +build. After each change you need to commit and check the build result. This +leads to a lot of unnecessary commits. + +Therefor it is needed to run the build locally on the developer PC. The tool +which enabled this is [act]. It uses [Docker] to create a build runner and +executes the [GitHub Actions] build with it. + +[act] can by installed by [Chocolatey] by this single command: `choco install act-cli`. + +We use the `Medium Docker Image` for our MOOSE builds to work properly. +Unfortunately the docker images used by [act] are not as up to date as the +images used by [GitHub Actions]. So we needed to add a build step with +`sudo apt-get -qq update`. + +The build jobs needs `TOKENS` to run properly. So you have to create a PAT +([Personal Access Token]). A classic Token with read rights is enough to run +the build, as long as don't want to push the results. + +{: .important } +> The push step is only executed if the variable `FORCE_PUSH` with value `true` is set. +> - This is only needed if the push step itself must be change and tested! +> - Add parameter `--var FORCE_PUSH=true` to your [act] commando. +> - You and your PAT needs write access to the target repos, too. + +Save your PAT in the file `.secrets` in the main folder +of the MOOSE repository. This file is added to `.gitignore`, so it is not +recognized by git for commits. Add the following line to `.secrets`: + +``` +BOT_TOKEN= +``` + +To run the builds use these commands: +- `act push -W .github/workflows/build-includes.yml` +- `act push -W .github/workflows/build-docs.yml` + +[GitHub Actions]: https://docs.github.com/en/actions +[act]: https://github.com/nektos/act +[Docker]: https://www.docker.com/ +[Chocolatey]: https://community.chocolatey.org/ +[Personal Access Token]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens