WebUI_build#

AMD modules and Web UI build#

Overview#

One of the most recommended optimization techniques for web pages are reduction of the number of http request and a minimization of JavaScript file size. Web UI uses a lot of JavaScript files. These files can be concatenated into a single file and then optimized by JavaScript compiler which strips comments, white-spaces and does other optimizations.

Design#

The main idea is transform FreeIPA JavaScript files into AMD modules and build them with a builder into single file, then the file will be further optimized by JavaScript compiler.

Introducing Dojo#

This feature has two topics which are closely related: AMD modules with AMD loader, and a builder. A loader from a dojo/dojo library is used as a loader and a build tool from dojo/util library as a builder. Including source codes of both libraries in FreeIPA repository is not desired because of their number and size. Therefore FreeIPA git repository contains only their compiled versions. The builder and a Dojo library are both built by the builder itself.

Following tools were developed to simplify maintenance of Dojo Libraries:

util/prepare-dojo.sh - can checkout dojo/dojo or dojo/util from github repositories to a directory outside FreeIPA directory. Then it can apply custom patches on a builder and make symbolic links src/dojo and src/build which points to the check-outed repositories.
util/make-dojo.sh - creates custom build of dojo/dojo. Requires util/prepare-dojo.sh to be run before. Then it calls compile.sh to minimize the build.
util/make-builder.sh - creates custom build of dojo/util/build (builder). Requires util/prepare-dojo.sh to be run before. Then it calls compile.sh to minimize the build.
util/compile.sh - compiles a build made by a builder using a uglify.js compiler. Usually no need to use it alone.

Uglify.js#

Builder transforms AMD packages into single or more files and does basic minimization like stripping of comments. For further minimization a proper JavaScript compiler has to be used. Uglify.js was chosen because of it’s capabilities and also because it can be run in rhino environment and thus to avoid problems of not having a common compiler on targeted platforms. For easier usage a wrapper scripted was developed: util/uglifyjs/uglify SOURCE TARGET. It will be better to run under node.js when all target platforms will support it.

Building FreeIPA#

Web UI JavaScript files can be divided into three sets: third party libraries, FreeIPA code and Dojo library. Lets call them layers. FreeIPA now uses 4 third party libs. 2 of them are already compiled and the rest has small size. No bigger optimization techniques are required here. Dojo is already prepared for optimization. The focus of optimization is therefore on FreeIPA layer.

AMD module wrapping#

Dojo/util/build tool (builder) is used as a builder of both FreeIPA and Dojo layer. The builder works best with AMD modules. Full modification of FreeIPA layer into AMD modules would require splitting files by component and redefining a lot of dependencies which is a huge tasks. Different approach is taken to avoid this problem at the moment: Existing files are only wrapped with AMD module definition. They should be converted to proper modules - with single purpose and clean dependencies, gradually in a future. Same applies on Web UI tests.

Build#

A build of FreeIPA layer is done by calling util/make-ui.sh. This script is run at make-all phase of an RPM build. Make-ui.sh is not checking errors during the build because we don’t provide dojo/dojo source codes and thus there are dependency errors. Dojo builder can’t be configured to ignore certain dependency errors and also can’t be built against already built library. The layer is built even though there are dependency errors, it’s not a blocker but it will be nice to fix it in a future (in dojo upstream or by custom patch).

New directory structure#

Web UI directory structure was change because the old one didn’t suffice new needs which are:

separate layers into own directories - for AMD
easy switch between source code version and built version during testing

Following directories in a ui directory were created:

src - contains all source codes. Special pages (login, logout, reset password) JS code was not moved yet. Not used in production.
js - a production directory. At production it should contain built layers, at development symbolic links to related folders in src or build directories.
release - working dir at build phase
build - contains output of make-ui.sh and make-dojo.sh scripts.

All FreeIPA layer JS sources were move to src/freeipa folder. Libraries (jquery, jquery ui, bbq, json2) were moved to src/libs.

Development profiles#

Several approaches of debugging of UI can be considered when developing UI. Developer can either use a machine with installed FreeIPA or he can start Web UI from a development directory. When the latter case is used, the Web UI uses JSON files stored in test/data directory which simulates JSON-RPC communication. Because there is no communication with server, we can call this approach an ‘offline version’.

Offline version#

Offline version uses FreeIPA layer source codes and built version of dojo/dojo library by default. It can use built version of FreeIPA layer or source codes of dojo/dojo library by changing of symbolic links in js directory. To make this task easier, an ‘util/change-profile.sh’ script was developed. The script can also set `git update-index –assume-unchanged` to prevent git from notifying about this change.

Test server#

Different approach must be applied when using test server for debugging. In this case source codes or built layers must be copied to /usr/share/ipa directory on the test server. To make the task easier util/sync.sh script was created. Developer can specify which parts of the UI in which form he wants to copy. The tool contains directory mappings so developer doesn’t have think about where source codes or build versions are placed. Look to Use Cases chapter or tool’s help for more information.

Use Cases#

Users#

None

Developers#

Note: all commands are run from install/ui directory of FreeIPA source dir.

Make new FreeIPA layer build#

only useful for debugging. It’s done automatically in make all phase of RPM build.
run $ util/make-ui.sh

Set environment to debug source codes of FreeIPA layer using offline version#

it’s the default profile after checkout
to switch back from other profile:
- $ util/change-profile.sh -p source
open index.html by a browser using file:// protocol:
- file:///home/login/path-to-freeipa/freeipa/install/ui/index.html
- when using Chrome, run it with –disable-web-security option otherwise XHR won’t work

Set environment to debug built FreeIPA layer using offline version#

$ util/change-profile.sh -p compiled
open browser same way as in previous use case
note: this doesn’t create the compiled version, you have to run it separately

Copy source codes of FreeIPA layer on test server#

$ util/sync.sh –host root@test.example.com –freeipa
you can replace root with any user who can write into /usr/share/ipa
add –clean option if you want to delete all files from target dir
run $ util/sync.sh –help to get more information about others folders (images, root dir, libs) and option shortcuts

Copy built FreeIPA layer on test server#

$ util/sync.sh –host root@test.example.com –freeipa –compiled

Update internal.py (strings for Web UI) on test server#

$ util/sync.sh –host root@test.example.com –strings –restart
note: –restart restarts httpd on test server (systemctl restart httpd.service). Required for changes to take effect.

Clone Dojo repositories#

should be run only once
required only for Dojo and Builder build
$ util/prepare-dojo.sh –all
- creates dojo folder at the same dir as freeipa dir is. Can be changed by –dir option, but it’s not well tested.
- clones dojo/dojo.git and dojo/util.git into dojo folder
- checkouts both repos to tag 1.8.1 (should be change later when updating dojo)
- applies custom patches in util/build/patches dir on dojo/util repo
- makes src/dojo and src/build symbolic links
can be fine-tuned by running with different options, check –help

Make new Dojo lib build#

required when FreeIPA layer has new dependency
requires to have dojo cloned
new dependencies should be define in src/dojo.profile.js in layer.include list
run $ util/make-dojo.sh

Make new Builder build#

required when a change in a builder is needed. Usually shouldn’t be.
requires to have dojo cloned
recommended workflow:
- clone dojo if not done: $ util/prepare-dojo.sh –all
- checkout desired version, if needed $ util/prepare-dojo.sh –dojo –util –checkout –branch VERSION
- make required changes in dojo-root/util/build
- run $ util/make-builder.sh
- warning: builder is overwritten on successful build, use git reset to change it back if needed
- if all OK, create a patch file with the changes. Name should be: XXX-dojo-build-NAME-YY-commit-message.patch, where XXX is a the following number than in the last patch, NAME is your login, nick and YY is your patch number starting from 00
- store the patch into util/build/patches directory (will require a force option on git add)
- make patch of all these changes and send it for review

Implementation#

Any additional requirements or changes discovered during the implementation phase.

Feature Management#

From new user feature POV doesn’t affect Web UI or CLI.

Major configuration options and enablement#

No configuration options.

Replication#

No impact.

Dependencies#

rhino 1.7R3 on build machine (minimum version with common JS modules support)

Impact on other development teams#