Skip to content

Latest commit

 

History

History
315 lines (224 loc) · 16.1 KB

MAINTENANCE.md

File metadata and controls

315 lines (224 loc) · 16.1 KB
Raw

Maintaining GMT

Below are instructions for developers and advanced users.

Contents

Building the source code

Refer to the building guide for instructions on building GMT from source. Several sections include a Note for developers that provides helpful or necessary information for running tests and building the documentation.

Building the documentation

To build the GMT documentation you need to install the Sphinx documentation builder. After configuring and building GMT (see the building guide), you can build GMT documentation using the following commands within the build directory:

cmake --build . --target docs_depends     # Generate images included in the documentation
cmake --build . --target optimize_images  # Optimize PNG images for documentation [optional]
cmake --build . --target animation        # Generate animations included in the documentation [optional]
cmake --build . --target docs_man         # UNIX manual pages
cmake --build . --target docs_html        # HTML manual, tutorial, cookbook, and API reference

Note: Refer to the file admin/bashrc_for_gmt for useful aliases for building the documentation.

Note: pngquant is needed for optimizing images.

Testing GMT

GMT ships with more than 1000 tests to make sure that any changes won't break its functionality. In addition to the tests located in the test/ directory, GMT tests all the plots included in its documentation. The documentation tests are located in the doc/scripts/ directory. The majority of GMT tests are plot-based, with each test requiring a bash script for generating the plot and a reference PostScript file. These tests pass if the difference between a new plot generated using the test script and the reference PostScript file is less than a defined threshold. Other tests compute grids, tables, or other output, with the test passing if a suitable comparison is made against a reference case.

Tests that are known to fail are excluded by adding # GMT_KNOWN_FAILURE anywhere in the test script.

Configuring CMake for testing GMT

To enable testing, you need to uncomment the following lines in your ConfigUserAdvanced.cmake when configuring GMT:

enable_testing()
set (DO_EXAMPLES TRUE)
set (DO_TESTS TRUE)
set (DO_ANIMATIONS TRUE)

set (SUPPORT_EXEC_IN_BINARY_DIR TRUE)

set (DO_API_TESTS ON)

Optionally, uncomment the following line and change 4 to the number of ctest jobs to run simultaneously.

set (N_TEST_JOBS 4)

Running tests

After configuring CMake and building GMT, you can run all the tests by running this command in the build directory:

cmake --build . --target check

You can also run ctest commands in the build directory. Below are some common used ctest commands.

  • Run all tests in 4 parallel jobs:

     ctest -j 4

  • Re-run all failing tests in previous run in 4 parallel jobs:

     ctest -j 4 --rerun-failed

  • Select individual tests using regexp with ctest:

     ctest --output-on-failure -R ex2[3-6]

Note: Refer to the file admin/bashrc_for_gmt for useful aliases for running the tests.

Reviewing test failures

There are several tests that are "known to fail" for GMT. Unless the GMT_ENABLE_KNOWN2FAIL variable is set when configuring CMake or setting up ConfigUserAdvanced.cmake, these tests are excluded when running ctest using the instructions provided in the Running tests section. Therefore, you should expect all tests to pass unless something new is broken.

Information about failing tests is produced in test/fail_count.txt inside the build directory. For plot-based tests, the subdirectories test/ and doc/scripts/ inside the build directory contain folders for each failing test. For plot-based tests, the directory associated with each failing tests contains a gmtest.sh script, a gmt.conf file, an alias to the test script, a PostScript file and PDF document generated by the test script, and a PNG image that shows differences between the reference plot and new plot in magenta. In addition to these files, running the failing tests with verbose output can be helpful for evaluating failures:

    ctest --rerun-failed --verbose

Updating reference plots for tests

Pull requests should avoid needing to change PostScript files in the test/ and doc/scripts/ directories. However, if this is unavoidable, new PostScript reference files can be generated by running ctest -R <test-script-name> in the build directory after following the building guide and the Configuring CMake for testing GMT instructions. The new PostScript file can then be copied from the appropriate subdirectory within build/test/ or build/doc/scripts/ to test/ or doc/scripts/ respectively.

Creating source packages

Edit cmake/ConfigDefault.cmake and set GMT_PACKAGE_VERSION_MAJOR, GMT_PACKAGE_VERSION_MINOR, and GMT_PACKAGE_VERSION_PATCH. Also set GMT_PUBLIC_RELEASE to TRUE. Then create source packages with:

cmake --build . --target gmt_release      # export the source tree and documentation
cmake --build . --target gmt_release_tar  # create tarballs (in tar.gz and tar.xz formats)

Packaging

Currently, packaging with CPack works on macOS (Bundle, TGZ, TBZ2), Windows (ZIP, NSIS), and UNIX (TGZ, TBZ2). On Windows you need to install NSIS. After building GMT and the documentation, build and place the executables, including the supplements, with

cmake --build . --target install

and then create the package with either one of these:

cmake --build . --target package
cpack -G TGZ|TBZ2|Bundle|ZIP|NSIS

Updating the development source codes

Assuming you did not delete the build directory, this is just as simple as

cd <path-to-gmt>
git pull
cd build
cmake --build .
cmake --build . --target install

CMake will detect any changes to the source files and will automatically reconfigure. If you deleted all files inside the build directory you have to run CMake again manually.

Note: Refer to the file admin/bashrc_for_gmt for useful aliases for updating the development source code

Debugging GMT

Guides for debugging GMT are provided in the developer resources section of the GMT documentation.

Using build and test aliases

The file admin/bashrc_for_gmt contains useful aliases for building and testing GMT that some developers chose to use. New pull requests with other aliases that you find helpful are welcome. This file is version controlled, so you should copy the file to a different location in order to edit and use it. For example, use these commands to copy it to your home directory:

cd <path-to-gmt>
cp admin/bashrc_for_gmt ~/.bashrc_for_gmt

Here are the steps for setting up bashrc_for_gmt after copying it to a new location:

  • If you do not have ninja installed, you will need to change builder=ninja to builder=make and Bname="Ninja" to Bname="Unix Makefiles". Ninja is recommended for speeding up build times.
  • You may need to update pngview=open and pngview=open depending on your preferred program for viewing files.
  • Optionally, change ncores=4 to the number of cores to use for building and running tests.
  • Change MATLAB=/Applications/MATLAB_R2019a.app to the path for your version of the MATLAB app.
  • Set REPO_DIR to the path that contains the local git clone copy of the GMT repository.
  • Set DATA_DIR to the path that contains the folders dcw-gmt-2.0.0/ and gshhg-gmt-2.3.7/ for the dcw and gshhg datasets respectively. If these folders are not located in the same path, you can instead delete the line (DATA_DIR=<path to directory containing GSHHG and DCW>) and set the individual paths to the GSHHG and DCW source by changing (export GMT_GSHHG_SOURCE=${DATA_DIR}/gshhg-gmt-2.3.7) and (export GMT_DCW_SOURCE=${DATA_DIR}/dcw-gmt-2.0.0).
  • Edit the file ~/.bashrc to include the line source <path>/bashrc_for_gmt. If you set up bashrc_for_gmt as a hidden file in your home directory, this line should be source ~/.bashrc_for_gmt.

Here are some of the shortcuts included in bashrc_for_gmt:

  • gmt6 and gtop can be used to quickly cd to the top of the GMT source directory and repository base respectively.
  • gmtfind can be used to list all source, docs, scripts, and text files where a string appears in the file (e.g., gmtfind "Grid increment is" returns all files that contain the string 'Grid increment is'). This includes all files recursively from the current working directory; gtop or gmt6 can be used prior to this command to get to the source directory or repository base.
  • cmakegmtd, cmakegmtr, and cmakegmtx configures cmake for debug, release, and XCode debug respectively.
  • dlog and rlog can be used to open the debug and release build check error logs respectively.
  • There are several aliases with various combinations of pulling new changes, deleting the build directories, configuring cmake, and building the source code. Each of these are documented with comments in bashrc_for_gmt.
  • checkdbuild and checkrbuild can be used to run the tests for the debug and release builds respectively.
  • vpngdbuild and vpdfdbuild can be used to open the results from all failing image-based tests.
  • view_png_failures_r and view_pdf_failures_r can be used for view failures of the release build with a lag between opening each file.

Continuous Integration

We use GitHub Actions Continuous Integration (CI) services to build and test the project on Linux, macOS, and Windows.

There are 11 configuration files located in .github/workflows/:

  1. backport.yml (Backports Pull Requests with specific labels into the matching branch)

    This workflow backports Pull Requests labelled by "backport xxx" into the "xxx" branch. For example, the workflow backports Pull Requests labelled "backport 6.1" into the "6.1" branch. This workflow does not apply to Pull Requests from forks (issue #3827).

  2. build.yml (Build GMT and run a few simple tests)

    This workflow is run on every commit to the master branch and Pull Request branches. The workflow configures and builds GMT on Linux, macOS, and Windows and runs some simple tests. The workflow uses the scripts in the ci/ directory for downloading/installing dependencies (ci/download-coastlines.sh and ci/install-dependencies-*.sh), configuring GMT (ci/config-gmt-*.sh), building GMT, and running some simple tests (ci/simple-gmt-tests.*).

  3. check-links.yml (Check links in the repository and website)

    This workflow is run every Sunday at 12:00 (UTC) to check all external links in plaintext and HTML files. It will create an issue if broken links are found.

  4. ci-caches.yml (Cache GMT remote data files and Windows vcpkg libraries needed for GitHub Actions CI)

    This workflow is run every Sunday at 12:00 (UTC). If new remote files are needed urgently, maintainers can manually uncomment the 'pull_request:' line in the ci-caches.yml file to refresh the cache. The workflow uses the script ci/install-dependencies-windows.sh to cache the vcpkg libraries.

  5. code-validator.yml (Validate code consistency)

    This workflow is run on every commit to the master branch and Pull Request branches. It runs the scripts src/gmt_make_PSL_strings.sh, src/gmt_make_enum_dicts.sh, and src/gmt_make_module_purpose.sh and exits with an error if any changes are detected by git. It also checks the GMT version year in ConfigDefault.cmake and the permissions of the bash scripts.

  6. docker.yml (Build GMT on different Linux distros using dockers)

    This workflow is run when Pull Requests are merged into the master branch. It downloads/installs dependencies and configures and builds GMT on different Linux distributions using dockers.

  7. docs.yml (Build documentation on Linux/macOS/Windows)

    This workflow is run when Pull Requests are merged into the master branch, if the Pull Request involved changes to files in the doc/, .github/workflows/, or ci/ directories.

    The workflow also handles the documentation deployment:

    • Updating the development documentation by pushing the built HTML pages from the master branch into the dev folder of the gh-pages branch.
    • Updating the latest documentation link to the new release.

  8. draft-release.yml (Drafts the next release notes)

    This workflow is run to draft the next release notes when new tags are made. It downloads GMT tarballs, macOS bundle and Windows installers from the GMT FTP server, calculate their sha256sums and save into gmt-X.Y.Z-checksums.txt, draft the release notes, and uploads them as release assets. Maintainers still need to review the draft release notes and click the "publish" button.

  9. lint-checker.yml (Run cppchecks)

    This workflow is run every day at 12:00 (UTC) to run admin/run_cppcheck.sh on the source code.

  10. scm-check.yml (Check for new scientific color maps releases)

    This workflows is run every Sunday at 12:00 (UTC) to check whether there has been a new release of the Scientific colour maps. If a new release is found, it will open an issue automatically.

  11. tests.yml (Tests on Linux/macOS/Windows)

    This workflow is run when Pull Requests are merged into the master branch, if the Pull Request involved changes to the folders that contain source code, workflows, tests, or scripts for generating documentation figures. It runs the full GMT test suite on Linux, macOS, and Windows.

  12. release-drafter.yml (Drafts the next release notes)

    This workflow is run to update the next releases notes as pull requests are merged into master.

Updating the changelog

The Release Drafter GitHub Action will automatically keep a draft changelog at https://github.com/GenericMappingTools/gmt/releases, adding a new entry every time a Pull Request (with a proper label) is merged into the master branch. This release drafter tool has two configuration files, one for the GitHub Action at .github/workflows/release-drafter.yml, and one for the changelog template at .github/release-drafter.yml. Configuration settings can be found at https://github.com/release-drafter/release-drafter. The maintenance documentation for this workflow is based on the PyGMT Maintenance Documentation.

The drafted release notes are not perfect, so we will need to tidy it prior to publishing the actual release notes at https://docs.generic-mapping-tools.org/latest/changes.html.

  1. Go to https://github.com/GenericMappingTools/gmt/releases and click on the 'Edit' button next to the current draft release note. Copy the text of the automatically drafted release notes under the 'Write' tab to doc/rst/source/changes.rst.
  2. Open a new Pull Request using the title 'Changelog entry for GMT X.Y.Z' with the updated release notes, so that other people can help to review and collaborate on the changelog curation process described next.
  3. Edit the change list to remove any trivial changes (updates to the README, typo fixes, CI configuration, etc).
  4. Edit the formatting to use ReST style.
  5. Add links in the changelog to elements of the documentation as appropriate.