Skip to main content

Package organisation

for contributors

How the plant source is laid out, how we develop it (gitflow, testing, CI), how to compile it, and how the R/C++ interface is generated. Aimed at contributors working on the package itself.

Development

Directory structure

The code is organised as follows. This structure reflects the structure generally used for R packages including C++ source.

├── inst
│   ├── RcppR6_classes.yml    # Used by RCppR6 to generate the interface for Rcpp (C++ classes)
│   ├── RcppR6_functions.yml  # Used by RCppR6 to generate the interface for Rcpp (C++ function)
│   ├── include
│   │   ├── plant.h # THE main header file for package - sources other headers
│   │   └── plant   # header (.h) files for different components, many including implementation
│   ├── reference_plant_ff16 # Reference plant physiology ff16
│   ├── docs            # Tables of variables for models
│   └── scripts         # Tools used in package management
├── man                     # R Documentation files (*.Rd)
├── R                           # R functions
├── scripts             # R scripts
├── src                     # C++ files with implementation where not included in headers
├── tests               # Tests
└── vignettes           # Vignettes
Note

The workflow itself — issues and the board, feature branches, what the automated checks do and don’t prove, squash merges, and what a reviewer is looking for — is set out in How we work. This section covers only the parts specific to building this package.

Issues

We use

  • github issues for feature requests, bug reports and developer coordination.
  • github projects for planning of work, including roadmaps, milestones and sprint planning.

Gitflow workflow

We use the Gitflow workflow.

This means we have at least two main branches always present:

  1. a master branch (master) for stable releases
  2. a development branch (develop) where all development takes place

We also use feature branches which branch from the development branch where we can then introduce new features and then pull them back into the development branch. (though we don’t usually have a release branch).

Testing

Continuous integration via GH Actions

Compiling

There are two main ways to compile the package:

  1. Via a makefile, which defines a set of targets that can be built via the make command in terminal/command line
  2. Directly within R using pkgbuild::compile_dll(compile_attributes = FALSE, debug=FALSE)
  3. Directly within R using devtools::load_all() (can be slwoer, use with cuation. See below.)

Makefile

The makefile is defined in the root directory of the package and is called Makefile. It defines multiple targets in the package build process. Each rule defines a target and the commands to run to build that target. These are called by running

#| eval: false
make target_name

where target_name is the target to build. Targets are as follows:

  • all: Compiles the package and builds the function documentation to /man
  • compile: Builds the dll
  • debug: Builds the dll with debug information
  • install: Installs this package (R CMD INSTALL .)
  • build: Builds plant (R CMD build --no-build-vignettes)
  • RcppR6: Update or install RcppR6 files
  • attributes: Scans the source files for attributes and generate code as required. Generates the bindings required to call C++ functions from R for functions labelled with the Rcpp::export attribute
  • roxygen: Compiles the documentation for R
  • test Run the test suite
  • check: Checks plant for common problems
  • clean: Deletes *.o and *.so files that were produced when compiling the package
  • vignettes: Builds the vignettes

We use R to execute the compiling via pkgbuild::compile_dll, evem within the makefile.

devtools::load_all produces a slow binary

Which build method you use to compile can drastically change spped at runtime. In particular, different levels of compiler optimisation are triggered based on the method used to complile the pacakge.

An easy way to compile and load the binary is by running

#| eval: false
devtools::load_all()

However, this compiles code without optimisation, cuasing the model to run much slower. Similarly, by default, pkgbuild::compile_dll has argument debug=TRUE which leads causes debug symbols to be included and slow runtime.

If you compile with optimisation, the code runs much faster. By default, R CMD INSTALL, devtools::install() or make will compile with optimisation. You will want to clean first.

When developing, you can still get optimisation by compiling as follows. After changing some cpp code, run

pkgbuild::compile_dll(debug=FALSE, compile_attributes = FALSE)
devtools::load_all()

The first line recompiles with optimisation, and the second line loads the package with the new code. If the code is already compiled, devtools::load_all won’t recompile it. If you skip the first line, your code will be recompiled with debug symbols, and will be slow.

Also, including compile_attributes = FALSE in the first line avoids the need to recompile the Rcpp exports.

Profiling

Building/updating the website

Website maintained on a separate branch gh-pages. Website built with pkgdown, drawing on files in the package such as

  • Readme
  • News
  • folder man:
  • folder vignettes:

To build and push the website, do as follows:

  1. Open up package on develop or master
  2. Clone branch containing the website into folder gh-pages, within root of pacakges
#| eval: false
git clone --branch gh-pages --single-branch git@github.com:traitecoevo/plant.git gh-pages
  1. Rebuild the website using pkgdown::build_site()
  2. Go into the gh-apges folder, commit and push

If you just want to build the wsbite locally, you only need to complete step 3.

Program design

plant is build as an R package using C++ code, linked via Rcpp. Rcpp provides “provides R functions as well as C++ classes which offer a seamless integration of R and C++.”

Two common challenges with using Rcpp are 1) that there is a lot of interface code (or “boilerplate”) needed, 2) Rcpp “modules” can be slow to load. To overcome these we use the package RcppR6 to generate the interface. The RcppR6 package aims to provide a simple way of generating boilerplate code for exposing C++ classes to R. It is similar in many ways to Rcpp “modules” but without the slow load time. RcppR6 creates an interface using Rcpp’s “attributes” facilities to build the actual R/C++ glue. The R6 R package is the reference class that we use for wrapping the generated class. To achieve the link, class definitions are written into a YAML file.

Templating

We use templating to enable plant to work with multiple strategies and lets us write code that is strategy generic. The templated class definitions are found in inst/include/plant and their function implementations are found either in the .h files or in /src.

We then use the RcppR6 package combined with the .h files and class definitions (found in inst/RcppR6_classes.yml) into Rcpp which we can then be used in the .R files (in /R) or exported.

Templating Resources:

Classes and their functions

The main c++ header files (i.e. *.h) are stored within the directory inst/include/plant/. In many cases, the implementations is written directly into the header files. Otherwise, the corresponding source file (i.e. *.cpp) is stored within the folder src.

inst/include/plant.h includes them all. The headers are grouped in folders by role:

Folder Holds
models/ each model’s physiology and environment (ff16_model.h, ff16_environment.h, k93_model.h, tf24_model.h and its variants), and what models share: canopy shape, extrinsic drivers, the resource spline
community/ the individual (individual.h), the node, species and patch built from it, and their stochastic counterparts (stochastic_node.h, stochastic_species.h, stochastic_patch.h)
solvers/ what runs a patch or an individual through time: deterministic_patch_runner.h (the deterministic solver, formerly SCM), stochastic_patch_runner.h, their shared base patch_runner.h, individual_runner.h, the node schedule (schedule.h, schedule_utils.h) and events (events.h)
disturbances/ disturbance regimes and the fitness window
gradients/ the census and the exact gradients of a stand
numerics/ quadrature (qag.h, qk.h and their rules), root-finding (uniroot.h), and gradient helpers
(top level) parameters.h, control.h, util.h, and the RcppR6 support headers

The ODE solver, interpolators and splines come from the odelia package.

Models

The foundation of the plant package are sub-models for an individual species’ physiology, which plant calls models. The FF16 model is the default model. However, the plant package has been written to enable new models to be added, and then use the same machinery for modelling size-structured population dynamics, we do this by using templating in c++ (see Templating).

Adding a new model requires some changes to be made throughout the code and then the code recompiled. To achieve this, you’ll need to download the package code from Github (new models cannot be added using an installed R package) and follow the instructions below. For a fuller walkthrough of the biology and structure of a model, see implementing a model.

As a naming convention, models are named using two initials (eg. the first two authors) followed by a year, e.g. FF16. A single letter suffix can be added to indicate a variant of an existing model, e.g. TF24t.

Adding a new model

Let’s assume you want to add a new model called XX99. plant has no generator for a new model: every templated binding and every R dispatch table names each model by hand, and the checklist of what to change is the plant-new-model skill in plant’s .claude/skills/. Most new models are a variant of an existing model, written as a subclass of it (as TF24f, TF24t and TF24w are of TF24). The main new files are:

  • inst/include/plant/models/xx99_model.h (C++ header describing the model, with its methods defined inline)
  • a block for the new model in src/model_exports.cpp (the schedule exports for its Parameters, and its arm of strategy_scientific_version())
  • R/xx99.R (R constructors for the new model’s individual and parameters, and its expand_state function)
  • tests/testthat/test-model-xx99.R (Tests of the new model)

In addition, files such as inst/RcppR6_classes.yml, inst/include/plant.h, R/strategy_support.R, R/default_parameters.R and tests/testthat/helper-plant.R must name the new model; the skill lists every one.

Before trying to add a new model, make sure your project is at a state where you are happy to go back to if you decide to, ie: do this in a new branch or fork. There is no easy way to undo the changes so you will have to do git reset --hard HEAD if you want to go back. You will still have to delete the new and untracked files listed above.

Then run make RcppR6 (rather than RcppR6 directly, because the target re-adds guards a bare regeneration drops), make roxygen and make test to regenerate the interface and run the test suite with your new model.

If that works you can then modify inst/include/plant/models/xx99_model.h to reflect any changes in biology you desire for the new model. If you make changes to your new model’s parameters you will have to update the files:

  • inst/RcppR6_classes.yml - see top-level section under the new model’s name, describing your parameters.
  • tests/testthat/test-model-xx99.R with suitable tests for your new model.

Then recompile and test as above.

After the project compiles, you will notices even more files that have changed, for example in files like NAMESPACE, R/RcppExports.R etc. These files are auto-generated and therefore modifications are triggered from files listed above.