Quick ToolboxToolbox Tutorial for the Gold Lab
by Adrian Radillo, 13 Jan 2020
What's ToolboxToolbox?
ToolboxToolbox is a collection of MATLAB routines gathered in this GitHub repository. Its aim is to automate the MATLAB Path setup on a project-by-project basis. I refer you to the readme page and the Wiki for detailed info on its capabilities. In the present tutorial, I will explain how an example MATLAB project from the Gold Lab may be set up with ToolboxToolbox.
An example set-up in the Gold Lab
Installing ToolboxToolbox
The first step is to install ToolboxToolbox on the machine from which you wish to develop your MATLAB code.
The standard way to install ToolboxToolbox is described in the following steps:
1/ Find out your MATLAB userpath by typing the following at the MATLAB console:
2/ Clone ToolboxToolbox to your computer at the path returned by the userpath command above. If you wish to do this via a UNIX terminal, you may type the following at the command line (this assumes that git is configured on your computer): 3/ If no startup.m file exists in your userpath, create one and paste onto it the contents from this example file. If you already have a startup.m file and know what it does, then you should be able to understand what the example file just mentioned does, and to fuse the two files in a way that makes sense for you. At this point, ToolboxToolbox is installed on your computer. Congratulations! Every once in a while a new version is available on GitHub. To get it, simply use the git pull command. From the UNIX command line, for instance, you would do:
Setting up a project to use ToolboxToolbox
Now that ToolboxToolbox is installed on your computer, you might notice that at MATLAB startup, MATLAB's path is reset to factory state, and ToolboxToolbox is added to the path. This is what ToolboxToolbox does with its default settings. This behavior may be changed in the startup.m file, but this falls out of the scope of the present tutorial.
The example project that we will work with for now is a GitHub repository named Lab_TestRepository. Note that ToolboxToolbox is particularly well suited for GitHub-hosted projects. Although we won't cover this here, it can also work with non-git projects. A key distinction operated within ToolboxToolbox is the one between projects and toolboxes. Both a project and a toolbox are folders that can be added to the MATLAB Path. However, projects are folders that you are frequently modifying (because you are developing the code they contain for instance), whereas toolboxes are folders that you (almost) never manually edit. For ToolboxToolbox, our Lab_TestRepository folder will be a project because it represents the MATLAB project that we want to work on at the moment. As such, it is convenient to place/clone it into the default location for ToolboxToolbox projects, which is the 'projects/' folder located in your userpath folder. I recommend you create this folder (and the 'toolboxes/' folder too) if it doesn't exist. Here is the code to do this, and clone our project from the command line. What makes our Lab_TestRepository folder a ToolboxToolbox project is both the fact that it lives in the 'projects/' folder and that somewhere in its subfolders lies a file named 'Lab_TestRepository.json' with the appropriate configuration content. Note that the file name should match the folder name. In our case, the .json file lives in Lab_TestRepository/dummy_folder1/dummy_folder2/Lab_TestRepository.json and its content is:
We notice that there are 6 JSON cells in this file. The two compulsory fields in every cell are "name" and "type". The "name" field should be the name of the folder that will be added (with its subfolders) to the MATLAB path whenever the present project is used with ToolboxToolbox. Let me remind you here that, with the default settings, such folders should be placed in the 'projects/' or in the 'toolboxes/' folders. Furthermore, it is good practice not to create any folder yourself in the 'toolboxes/' folder. ToolboxToolbox is smart enough to update (with a pull from GitHub for instance) the contents of the toolbox if the folder already exists, and to create the folder (with a clone from GitHub) if it doesn't exist. With projects, I recommend that you manually clone your repos and set them to the appropriate branch yourself (if you use git branches). Now, let's explain the cells from the .json file one by one. The first cell is:
The first cell should always be a self-reference to the project. The "name" field is set to "Lab_TestRepository" because this is the name of the folder (within 'projects/') that contains my present project. The field "type" is set to "git" because this is a git-managed project. The field "url" is set to the GitHub URL of the project (this way, if I ask ToolboxToolbox to update this project, it will do so from that URL). Finally, the "update" field is set to "never" because I usually avoid automatic update of my ToolboxToolbox projects. The reason for this is that I often develop my code from several computers, and usually have several branches in each git project, and forbidding ToolboxToolbox update is the easiest way to set my local repository into the state I want them to be.
Advanced remarks on branches
Let's be more explicit about the last point. You may notice that our Lab_TestRepository has the three branches dev, github and master. If I wish to work on the dev branch, all I need to do is set my repo to that branch
and call tbUseProject('Lab_TestRepository') from MATLAB. If, later on, I wish to fall back to the master branch, I simply revert back to it:
and I can call again tbUseProject('Lab_TestRepository') from MATLAB.
As a rule of thumb, if the "update" field of the ToolboxToolbox configuration for a project is set to "never", then ToolboxToolbox won't even attempt a git pull on your project. If the "update" field is left blank or is missing, ToolboxToolbox will attempt a git pull from your project, regardless of the branch it is on.
The second cell from our .json file is:
This cell is a good example of how to set an other ToolboxToolbox project as a dependency to your current ToolboxToolbox project. Here, I assume that I have already cloned the project Task_SingleCP_DotsReversal from GitHub into my 'projects/' folder and set it to the state I want (i.e. a specific git branch, if needed). The only new field, compared to the first cell, is the "toolboxRoot" field, which is set to "#Task_SingleCP_DotsReversal". This setting let's ToolboxToolbox know that the folder (and its subfolders) that should be added to the MATLAB path lives in "projects/Task_SingleCP_DotsReversal". If you don't specify this field, ToolboxToolbox will treat the whole configuration cell as a toolbox instead of a project, and thus clone the GitHub repo into your 'toolboxes/' folder.
The third cell is:
The interesting new field it contains is "flavor". You may set this field to any string that can be provided as argument to the 'git checkout' command. Thus, you can provide any tree-ish. Most commonly, I use either a branch name, the first few characters of a commit number, or a tag name (i.e. a Release name). In our present example, the "flavor" field is set to "v1.0", which is both a tag and a particular release of the Lab_Matlab_Control toolbox. Because no "toolboxRoot" field appears for this dependency, ToolboxToolbox will interpret the dependency as a toolbox and look for it (or clone it) in the toolboxes/ folder. Furthermore, because we have specified a flavor, the folder name will have the string "v1.0" appended to its name. So, the actual folder (and its subfolders) that will be added to my MATLAB path upon calling ToolboxToolbox for my Lab_TestRepository project will be "/path/to/your/matlab/userpath/toolboxes/Lab_Matlab_Control_v1.0/". As one should expect, ToolboxToolbox will make sure that the aforementioned folder is set to the "v1.0" release state (by calling "git checkout v1.0").
Cells 4 and 6 are very similar, so I will treat them together.
Notice that the "type" field is now set to "include" as opposed to "git". This triggers a particular behavior in ToolboxToolbox. Namely, ToolboxToolbox will fetch the settings for these toolboxes from the publicly available online ToolboxRegistry. You can see for yourself what the settings for Lab_Matlab_Utilities and mQUESTPlus are. Notice how having these settings directly hosted on the public registry simplifies the syntax. No other field is required in our .json file. What is more, if any of these toolboxes had required additional dependencies, these would have been listed in the ToolboxRegistry, and ToolboxToolbox would have fetched them automatically for us. The "include" feature is thus very powerful for projects with nested dependencies. As a rule of thumb, always use the type "include" when using a toolbox from the ToolboxRegistry. You may also add your own toolboxes to this registry as explained in the README.
Finally cell 5 is:
It contains no unknown field to us. Upon seeing this dependency setting, ToolboxToolbox will attempt to go to "/path/to/your/matlab/userpath/toolboxes/mgl/". If the folder exists, it will then call "git pull" from within it. If it doesn't, it will call "git clone https://github.com/justingardner/mgl.git" from /path/to/your/matlab/userpath/toolboxes/.
By now, hopefully, you understand how the ToolboxToolbox .json project configuration file works. All that remains to do to set your MATLAB path as you want it for your "Lab_TestRepository" project is to call from the MATLAB console:
tbUseProject('Lab_TestRepository')