The HPC utility
The HPC utility (or hpcutil) is an CLI tool developed by the Technical Group of DCCN. The usage of it is similar to the CLI of git where a group of (sub-)sub-commands are provided in a hierarchical manner.
The in-terminal help for a subcommand and the supported flags of it are always available via the -h option. The CLI also supports tab-completion in BASH which means the suggested subcommands or flags is available by pressing the TAB key twice.
One can also use the avail subcommand to show a list of available commands, e.g.
$ hpcutil avail
+--------------------------------+--------------------------------+
| COMMAND | DESCRIPTION |
+--------------------------------+--------------------------------+
| hpcutil avail | Print availab sub-commands. |
+--------------------------------+--------------------------------+
| hpcutil cluster config | Print Torque and Moab server |
| | configurations. |
+--------------------------------+--------------------------------+
| hpcutil cluster job meminfo | Print memory usage of a |
| [id] | running job. |
+--------------------------------+--------------------------------+
| hpcutil cluster job trace | Print trace log of a job. |
| [id] | |
+--------------------------------+--------------------------------+
| ... ... ... | ... ... ... |
+--------------------------------+--------------------------------+
The two major subcommands, cluster and webhook, are detailed further below:
The cluster subcommand
The cluster subcommand can be used to retrieve job or resource (node, software license) information of the HPC cluster. To get the in-terminal help of the cluster subcommand, one uses the following command in the terminal:
$ hpcutil cluster -h
where -h is optional.
It shows another level of subcommands that are available, for instance:
Available Commands:
config Print Torque and Moab server configurations.
job Retrieve information about a cluster job.
matlablic Print a summary of the Matlab license usage.
nodes Retrieve information about cluster nodes.
qstat Print job list in the memory of the Torque server.
One can then take one from those available command to move onto another level of the sub-commands. For example, if one wants to get nodes resource information, one does
$ hpcutil cluster nodes
and via the supported subcommands shown below, you will be able to get resource information such as the disk/memory usage on a list of cluster nodes, or a summary table of the running VNC sessions on the cluster.
Available Commands:
diskfree Print total and free disk space of the cluster nodes.
memfree Print total and free memory on the cluster nodes.
vnc Print list of VNC servers on the cluster or a specific node.
Example: list MATLAB licenses allocated by DCCN users
$ hpcutil cluster matlablic
Example: list VNC sessions
To get all VNC sessions running on the cluster’s access nodes, one can do:
$ hpcutil cluster nodes vnc
To get VNC sessions on a given host (e.g. mentat001.dccn.nl), one does:
$ hpcutil cluster nodes vnc mentat001.dccn.nl
To get VNC sessions owned by a given user (e.g. honlee), one does:
$ hpcutil cluster nodes vnc -u honlee
One could combine the last two examples to find VNC sessions owned by a user on a specific host. For example, the following command will find VNC sessions owned by user honlee on host mentat001.dccn.nl.
$ hpcutil cluster nodes vnc -u honlee mentat001.dccn.nl
Example: show all cluster jobs
$ hpcutil cluster qstat
Example: check memory utilization of a running job
Assuming a running job with ID 1234567, the owner of the job can perform the following command to check the memory usage:
$ hpcutil cluster job meminfo 1234567
or apply the command to the watch command to monitor the memory usage in real time:
$ watch hpcutil cluster job meminfo 1234567
Example: get job’s trace log
Assuming a job with ID 1234567, the job trace log (in the last 3 days) can be obtained from the Torque server via the following command:
$ hpcutil cluster job trace 1234567
The webhook subcommand
The webhook subcommand is used to manage the webhook facility of the HPC cluster. The webhook facility allows you to setup automatic job submission to the HPC cluster upon events of an external service/application supporting webhooks.
To get the in-terminal help of the webhook subcommand, one uses the following command in the terminal:
$ hpcutil webhook -h
where -h is optional.
Instructions about creating and enabling webhook is provided by This link. The instruction here will focus on the management perspective of the webhooks.
There are five subcommands supported:
Available Commands:
create Create a new webhook.
delete Delete an existing webhook.
info Retrieve information of an existing webhook.
list List webhooks.
trigger Trigger webhook manually with a payload.
Example: create a new webhook
Assuming that we want to create a new webhook associated with a Torque cluster job script qsub.sh (an example can be found here, we can used the following command:
$ hpcutil webhook create qsub.sh
On success, it returns the actual webhook URL which can be then registered at a webhook trigger, such as the webhook for a GitHub repository.
One could also give a short descript to the created webhook so that it can be easily identified later. This is done through the -n flag. For example,
$ hpcutil webhook create qsub.sh -n "My first webhook"
Example: list available webhooks
For listing available webhooks, one does:
$ hpcutil webhook list
Every returned webhook has a unique id. For instance,
1e846adf-462b-4a7b-b183-651909072b79
Description : My first webhook
Creation time : 2019-04-03T08:28:38Z
Script path : /home/tg/honlee/qsub.sh
Webhook URL : https://hpc-webhook.dccn.nl:443/webhook/1e846adf-462b-4a7b-b183-651909072b79
This unique id, i.e. 1e846adf-462b-4a7b-b183-651909072b79 in the example above, is used in info, delete and trigger subcommands to identify a webhook.
Tip
The tab-completion is also applicable to the webhook ids. This is useful to selecting a valid webhook id for the info, delete and trigger subcommands.
Example: trigger a webhook
Normally the webhook is triggered externally upon an event (e.g. a commit to GitHub, a new message posted on Twitter, etc.). The trigger subcommand here is only meant to help you test the webhook by manually trigger it with a provided payload.
Assuming we have prepared a GitHub webhook payload file called payload.json (some payload examples can be seen here), we can trigger a webhook with ID 1e846adf-462b-4a7b-b183-651909072b79 with the payload using the following command:
$ hpcutil webhook trigger 1e846adf-462b-4a7b-b183-651909072b79 -l payload.json -t json
where the -t json is redundent in this case as by default, the payload is take as JSON format. If your payload is in another format (e.g. XML or plain text), you will need to use the -t option to specify it.