CopyXnat¶

A utility for downloading or copying projects between XNAT servers.

This is a prerelease. It may not work as intended, and may lead to data loss. Please use caution when running on production servers

Quick start¶

Please make sure your XNAT archive and PostgreSQL database are fully backed up before running CopyXnat on a real XNAT server. CopyXnat may modify and overwrite your data.

Install and run using pip¶

Python 2.7 or 3.6+ is required, and pip should be installed.

It is recommended that you do not modify your system Python installation. Use a tool such as Homebrew (macOS) to install the latest python version without affecting your system python. Then, create a dedicated virtual python environment using virtualenv and virtualenvwrapper or similar tools.

Once you have a suitable python environment set up, install with pip:

pip install copyxnat --upgrade
copyxnat --help

You can use pip’s --user flag to install dependencies in the user space instead of globally.

Install and run using Docker¶

If you have Docker installed you can run CopyXnat in a container. Note that interactive mode (-it) is generally required in order to enter server passwords where prompted.

docker run --rm -it tomdoel/copyxnat --help

Example use cases¶

Show project contents¶

#### Print a list of projects, subjects etc on an XNAT server

copyxnat show -s "https://YOUR-SERVER-URL" -u "YOUR-USER-NAME"

#### Show differences between two projects

copyxnat compare --src-host "https://SOURCE-SERVER-URL" --src-user "XNAT-USER-SOURCE" --project "PROJECT-NAME-SRC:PROJECT-NAME-DST" --dst-host "https://DEST-SERVER-URL" --dst-user "XNAT-USER-DEST"

Download projects¶

#### Download XNAT projects onto your computer

copyxnat export --src-host "https://YOUR-SERVER-URL" --src-user "YOUR-USER-NAME" --project "PROJECT1,PROJECT2"

#### Download all projects you have access to onto your computer

copyxnat export --src-host "https://YOUR-SERVER-URL" --src-user "YOUR-XNAT-USER-NAME"

Copy projects from one XNAT server to another¶

#### Duplicate a project on a destination server

copyxnat copy --src-host "https://SOURCE-SERVER-URL" --src-user "XNAT-USER-SOURCE" --project "PROJECT-NAME" --dst-host "https://DEST-SERVER-URL" --dst-user "XNAT-USER-DEST"

#### Duplicate a project and rename

copyxnat copy --src-host "https://SOURCE-SERVER-URL" --src-user "XNAT-USER-SOURCE"  --project "PROJECT-NAME:NEW-PROJECT-NAME" --dst-host "https://DEST-SERVER-URL" --dst-user "XNAT-USER-DEST"

#### Duplicate a project on the same server (with a new name)

copyxnat copy --src-host "https://SERVER-URL" --src-user "XNAT-USER-SOURCE"  --project "PROJECT-NAME:NEW-PROJECT-NAME" --dst-host "https://SERVER-URL" --dst-user "XNAT-USER-DEST"

Maintenance¶

#### Rebuild catalog for all experiments

copyxnat rebuild-catalog -s "https://YOUR-SERVER-URL" -u "YOUR-USER-NAME"

#### Reset OHIF viewer for all experiments

copyxnat ohif -s "https://YOUR-SERVER-URL" -u "YOUR-USER-NAME"

Command-line parameters¶

The parameters vary depending which command you are running. To see help for a particular command, run with the command name and --help, for example:

copyxnat copy --help

Parameter	Short form	Default	Description	Notes
`--help`	-h		Shows general help
command `--help`	command `-h`		Shows detailed help for this command
`--version`			Shows version number
command			Command to run. One of `copy`, `export`, `show`, `compare`, `check-datatypes`, `ohif`, `rebuild-catalog`	See above for examples of each command.
`--src-host` hostname	`-s`		hostname of source XNAT server
`--src-user` username	`-u`		XNAT username for source server
`--project` _projectstring	`-p`	all projects	list of project IDs to process	Use a colon if you want to rename project IDs between source and destination servers, eg `srcId:dstId`. To process multiple projects, separate the IDs with commas, eg `proj1,proj2`. Do not include any spaces. You can do both, e.g. `prj1,prj2:dst2,prj3`
`--dst-host` hostname	`-d`		hostname of destination XNAT server	Only for some commands
`--dst-user` username	`-w`		XNAT username for destination server	Only for some commands
`--fix-scan-types`	`-f`	False	Fix undefined scan types on the copy	If the source server does not correctly specify the scan type, CopyXnat will attempt to determine the scan type and set it on the destination server
`--transfer-mode` mode	`-t`	file	Specifies how files will be copied or downloaded. One of `file`, `zip`, `meta`, `rsync`	`file` to copy one file at a time. `zip` to zip up each resource collection as a zip file (generally faster). `meta` to transfer metadata only, not resource files. `rsync` to transfer files using rsync (experimental)
`--limit-subjects` limit	`-l`	no limit	Process a maximum number of subjects specified by the integer limit.	During a copy, the limit will exclude subjects which already exist on the destination and have not changed.
`--skip-existing`	`-n`	False	Only copy new experiments; do not synchronise existing experiments	Speeds up copying process when some experiments have already been copied to the destination server. However will not copy over new scans and resources that have been added to existing experiments.
`--verbose`	`-v`	False	Output additional information	For debugging and for providing additional information about what the command is doing.
`--insecure`	`-k`	False	Do not verify server certificates	Do not use in production as is vulnerable to MITM. For testing or working with self-signed certificates within a secure private network.
`--dry-run`	`-y`	False	Do not make changes on the destination server	Useful for seeing what changes a command may make before running it for real. May cause errors in some commands because the expected changes on the destination server didn’t happen.
`--overwrite-existing`	`-o`	False	Overwrite existing data on the destination server	When copying, will modify and overwrite existing data
`--ohif-rebuild`	`-g`	False	When copying, trigger an OHIF viewer rebuild after each experiment copy.	The OHIF Viewer does not always detect new data being added to existing experiments. This will cause a session rebuild when data are copied, but this will take additional time.
`--rsync-src-user`	`-i`		ssh source server username for copy via rsync	Experimental. Only used for copy using `--transfer-mode rsync`
`--rsync-dst-user`	`-w`		ssh destination server username for copy via rsync	Experimental. Only used for copy using `--transfer-mode rsync`

Supported XNAT data levels¶

CopyXnat can process the following XNAT data hierarchies.

Projects
- Project Resources
- Subjects
  - Subject Resources
  - Experiments
    - Experiment Resources
    - Scans
      - Scan Resources
    - Assessors
      - Assessor In-Resources (copy not supported)
      - Assessor Out-Resources
    - Reconstructions (copy not supported)
      - Reconstruction In-Resources
      - Reconstruction Out-Resources

Current limitations¶

Reconstructions can be downloaded but cannot be copied to another server
Assessor In-Resources can be downloaded but cannot be copied to another server
Support for eCat files is limited and untested

Using CopyXnat in your own code¶

Importing via pip¶

You can import and run the module with pip and then import from your code

from copyxnat.xnat.run_command import run_command
from copyxnat.commands.show_command import ShowCommand

result = run_command(command=ShowCommand,
                     src_host=src_host,
                     src_user=src_user,
                     src_pw=src_pw,
                     project_filter=None)

Local development: running the command-line utility directly¶

You can run directly from the source code. This is primarily useful if you are modifying the code.

pip install -r requirements-dev.txt
python copyxnat --help

Links¶

Authors¶

Tom Doel

Copyright¶

License¶

This project is released under the MIT License. Please see the license file for details.