Storutil is run from the command line. It uses file locking, so it can be run
on an active server. To free up storage, run from your code directory:
python storutil.py -homedir /path/to/server -prune-localstore -prune-entangled
To generate the report, run: (-report-html and -report-tab are also available)
python storutil.py -homedir /path/to/server -report-csv /path/report.csv
You can do both at once, with one pass through the filesystem to save I/O. The report will show post-prune numbers.
python storutil.py -homedir /path/to/server -prune-localstore -prune-entangled -report-html /path/report.html
Your entangled and localstore directories should be pruned periodically. You can run it manually, or schedule the process using cron. Using cron, the stdout and stderr should be written to a logfile. Storutil will not start if another one is already running on the same server directory, so you do not have to worry about cron starting multiple overlapping copies.
The report shows both raw and apportioned (App'd) bytes for each user. Apportioned bytes divides the storage used by multiple-recipient messages evenly among the recipients. Suppose that a 16 MByte message is sent to four users on your server. The message only takes up 16 MBytes of your storage. Raw bytes would show 16 MBytes for each recipient, overstating the usage. Apportioned bytes would show 4 Mbytes per recipient for that message. If you intend to impose quotas or bill for usage, Total App'd Bytes is the correct field to use.
To decide what to delete, the prune function looks at the date field in each block. If you have set custom max_age values in your server config.txt, storutil will use those in place of the defaults for the localstore prune. For message announcements and address claims, the program looks at each message or claim individually and rewrites the block with the expired entries removed. If all the entries are expired, the block is deleted.
The "Approx total bytes reclaimed" value does not include the small amount of space freed by trimming message announcement and address claim blocks, and will therefore be slightly low.
The report function writes out a tab-delimited file as it reads the localstore. This file is then sorted to match data blocks with messages. The second report pass reads the sorted file and writes a second intermediate file, which is then sorted to match messages with keys. The third report pass reads the second intermediate file and writes out the report. The report function uses the system sort utility ("gsort" or "sort" unless you specify otherwise.)
Command line options: