Post-processing scripts

From NZBGet
(Redirected from Postprocessing scripts)
Jump to: navigation, search

Contents

Intro

NZBGet version: 11.0 and after
NOTE: the information in this document applies to NZBGet 11. Version 11 introduces a new post-processing scripts concept. This document is not applicable to NZBGet 10 or older versions.

After the download of nzb-file is completed NZBGet can call post-processing scripts (pp-scripts). The scripts can perform further processing of downloaded files such es delete unwanted files (*.url, etc.), send an e-mail notification, transfer the files to other application and do any other things. Please note that the par-check/repair and unpack are performed by NZBGet internally and are not part of post-processing scripts. You can activate par-check/repair and unpack without using of any post-processing scripts (NOTE: versions up to 9.1 required a post-processing script for unpack. Version 10.0 and later have built-in unpack).

Using pp-scripts

There is an option ScriptDir defining the location of post-processing scripts. To make script available in NZBGet put the script into this directory. Then go to settings tab in web-interface (if you were already on settings tab switch to downloads tab and then back to settings tab to reread the list of available pp-scripts from the disk).

Menu at the left of page should list all pp-scripts found in ScriptDir. Select a script to review or change its options (if it has any).

When a new nzb-file is added to queue it doesn't have any pp-scripts assigned to it by default. You can define what scripts should be called for the nzb-file using option DefScript in section POST-PROCESSING SCRIPTS. You can also define a different set of pp-scripts for each category.

Please note that these settings define "defaults" for nzb-file. It's possible to alter the assigned pp-scripts for each nzb-file individually. To do so click on the nzb-file in the list of downloads, then click on PP-Parameters.

If you use more than one script for one nzb-file it can be important to set the correct order of execution. The order is controlled by the option ScriptOrder. That is a global setting affecting all categories and also defining the order scripts are listed in the edit download dialog.

The rest of this article describes how to write pp-scripts.

Configuration definition

Each script must have a configuration definition. The definition starts with signature:

### NZBGET POST-PROCESSING SCRIPT

To end the definition the same signature is used.

Example of very simple post-processing script:

#!/bin/sh 

##############################################################################
### NZBGET POST-PROCESSING SCRIPT                                          ###

# Print test message.
#
# This is a test script. It prints one message to log. Here in the
# long description we could write instructions for user.
#
# The long description can have multiple paragraphs when needed. Like
# this one.

### NZBGET POST-PROCESSING SCRIPT                                          ###
##############################################################################

echo "post-processing script test"

The first part of configuration is a short description:

# Print test message.

This line is a short description of what our pp-script does. The short description stays near the script name in the edit download dialog. This should be a short one sentence description. The long description is separated with an empty line.

What is considered a script

NZBGet scans all files in ppscript-directory (option ScriptDir) and all files in first-level subdirectories looking for script definition signature (### NZBGET POST-PROCESSING SCRIPT). If a signature is found, the file is considered to be a script.

NOTE: for performance reasons NZBGet scans only first 10KB of the script. Put the script definition at the beginning of the script. The script definition can be longer than 10KB but it must start within first 10KB of the script.

Configuration options

Let's say we are writing a script to send E-Mail notification. The user need to configure the script with options such as SMTP-Server host, port, login and password. To avoid hard-coding of these data in the script NZBGet allows the script to define the required options in the configuration section. The options are then made available on the settings page for user to configure.

Every option belong to a configuration section. In the next example we define a new section with two options:

##############################################################################
### OPTIONS                                                                ###

# SMTP server host.
#Server=smtp.gmail.com

# SMTP server port (1-65535).
#Port=25

If your script have a lot of options you can define more than one section but you should try to avoid this. Splitting your script into several scripts may be a better alternative.

When the user saves settings in web-interface the pp-script configuration options are saved to NZBGet configuration file using the script name as prefix. For example:

EMail.py:Server=smtp.gmail.com
EMail.py:Port=25

When the pp-script is executed NZBGet passes the values of all configuration options to the script using environment variables. This will be described later in this document.

Since NZBGet automatically adds script name when saving the options, the options defined in different scripts are not interfere with each other or with NZBGets own options. It's not necessary to add a prefix to option names.

Bad:

# SMTP server host.
#EMailServer=smtp.gmail.com

# SMTP server port (1-65535).
#EMailPort=25

Good:

# SMTP server host.
#Server=smtp.gmail.com

# SMTP server port (1-65535).
#Port=25

Notice the options EMailServer and EMailPort renamed to Server and Port.

Post-processing parameters

Sometimes a script might need an option specific for nzb-file. Best example - unpack password. NZBGet allows pp-script to define nzb-file specific options, so called post-processing parameters (pp-parameters). PP-Parameters must be declared in the configuration definition of the script. NZBGet reads the definition and adds necessary ui-elements to edit download dialog on page pp-parameters.

Post-processing parameters are declared using configuration section "POST-PROCESSING PARAMETERS". For example a video recode script could have a parameter "Resolution":

##############################################################################
### POST-PROCESSING PARAMETERS                                             ###

# Video resolution (default, 480p, 720p, 1080p).
Resolution=default

Testing script

To test the script save it to text file, make it executable and put into ScriptDir. Then refresh the page in web-browser (a reload of NZBGet is not needed), click on any download in the download list, then click on button PP-Parameters. Here select your script. When the download is finished our script is executed.

TIP: to test a script after you made changes to it, use command "Post-process again" from the history page in web-interface.

Our simple test script prints a message to standard output stream:

echo "post-processing script test"

NZBGet receives the message and adds it to Messages-log. You can set message kind using prefixes:

echo "[DETAIL] This is detail-message"
echo "[INFO] This is info-message"
echo "[WARNING] This is warning-message"
echo "[ERROR] This is error-message"

Script environment

NZBGet passes different information to the script. All data is passed as environment variables (env-vars).

NZB-File information

The information about nzb-file which is currently processed:

Env-var Description
NZBPP_DIRECTORY path to destination dir for downloaded files.
NZBPP_NZBNAME User-friendly name of processed nzb-file as it is displayed by the program. The file path and extension are removed. If download was renamed, this parameter reflects the new name.
NZBPP_NZBFILENAME Name of processed nzb-file. If the file was added from incoming nzb-directory, this is a full file name, including path and extension. If the file was added from web-interface, it's only the file name with extension. If the file was added via RPC-API (method append), this can be any string but the use of actual file name is recommended for developers.
NZBPP_CATEGORY Category assigned to nzb-file (can be empty string).
NZBPP_PARSTATUS Result of par-check:
  • 0 = not checked: par-check is disabled or nzb-file does not contain any par-files;
  • 1 = checked and failed to repair;
  • 2 = checked and successfully repaired;
  • 3 = checked and can be repaired but repair is disabled.
NZBPP_UNPACKSTATUS Result of unpack:
  • 0 = unpack is disabled or was skipped due to nzb-file properties or due to errors during par-check;
  • 1 = unpack failed;
  • 2 = unpack successful.

Example: test if download failed

if [ "$NZBPP_PARSTATUS" -eq 1 -o "$NZBPP_UNPACKSTATUS" -eq 1 ]; then
	echo "[ERROR] This nzb-file has failure status (par-check or unpack failed)"
fi

NZBGet configuration options

All NZBGet configuration options are passed using env-vars with prefix NZBOP_. The variable names are written in UPPER CASE. For Example option ParRepair is passed as environment variable NZBOP_PARREPAIR. The dots in option names are replaced with underscores, for example NZBOP_SERVER1_HOST. For options with predefined possible values (yes/no, etc.) the values are passed always in lower case.

Example: test if option "Unpack" is active

if [ "$NZBOP_UNPACK" != "yes" ]; then
	echo "[ERROR] Please enable option \"Unpack\" in nzbget configuration file"
fi

Post-processing script configuration options

Post-processing script configuration options (pp-options) are passed using env-vars with prefix NZBPO_. There are two env-vars for each option:

  • one env-var with the name exactly as defined by pp-option;
  • another env-var with the name written in UPPER CASE and with special characters replaced with underscores.

For example, for pp-option Server.Name two env-vars are passed: NZBPO_Server.Name and NZBPO_SERVER_NAME.

NOTE: In a case the user has installed the script but have not saved the configuration, the options are not saved to configuration file yet. The pp-script will not get the options passed. This is a situation your script must handle. You can either use a default settings or terminate the script with a proper message asking the user to check and save configuration in web-interface. Example (python):

required_options = ('NZBPO_FROM', 'NZBPO_TO', 'NZBPO_SERVER', 'NZBPO_PORT', 'NZBPO_ENCRYPTION',
	'NZBPO_USERNAME', 'NZBPO_PASSWORD', 'NZBPO_FILELIST', 'NZBPO_BROKENLOG', 'NZBPO_POSTPROCESSLOG')
for optname in required_options:
	if (not optname in os.environ):
		print('[ERROR] Option %s is missing in configuration file. Please check script settings' % optname[6:])
		sys.exit(POSTPROCESS_ERROR)

Post-processing script parameters

Post-processing parameters are passed using env-vars with prefix NZBPP_. Like pp-options there are two env-vars for each parameter:

  • one env-var with the name exactly as defined by pp-parameter;
  • another env-var with the name written in UPPER CASE and with special characters replaced with underscores.

For example, for pp-parameter Resolution two env-vars are passed: NZBPO_Resolution and NZBPO_RESOLUTION.

NOTE: when nzb-file is added to download queue it doesn't have any pp-parameters. Only if the user opens edit download dialog and changes pp-parameters they are getting assigned to nzb-file and will be passed to pp-script. The script MUST work properly even if no pp-parameters were passed; it MUST use default values for pp-parameters.

Example (bash):

Resolution=$NZBPP_RESOLUTION
if [ "$Resolution" = "" ]; then
	Resolution="480p"
fi

Example (python):

Resolution="480p"
if 'NZBPP_RESOLUTION' in os.environ:
	Resolution=os.environ['NZBPP_RESOLUTION']

Exit codes

NZBGet checks the exit code of the script and sets the status in history item accordingly. Scripts can use following exit codes:

Exit code Description
93 Post-process successful (status = SUCCESS).
94 Post-process failed (status = FAILURE).
95 Post-process skipped (status = NONE). Use this code when you script terminates immediately without doing any job and when this is not a failure termination.
92 Request NZBGet to do par-check/repair for current nzb-file. This code can be used by pp-scripts doing unpack on their own.

Scripts MUST return one of the listed status codes. If any other code (including 0) is returned the history item is marked with status FAILURE.

Example: exit code

POSTPROCESS_SUCCESS=93
POSTPROCESS_ERROR=94
echo "Hello from test script";
exit $POSTPROCESS_SUCCESS

Tips

Testing NZBGet version

NZBGet version is passed in env-var NZBOP_VERSION. Example values:

  • for stable version: nzbget-11.0;
  • for testing version: nzbget-11.0-testing-r620

Example (python):

NZBGetVersion=os.environ['NZBOP_VERSION']
if NZBGetVersion[0:5] < '11.1':
	print('[ERROR] This script requires NZBGet 11.1 or newer. Please update NZBGet')
	sys.exit(POSTPROCESS_ERROR)

Communication with NZBGet via command line

NZBGet can be controlled via command line using remote commands. For documentation on available commands see Command line reference.

When calling NZBGet you need to know two things:

  • the location of NZBGet binary;
  • the location of NZBGet configuration file.

These both information are available in env-vars NZBOP_APPBIN and NZBOP_CONFIGFILE.

Example: soft-pause download queue:

"$NZBOP_APPBIN" -c "$NZBOP_CONFIGFILE" -P

Communication with NZBGet via RPC-API

With RPC-API more things can be done than using command line. For documentation on available RPC-methods see RPC API reference.

Example: obtaining post-processing log of current nzb-file (this is a short version of script Logger.py supplied with NZBGet):

import os
import sys
import datetime
from xmlrpclib import ServerProxy

# Exit codes used by NZBGet
POSTPROCESS_SUCCESS=93
POSTPROCESS_ERROR=94

# To get the post-processing log we connect to NZBGet via XML-RPC
# and call method "postqueue", which returns the list of post-processing job.
# The first item in the list is current job. This item has a field 'Log',
# containing an array of log-entries.
# For more info visit http://nzbget.sourceforge.net/RPC_API_reference

# First we need to know connection info: host, port, username and password of NZBGet server.
# NZBGet passes all configuration options to post-processing script as
# environment variables.
host = os.environ['NZBOP_CONTROLIP'];
port = os.environ['NZBOP_CONTROLPORT'];
username = os.environ['NZBOP_CONTROLUSERNAME'];
password = os.environ['NZBOP_CONTROLPASSWORD'];

if host == '0.0.0.0': host = '127.0.0.1'

# Build an URL for XML-RPC requests
rpcUrl = 'http://%s:%s@%s:%s/xmlrpc' % (username, password, host, port);

# Create remote server object
server = ServerProxy(rpcUrl)

# Call remote method 'postqueue'. The only parameter tells how many log-entries to return as maximum.
postqueue = server.postqueue(10000)

# Get field 'Log' from the first post-processing job
log = postqueue[0]['Log']

# Now iterate through entries and save them to the output file
if len(log) > 0:
	f = open('%s/_postprocesslog.txt' % os.environ['NZBPP_DIRECTORY'], 'w')
	for entry in log:
		f.write('%s\t%s\t%s\n' % (entry['Kind'], datetime.datetime.fromtimestamp(int(entry['Time'])), entry['Text']))
 	f.close()

sys.exit(POSTPROCESS_SUCCESS)

Script consisting of multiple files

If your script consists or multiple files, put the script and all related files into a subdirectory within ppscripts-directory. NZBGet scans the first-level subdirectory of ppscripts-directory and will find your main script file (the file containing script definition signature). All other files will be ignored. You can also created subdirectories in your script-directory. These subdirectories will not be scanned.

Example:

  ppscripts                                <directory with pp-scripts, option ScriptDir>
  ppscripts\EMail.py                       <E-Mail script supplied with NZBGet>
  ppscripts\MyCoolScript                   <directory for your cool script>
  ppscripts\MyCoolScript\MyCoolScript.py   <your cool script>
  ppscripts\MyCoolScript\Util.py           <module required for your script>
  ppscripts\MyCoolScript\DB.py             <module required for your script>

TIP: if you have many files it's better to put them in a separate subdirectory. In this case the files will not be scanned by NZBGet and the web-interface might load a little bit faster. Example:

  ppscripts                                <directory with pp-scripts, option ScriptDir>
  ppscripts\MyCoolScript                   <directory for your cool script>
  ppscripts\MyCoolScript\MyCoolScript.py   <your cool script>
  ppscripts\MyCoolScript\lib               <directory with modules required for your cool script>
  ppscripts\MyCoolScript\lib\Util.py       <module required for your script>
  ppscripts\MyCoolScript\lib\DB.py         <module required for your script>

Binary script

If your script is a compiled binary executable it obviously can't have a proper script definition and will not be detected by NZBGet as a script. To solve the problem put the executable into a subdirectory and create a starter script (bash, python, etc.) with a proper script definition. The starter script should then call the binary executable.

Windows batch file

Batch files do not treat #-character as comment. But NZBGet requires post-processing script signature. We overcome this using command goto at the start of script to skip the script definition lines:

@echo off
goto start:
##############################################################################
### NZBGET POST-PROCESSING SCRIPT                                          ###

# Brief description goes here.
#
# Lengthier description and instructions go here. Blah, blah blah
# and so on and so forth....

##############################################################################
### OPTIONS																   ###

# An option goes here, do you want that? (yes, no).
#
# The explanation of the option goes here...
#DoYouWantThisOption=no

### NZBGET POST-PROCESSING SCRIPT                                          ###
##############################################################################

:start
echo Hello, world!
echo You chose "Do you want this option?": "%NZBPO_DOYOUWANTTHISOPTION%"
echo Here is a smattering of NZBget server variables:
echo NZBPP_DIRECTORY="%NZBPP_DIRECTORY%"
echo NZBPP_NZBNAME="%NZBPP_NZBNAME%"

rem Return an exit code understood by nzbget
rem # Exit codes used by NZBGet
rem POSTPROCESS_SUCCESS=93
rem POSTPROCESS_NONE=95
rem POSTPROCESS_ERROR=94

exit /b 93
Donation



Get NZBGet at SourceForge.net. Fast, secure and Free Open Source software downloads

Personal tools