Torrent auto uploader by Janhouse

Version 2

Work in progress documentation written by Janhouse

Last updated 09.11.2010. 18:56

This document was created using OpenOffice.org :)

Table of Contents
1. About................................................................................................................................................................................... 4
1.1. Differences from version 1..................................................................................................................................... 4
1.1.1. Programming language.................................................................................................................................... 4
1.1.2. Structure................................................................................................................................................................. 4
1.1.3. Program parameters.......................................................................................................................................... 4
2. Setup.................................................................................................................................................................................... 5
2.1. Setup on Windows.................................................................................................................................................... 6
2.1.1. Prepairing for setup............................................................................................................................................ 7
2.1.1.1. Notepad++..................................................................................................................................................... 7
2.1.2. Setting up dependencies.................................................................................................................................. 8
2.1.2.1. Cygwin.............................................................................................................................................................. 8
2.1.2.2. Libsigc++......................................................................................................................................................... 9
2.1.2.3. Libtorrent from SVN..................................................................................................................................... 9
2.1.2.4. Xmlrpc-c........................................................................................................................................................ 10
2.1.2.5. rTorrent from SVN...................................................................................................................................... 10
2.1.2.6. Irssi from SVN............................................................................................................................................... 10
2.1.2.7. Perl modules................................................................................................................................................ 10
2.1.2.8. Apache 2.0.63 with PHP 5.2.14.............................................................................................................. 11
2.1.2.9. mod_scgi for Apache 2............................................................................................................................ 12
2.1.2.10. Browser upload plugin.......................................................................................................................... 12
2.2. Setup on Linux......................................................................................................................................................... 14
2.2.1. Setting up dependencies............................................................................................................................... 14
2.2.1.1. Libtorrent from SVN.................................................................................................................................. 14
2.2.1.2. Xmlrpc-c........................................................................................................................................................ 14
2.2.1.3. rTorrent from SVN...................................................................................................................................... 14
2.2.1.4. Perl modules................................................................................................................................................ 15
2.2.1.5. Browser upload plugin............................................................................................................................. 15
2.3. Manual download browser plugin setup....................................................................................................... 16
3. Usage................................................................................................................................................................................. 17
3.1. tuper.pl (Downloading and uploading).......................................................................................................... 17
3.1.1. Downloading...................................................................................................................................................... 17
3.1.2. Uploading............................................................................................................................................................ 17
3.2. eliminator.pl (Torrent and data removal)....................................................................................................... 19
3.3. Firefox plugin............................................................................................................................................................ 20
4. Technical stuff................................................................................................................................................................. 21
4.1. SQL database structure......................................................................................................................................... 22
4.1.1. Table "data"......................................................................................................................................................... 22
4.2. Random stuff............................................................................................................................................................ 24
4.2.1. Using original torrent NFO file instead of scraped one.......................................................................24
4.2.2. Using NFO if only default description available..................................................................................... 24
4.2.3. IMDB link scraping............................................................................................................................................ 24
4.2.4. RIP category as description........................................................................................................................... 24
4.2.5. Sample vhost...................................................................................................................................................... 25
4.2.6. Adding announce changer to uploading................................................................................................. 25
1. About

1.1. Differences from version 1

1.1.1. Programming language

Biggest difference is programming language used. Version 1 used PHP, version 2 uses Perl.

Programming language was changed to be able to run Uploader on Windows. Since Cygwin
doesn't support PHP but does support Perl, I switched to Perl.

By the way, Irssi uses Perl for its scripts, and since Uploader uses Irssi, this made it even more
preferable.

Since it uses other language, it has different dependencies. Read more about it in setup guide.

1.1.2. Structure
A lot of things were optimized in 2nd version and the code is a lot cleaner now.

All upload and download functions (or "subroutines" as they are called in Perl) are moved to
include/ directory and there is one tracker per file. Upload tracker files also contain category
conversion functions that were previously located inside converters.php.

Irssi scripts and irssi autojoin scripts are moved to starters/ directory.

Maner.php and eater.php is replaced by tuper.pl.

1.1.3. Program parameters

In version 2 it is possible to upload to multiple trackers from the same user.

If you don't pass any extra arguments when using get/aget then torrent is marked for uploading
to the tracker specified in the config file. But you can also pass tracker code to mark it for uploading
to specified tracker.

Same idea works for upload/aupload. If you don't pass tracker code as final argument, it will try to
upload to every tracker the torrent was marked to upload. If you pass tracker code as the final
argument, it will upload only to the specified tracker.

Torrent auto uploader by Janhouse - About 4

2. Setup
It is almost impossible to write guide for every operating system version. The following guide
covers general steps to set up uploader on Linux or Windows.

Torrent auto uploader by Janhouse - Setup 5

2.1. Setup on Windows

Following setup was tested on Windows Server 2008 Enterprise R2

At first, Windows version used rTorrent just like Linux version did, but after some testing it
turned out that it (rTorrent) used to crash often. Now I recommend setting it up using uTorrent.

With uTorrent auto removal feature or anything else related to XMLRPC won't work for the
moment. Theoretically it is possible to inplement auto removal feature in uTorrent using uTorrent
Web UI, but since I don't use uploader on Windows myself, it could take some time before I make
it.

Torrent auto uploader by Janhouse - Setup 6

2.1.1. Prepairing for setup

2.1.1.1. Notepad++
Most of the uploader scripts use Linux line endings meaning that there could be problems
reading them in Notepad or other too primitive Windows text editor. I recommend installing and
using Noepad++:
http://download.tuxfamily.org/notepadplus/5.8.2/npp.5.8.2.Installer.exe

Torrent auto uploader by Janhouse - Setup 7

2.1.2. Setting up dependencies

2.1.2.1. Cygwin
To run Uploader on Windows system you have to install Cygwin. You can get it from
http://www.cygwin.com/

Cygwin is Linux-like environment for Windows making it possible to port software running on
POSIX systems (such as Linux, BSD, and Unix systems) to Windows.

In the Cygwin setup you have to mark these extra packages for installation:

Possibly some of the following packages are not needed if you plan using uTorrent but it would
not hurt if you install them anyway.

✔ gcc

✔ gcc-g++

✔ gcc4

✔ gcc4-g++

✔ libgcc1

✔ libcurl-devel

✔ libncurses-devel

✔ libncursesw-devel

✔ ncurses

✔ ncursesw

✔ libtool

✔ make

✔ libopenssl

✔ openssl-devel

✔ patch

Torrent auto uploader by Janhouse - Setup 8

 ✔ pkg-config

✔ subversion

✔ wget

✔ cppunit

✔ lynx

✔ gettext

✔ gettext-devel

✔ glib2.0

✔ glib2.0-devel

✔ perl

Start cygwin and create source directory

mkdir src
cd src/

2.1.2.2. Libsigc++
(skip this step if using uTorrent instead of rTorrent)
wget http://ftp.gnome.org/pub/GNOME/sources/libsigc++/2.2/libsigc++-2.2.8.tar.gz
tar zxvf libsigc++-2.2.8.tar.gz
cd libsigc++-2.2.8.tar.gz
./configure
make
make install
cd ../

2.1.2.3. Libtorrent from SVN

(skip this step if using uTorrent instead of rTorrent)
svn co svn://rakshasa.no/libtorrent/trunk/libtorrent libtorrent-svn
cd libtorrent-svn
./autogen.sh
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
./configure --disable-mincore
make
make install
cd ../

Torrent auto uploader by Janhouse - Setup 9

2.1.2.4. Xmlrpc-c
(skip this step if using uTorrent instead of rTorrent)
svn co https://xmlrpc-c.svn.sourceforge.net/svnroot/xmlrpc-c/stable/ xmlrpc-c-svn
cd xmlrpc-c-svn
./configure --disable-cplusplus
make
make install
cd ../

2.1.2.5. rTorrent from SVN

(skip this step if using uTorrent instead of rTorrent)
svn co svn://rakshasa.no/libtorrent/trunk/rtorrent rtorrent-svn
cd rtorrent-svn
wget http://rtwi.jmk.hu/downloads/misc/rtow.diff
patch -p0 < rtow.diff
./autogen.sh
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
Add these lines to configure.ac and save it.
PKG_CHECK_MODULES(ncursesw, ncursesw >= 5.7,
CXXFLAGS="$CXXFLAGS $ncursesw_CFLAGS";
LIBS="$LIBS $ncursesw_LIBS")

./configure --with-xmlrpc-c
make
make install
cd ../

2.1.2.6. Irssi from SVN

svn co http://svn.irssi.org/repos/irssi/trunk/ irssi-svn
cd irssi-svn
./autogen.sh --with-perl-staticlib
make
make install
cd ../

2.1.2.7. Perl modules

perl -MCPAN -e 'install Convert::Bencode'
perl -MCPAN -e 'install HTML::Strip'
perl -MCPAN -e 'install URI::Escape'
perl -MCPAN -e 'install DBI'

Torrent auto uploader by Janhouse - Setup 10

 perl -MCPAN -e 'install DBD::SQLite'
perl -MCPAN -e 'install Digest::SHA1'
perl -MCPAN -e 'install XML::RPC'
perl -MCPAN -e 'install Net::BitTorrent::File'
perl -MCPAN -e 'install XML::Simple'
perl -MCPAN -e 'install Date::Parse'
Version 4.13 of Curl for Perl was broken so I recommend using 4.12 or checking if new working
version came out.
wget http://search.cpan.org/CPAN/authors/id/S/SZ/SZBALINT/WWW-Curl-4.12.tar.gz
tar zxvf WWW-Curl-4.12.tar.gz
cd WWW-Curl-1.12
perl Makefile.PL
make
make install
cd ../

I tested installation of these modules from CPAN on Windows Server 2008 and had no
problems. If you run into problems, go to CPAN, look up failing package and install it manually.

2.1.2.8. Apache 2.0.63 with PHP 5.2.14

To use manual upload plugin or rTorrent Web UI with SCGI you will have to set up Apache with PHP.

If you only want to use manual upload plugin, any webserver with PHP support will do just fine
because all that upload_plugin.php script does is executing tuper.pl and passing arguments to it.

Download and install Apache 2.0.63:

http://archive.apache.org/dist/httpd/binaries/win32/apache_2.0.63-win32-x86-
no_ssl.msi

I recommend installing it to “C:\Apache”.

Download and install PHP 5.2.14:

http://fr2.php.net/get/php-5.2.14-Win32.zip/from/fr.php.net/mirror

Extract files from ZIP archive to “C:\Apache\php”

Rename PHP confg file “C:\Apache\php\php.ini-recommended” to “C:\Apache\php\php.ini”.

Modify Apache configuration file “C:\Apache\Apache2\conf\httpd.conf” by adding

Torrent auto uploader by Janhouse - Setup 11

 LoadModule php5_module "c:/php/php5apache2.dll"
AddType application/x-httpd-php .php
PHPIniDir "c:/php"

to the end of the file. Then save it and restart Apache service.

2.1.2.9. mod_scgi for Apache 2

(skip this step if using uTorrent instead of rTorrent)

If you want to use WebUI or SCGI on Windows, you should set up SCGI module for Apache.

On Windows there is no SCGI module that comes with Apache 2, so you have to use 3rd party
module. I found one that worked for me with Apache 2.0.63.

Download SCGI module from:

http://www.zedshaw.com/downloads/scgi_rails/apache2-mod_scgi-1.7a-win32.zip

Copy mod_scgi.so to Apache's modules/ directory and add this to Apache configuration
(httpd.conf).
LoadModule scgi_module modules/mod_scgi.so
SCGIMount /rt1 127.0.0.1:5000
SCGIMount /rt2 127.0.0.1:5001

This will mount SCGI from port 5000 to /rt1 and 5001 to /rt2.

Save httpd.conf and restart Apache service.

2.1.2.10. Browser upload plugin

Create file upload_plugin.bat in Cygwin base directory (C:\cygwin\) with:
@echo off
C:
chdir C:\cygwin\bin
bash --login -i -c '/home/Username/tuper/tuper.pl %1 %2 %3 %4'

Save and close it.

Copy upload_plugin.php and default.upload_plugin_config.php to Apache document root

“C:\Apache\Apache2\htdocs”.

Torrent auto uploader by Janhouse - Setup 12

 Rename default.upload_plugin_config.php to upload_plugin_config.php and edit it. Change
variable $tuper_com to point to upload_plugin.bat (make sure that “get” parameter is there).
$tuper_com="C:\cygwin\upload_plugin.bat get";

Save and close the file.

Read about setting up Firefox plugin in section 2.3 on page 16 and using it in section 3.3 on
page 20.

Torrent auto uploader by Janhouse - Setup 13

2.2. Setup on Linux

2.2.1. Setting up dependencies

As with the guide for version 1, I will give information how to set it up on Debian based Linux
distribution (Debian/Ubuntu/others). If you know your way around the terminal you should have
no problems setting it up on any distribution.

I set it up on Ubuntu server 10.04.Setting up dependencies

Create directory for sources and use it to keep everything in order.

mkdir src
cd src/
Install packages required to buid shit
sudo apt-get install subversion build-essential libtool automake openssl libsigc++-
2.0-dev libncurses5-dev libncursesw5-dev libcurl4-openssl-dev libcppunit-dev irssi rar
mktorrent

2.2.1.1. Libtorrent from SVN

svn co svn://rakshasa.no/libtorrent/trunk/libtorrent libtorrent-svn
cd libtorrent-svn
./autogen.sh
./configure
make
sudo make install
cd ../

2.2.1.2. Xmlrpc-c
svn co https://xmlrpc-c.svn.sourceforge.net/svnroot/xmlrpc-c/stable/ xmlrpc-c-svn
cd xmlrpc-c-svn
./configure --disable-cplusplus
make
sudo make install
cd ../

2.2.1.3. rTorrent from SVN

svn co svn://rakshasa.no/libtorrent/trunk/rtorrent rtorrent-svn
cd rtorrent-svn
./autogen.sh

Torrent auto uploader by Janhouse - Setup 14

 ./configure --with-xmlrpc-c
make
sudo make install
echo "include /usr/local/lib" | sudo tee -a /etc/ld.so.conf
sudo ldconfig
cd ../

2.2.1.4. Perl modules

sudo perl -MCPAN -e 'install Convert::Bencode'
sudo perl -MCPAN -e 'install HTML::Strip'
sudo perl -MCPAN -e 'install URI::Escape'
sudo perl -MCPAN -e 'install DBI'
sudo perl -MCPAN -e 'install DBD::SQLite'
sudo perl -MCPAN -e 'install Digest::SHA1'
sudo perl -MCPAN -e 'install XML::RPC'
sudo perl -MCPAN -e 'install Net::BitTorrent::File'
sudo perl -MCPAN -e 'install XML::Simple'
sudo perl -MCPAN -e 'install Date::Parse'
Version 4.13 of Curl for Perl was broken so I recommend using 4.12 or checking if new working
version came out.
wget http://search.cpan.org/CPAN/authors/id/S/SZ/SZBALINT/WWW-Curl-4.12.tar.gz
tar zxvf WWW-Curl-4.12.tar.gz
cd WWW-Curl-1.12
perl Makefile.PL
make
sudo make install
cd ../

2.2.1.5. Browser upload plugin

Copy upload_plugin.php and default.upload_plugin_config.php to web server document
root “/home/tup/www/”.

Rename default.upload_plugin_config.php to upload_plugin_config.php and edit it. Change

variable $tuper_com to point to tuper.pl (make sure that “get” parameter is there).
$tuper_com="/home/tup/tuper/tuper.pl get";

Save and close the file.

Read about setting up Firefox plugin in section 2.3 on page 16 and using it in section 3.3 on
page 20.

Torrent auto uploader by Janhouse - Setup 15

2.3. Manual download browser plugin setup

This section explains client side setup. Server side setup of this plugin for Windows is described
in section 2.1.2.10 on page 12 and for Linux in section 2.2.1.5 on page 15.

Install uploader_firefox_plugin.xpi from SVN /trunk/browser_plugin/ by opening it with Firefox

Browser.

After installation restart Firefox.

Go to Tools > Add-ons. In Extensions tab find “Torrent auto uploader by Janhouse”.

Click on it and press “Preferences” button.

Preferences window should open up with 3 input fields: “link to uploader script”, “upload tracker
codes” and “authorization key”.

In “Link to uploader script” enter URL to upload_plugin.php. Usually:

http://127.0.0.1/upload_plugin.php

In “Upload tracker codes” write your tracker code(s) used in uploader configuration. If you want to
list more then one tracker, separate them with commas. For example:
fano,inp,mega

In “Authorization key” field write password you set up in upload_plugin_config.php in

$access_key variable. Default is:
password

Close the window and restart Firefox again.

Torrent auto uploader by Janhouse - Setup 16

3. Usage

3.1. tuper.pl (Downloading and uploading)

All the magic is done using tuper.pl. Depending on parameters passed to it, it will either
download or upload.

If you ned to run script in the background as a different proces and exit it immediately, you can
use tex.pl. tex.pl will execute tuper.pl and pass parameters passed to it further.

To escape from other program freezing, you will want to call tex.pl instead of tuper.pl when
calling it from rTorrent/Irssi/other.

3.1.1. Downloading
To start downloading torrent, pass "get" or "aget" as parameter to tuper.pl followed by tracker
code, torrent id on specified tracker and if needed, tracker code for the tracker you want to upload
to. If no upload tracker code is specified after torrent id, Uploader will mark that torrent has to be
uploaded to the tracker specified in configuration.

For example, to upload torrent with id 123456 from SceneAccess to your default configured
tracker, you would use:
./tuper.pl get scc 123456
To upload the same torrent to other tracker from upload trackers list, for example "Fano bittorrent
tracker", you would use:
./tuper.pl get scc 123456 fano

If it is meant to be executed automatically, instead of using "get", use "aget".

For example, executing download automatically from Irssi script:

system "/PATH/TO/tuper/tex.pl aget scc $id";

3.1.2. Uploading
To upload torrent that has been downloaded, use tuper.pl with "upload" or "aupload" parameter

Torrent auto uploader by Janhouse - Usage 17

followed by torrent info hash and if needed, tracker code it needs to be uploaded to.

For example, to manually upload torrent, that failed uploading previously, to every tracker it was
marked to upload to, use:
./tuper.pl upload 2773d37d2a4e2b1f3448879d245d10b041588917

If you want to upload it to only one of the specified trackers from upload list, for example "Fano
bittorrent tracker", use:
./tuper.pl upload 2773d37d2a4e2b1f3448879d245d10b041588917 fano

If uploading is meant to be done automatically, use "aupload" parameter instead of "upload".

Executing upload from rTorrent when download is complete (rTorrent configuration):

system.method.set_key =
event.download.finished,notify_me,"execute=/PATH/TO/tuper/tex.pl,aupload,
$d.get_hash="

Torrent auto uploader by Janhouse - Usage 18

3.2. eliminator.pl (Torrent and data removal)
Remover automatically removes torrents when they reach time or ratio goal.

The script is called eliminator.pl

Version 1 used XMLRP-C for removal but 2nd version does not. (At least not by default.)

To remove torrent the right way, you must execute eliminator script and pass torrent info hash to
it. It marks it as removed in the database and removes torrent from rTorrent. Files are removed
when all tied torrents are removed from rTorrent.

Best way to do this is to create rTorrent and Uploader configuration that sorts torrents in groups
and specify seeding goal rules that execute eliminator.pl when seeding goal is reached.

Torrent auto uploader by Janhouse - Usage 19

3.3. Firefox plugin
To use this plugin, go to download tracker configured with uploader.

Go to torrent page (usually browse.php or browse/)

Click on choosen torrent with right mouse button.

Select “Upload to ***” (replace *** with your upload tracker code).

New tab should be opened with upload_plugin.php and status message.

Torrent auto uploader by Janhouse - Usage 20

4. Technical stuff
This section should be sorted some time.

Torrent auto uploader by Janhouse - Technical stuff 21

4.1. SQL database structure
Empty database default.tuper.db can be found in SVN /trunk/tuper/. It is recommended to use it
in case of new setup.

4.1.1. Table "data"

Data table stores all info hashes, times, category id's and names used by Uploader.

✔ id - unique ID for entry (automatically generated when new entry is added)

✔ infohash - info hash of downloaded torrent (read from downloaded torrent file and added
when entry is created)
✔ name - name of the torrent (read upon scraping information from download tracker and
added when entry is created)
✔ file - file name of the torrent, nfo and txt (read upon scraping information from download
tracker and added when entry is created)
✔ category - category ID torrent will be uploaded to (found and added when entry is created)

✔ added - time when entry was added

✔ started - time when torrent started downloading (added when rTorrent sends command
that it has started downloading)
✔ downloaded - time when torrent was uploaded (added when torrent is uploaded)

✔ uploaded - time when torrent finished downloading (added when rTorrent sends
command that it has finished downloading)
✔ seeded - time when torrent started seeding uploaded torrent (added when rTorrent sends
command that it has started seeding)
✔ upload_infohash - info hash of uploaded torrent (added when torrent is uploaded)

✔ download_removed - time when downloaded torrent was removed (added when

download is removed from rTorrent)
✔ upload_removed - time when uploaded torrent was removed (added when upload is
removed from rTorrent)
✔ from - tracker code that torrent was downloaded from (added when entry is created)

Torrent auto uploader by Janhouse - Technical stuff 22

 ✔ to - tracker code that torrent was uploaded to (added when entry is created)

✔ upload_id - torrent ID on the tracker it was uploaded to (scraped from tracker when torrent
is uploaded)

Torrent auto uploader by Janhouse - Technical stuff 23

4.2. Random stuff

4.2.1. Using original torrent NFO file instead of scraped one

Add this line after $nfo definition in your upload function:
$nfo=&find_nfo($torrent, $nfo);

It will search for original .nfo file inside torrent download directory.

4.2.2. Using NFO if only default description available

Paste this in your upload function one line after $descr_txt declaraton or line after last change of
$nfo variable:
my $descr_txt=&read_file($nfo) if $descr_txt eq $no_descr_txt;

4.2.3. IMDB link scraping

my $match_imdb=qr/imdb.com\/title\/tt([0-9]*)/ms;
my $imdb="";
if($$descr_txt =~ $match_imdb){
$imdb="http://www.imdb.com/title/tt".$1."/";
err("Found IMDB link: ".$imdb);
}
$curlf->curl_formadd("t_link", $imdb);

4.2.4. RIP category as description

err(" Checking if category with ID $category should have NFO as
description...");
if($category !~ /^(11|6|5|7|42|9|10)$/){
err(" Not a video category, ripping NFO as
description...");
$curlf->curl_formadd("UseNFOasDescr", "yes");
}else{
err(" One of the video categories, won't ript NFO as
description...");
}

Torrent auto uploader by Janhouse - Technical stuff 24

4.2.5. Sample vhost
<VirtualHost *:80>
ServerName sample.com
DocumentRoot /home/tup/www/
ServerAlias sample.com
<Directory /home/tup/www>
DirectoryIndex index.php index.html
AllowOverride All
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

4.2.6. Adding announce changer to uploading

If the tracker you are uploading to requires manual announce address change then you can add
the following line to the upload subroutine in the include/upload_*.pm, right after $torrent variable
initialization.
$torrent=&changeannounce($torrent, "http://example.com/announce.php",
$temp_dir.$defname.".torrent");

Adding automatic torrent extraction

If you want to upload extracted torrents instead of usually rarred files, you can add the fillowing
line to the upload subroutine in the include/upload_*.pm, right after $torrent variable initialization.
$torrent=&rarextract($torrent, $subname, $defname);

It will search for rarred files, extract them in uploads directory specified by $upload_dir in
config.pm, copy over other files and make new torrent with mktorrent. So make sure that mktorrent
is installed if you want to use this.

Torrent auto uploader by Janhouse - Technical stuff 25