Torrent download machine
Torrent package consist of set of shell scripts for torrent processing with lightweight libbt torrent library written entirely in C. All shell scripts are compatible with busybox
/bin/sh with requirement of some convenience tools like
nohup, sed, tr, date
Besides command line tools, there exist also http server CGI script written in SH for WWW access to admin.
- batch multiprocessing (eg. 10 torrents simultaneously with nice priority)
- automatic seeding when torrent finishes (configurable)
- configurable ports
- WWW interface @torrent.cgi@ can be public (no password) as there is no delete provided.
- Accelerator keys (ALT-key) for CGI interface. See help for underlined keys.
- command line utilities @torrent_admin, btcheck-target@ with colorized output
- Per torrent and cumulative status
- start/stop/push and other commands avalable in command line and WWW interface
- lightweight - low memory footprint
- tracker scrape info. Suggest best seed torrent from target dir.
- watchdog for auto restart/queue processing with cron
- syslog watchdog events
- mail notify when torrent finishes
- no state database required. Directory tree is database (slow, but failsafe)
- stable and portable. Deloped on Asus WL-500g Deluxe
For queue torrent file processing there should exist tree directories:
- User should copy
.torrent files here
- Used for torrent processing
- Place where finished torrents are put.
Typical file flow is source -> work -> target
For every torrent file from source, directory with the same name is created. Then torrent file is moved into this directory and program btget is run in background and constantly supervised with
torrent_watchdog. When btget is run in backgroung it automaticly prealocates directories and files specified in torrent file. In case of crash or restart, btget checks this subdir for bad/good blocks.
Minimum config change after first install is setting initial directories which default to
Mail can be sent if MAILTO/FROM is set and proper mail client is installed. Preconfigured mail client is set. Default MAILER is mini_sendmail found in wl500g Oleg firmware, but can be overriden in
Package torrent (1.4-3) is installed on root and has the following files:
- CGI interface for busybox_http, thttpd, ...
- this scirpt should be run regulary to ensure queue processing. Install cron package or use script which periodicly runs this script (ef. every 30 minutes)
- sample script for running server
- command line interface
- configurable options for all utilities
- convnience utility for checksuming finished torrents in case o of svere disk crash/fsck problems
Package libbt (1.05-4) is installed on root and has the following files:
- Listing contents of torrent file
- Lightweight torrent downloading engine (also for seeding)
- Checking torrent consistence for good/bad blocks
This is quick explanation of the options:
- Updates active torrents status
- Quick show active torrents (without update)
- -c n
- Shows current.log of selected torrent
- All active torrent processing should stop/resume imediately
- -p n
- Push selected torrent to other queue
- Lists queued, active, suspended and completed torrents
- -r n
- Mark torrent for purging
- Purge removes all logs from completed torrents and clean removed torrents
- Watchdog forces torrent_watchdog processing
- -i n
- Info shows selected torrent info ((file content and size)
- Update scrape info from tracker for downloaded torrents
- Suggest torrent with best (leeches/seeds) ratio for seeding
- Equals -a -b
For copying .torrent files into source dir one could use samba, ftp, wget and other means of fetching. torrent_watchdog is meant to be run periodicly, but it can be issued at any time from command line.
Finised torrents in target can be always re/moved from target. One can use rm -rf torent-subdir or samba services. WWW interface does not consist such functionality due to inadvent removal from third party on public WWW page. Remove and Purge just do cleanup.
The most usefull command is Push which forces move from one queue to other. With Push one can start processing a torrent, stop active torrent or push from/to seeding state.
Do not push too much. Torrent_watchdog has a code for checking maximum number of active torrents. If all ports (BTGET_MAXPORT-BTGET_MINPORT+1) are occupied, then no new torrent will be enqueued, unless some are pushed away from seed queue. There is no expire time on seeding torrents! One must push it to done manually!
- status info
- status file when watchdog is running
- stdout output of btget run in background
- stderr output of btget run in background
- slow listing is due to directory traversal and .info file processing
- btget fails block checksum when empry directory is specified in torrent. Manual stop and move from work dir is required. Many other clients have the same issue and it is unclear if this is a bug in client or bad torrent. Most notable is eg. empty AUDIO_TS directory sprecified in .torrent file
- Add check for empty directory / patch btget
- Really Quick view
- Run two www servers for CGI interface. Public server should reside on some high port. eg 8008. I suggest busybox_httpd for this job as it lacks autodir features. For intranet I suggest thttpd with symlinks and autodir features on standard 80 port.
- Run torrent_watchdog every 30 minutes with cron. If the time is round, watchdog will update active torrents status. Ther is no need to manualy run update for current status.
- There is no bandwith managment available. QOS should be installed on router with bulk priority on torrent protocol (L7 or port range).
- QUESTION: Any idea how one might force upload/download bandwidth limits without a QOS-enabled router?
oleo July 05, 2006, at 08:57 AM