NSLU2-Linux
view · edit · print · history

Optware.Torrent History

Hide minor edits - Show changes to markup

March 29, 2008, at 07:51 AM by ralph --
Changed lines 118-119 from:
  • There is no bandwith managment available. QOS should be installed on router with bulk priority on torrent protocol (L7 or port range). QUESTION: Any idea how one might force upload/download bandwidth limits without a QOS-enabled router?
to:
  • There is no bandwith managment available. QOS should be installed on router with bulk priority on torrent protocol (L7 or port range).
    • QUESTION: Any idea how one might force upload/download bandwidth limits without a QOS-enabled router?
March 29, 2008, at 07:51 AM by ralph -- Bandwidth limit question
Changed lines 118-119 from:
  • There is no bandwith managment available. QOS should be installed on router with bulk priority on torrent protocol (L7 or port range).
to:
  • There is no bandwith managment available. QOS should be installed on router with bulk priority on torrent protocol (L7 or port range). QUESTION: Any idea how one might force upload/download bandwidth limits without a QOS-enabled router?
July 25, 2006, at 06:43 AM by oleo -- Add seeding practice
Changed lines 92-93 from:
to:

Do not push too much. Torrent_watchdog has a code for checking maximum number of active torrents. If all ports (BTGET_MAXPORT-BTGET_MINPORT+1) are occupied, then no new torrent will be enqueued, unless some are pushed away from seed queue. There is no expire time on seeding torrents! One must push it to done manually!

July 19, 2006, at 08:53 AM by oleo --
Changed lines 116-117 from:
to:
  • There is no bandwith managment available. QOS should be installed on router with bulk priority on torrent protocol (L7 or port range).
July 05, 2006, at 09:14 AM by oleo -- syslog feature
Added line 20:
  • syslog watchdog events
July 05, 2006, at 09:03 AM by oleo --
Changed line 9 from:
  • multiprocessing (eg. 10 torrents simultaneously)
to:
  • batch multiprocessing (eg. 10 torrents simultaneously with nice priority)
Changed lines 114-115 from:
to:
  • Run torrent_watchdog every 30 minutes with cron. If the time is round, watchdog will update active torrents status. Ther is no need to manualy run update for current status.
July 05, 2006, at 08:57 AM by oleo -- Inital description which should be done a long time ago.
Changed lines 1-2 from:

Info on the torrent scripts.

to:

Torrent download machine

Changed lines 4-7 from:
  • torrent_daemon
  • torrent_get
  • torrent_admin
to:

Torrent package consist of set of shell scripts for torrent processing with lightweight libbt torrent library written entirely in C. All shell scripts are compatible with busybox /bin/sh with requirement of some convenience tools like nohup, sed, tr, date Besides command line tools, there exist also http server CGI script written in SH for WWW access to admin.

Features

  • multiprocessing (eg. 10 torrents simultaneously)
  • automatic seeding when torrent finishes (configurable)
  • configurable ports
  • WWW interface @torrent.cgi@ can be public (no password) as there is no delete provided.
  • Accelerator keys (ALT-key) for CGI interface. See help for underlined keys.
  • command line utilities @torrent_admin, btcheck-target@ with colorized output
  • Per torrent and cumulative status
  • start/stop/push and other commands avalable in command line and WWW interface
  • lightweight - low memory footprint
  • tracker scrape info. Suggest best seed torrent from target dir.
  • watchdog for auto restart/queue processing with cron
  • mail notify when torrent finishes
  • no state database required. Directory tree is database (slow, but failsafe)
  • stable and portable. Deloped on Asus WL-500g Deluxe

Workflow

For queue torrent file processing there should exist tree directories:

source
User should copy .torrent files here
work
Used for torrent processing
target
Place where finished torrents are put.

Typical file flow is source -> work -> target For every torrent file from source, directory with the same name is created. Then torrent file is moved into this directory and program btget is run in background and constantly supervised with torrent_watchdog. When btget is run in backgroung it automaticly prealocates directories and files specified in torrent file. In case of crash or restart, btget checks this subdir for bad/good blocks.

Config

/opt/etc/torrent.conf

Minimum config change after first install is setting initial directories which default to

 SOURCE=/tmp/harddisk/torrent/source
 WORK=/tmp/harddisk/torrent/work
 TARGET=/tmp/harddisk/torrent/target

Mail can be sent if MAILTO/FROM is set and proper mail client is installed. Preconfigured mail client is set. Default MAILER is mini_sendmail found in wl500g Oleg firmware, but can be overriden in /opt/etc/torrent.conf.

Programs

Package torrent (1.4-3) is installed on root and has the following files:

/opt/share/www/cgi-bin/torrent.cgi
CGI interface for busybox_http, thttpd, ...
/opt/sbin/torrent_watchdog
this scirpt should be run regulary to ensure queue processing. Install cron package or use script which periodicly runs this script (ef. every 30 minutes)
/opt/etc/init.d/S80busybox_httpd
sample script for running server
/opt/sbin/torrent_admin
command line interface
/opt/etc/torrent.conf
configurable options for all utilities
/opt/bin/btcheck-target
convnience utility for checksuming finished torrents in case o of svere disk crash/fsck problems

Package libbt (1.05-4) is installed on root and has the following files:

/opt/bin/btlist
Listing contents of torrent file
/opt/bin/btget
Lightweight torrent downloading engine (also for seeding)
/opt/bin/btcheck
Checking torrent consistence for good/bad blocks

Torent admin

This is quick explanation of the options:

-u
Updates active torrents status
-q
Quick show active torrents (without update)
-c n
Shows current.log of selected torrent
-s
All active torrent processing should stop/resume imediately
-p n
Push selected torrent to other queue
-l
Lists queued, active, suspended and completed torrents
-r n
Mark torrent for purging
-g
Purge removes all logs from completed torrents and clean removed torrents
-w
Watchdog forces torrent_watchdog processing
-i n
Info shows selected torrent info ((file content and size)
-a
Update scrape info from tracker for downloaded torrents
-b
Suggest torrent with best (leeches/seeds) ratio for seeding
-ab
Equals -a -b

Usage

For copying .torrent files into source dir one could use samba, ftp, wget and other means of fetching. torrent_watchdog is meant to be run periodicly, but it can be issued at any time from command line.

Finised torrents in target can be always re/moved from target. One can use rm -rf torent-subdir or samba services. WWW interface does not consist such functionality due to inadvent removal from third party on public WWW page. Remove and Purge just do cleanup.

The most usefull command is Push which forces move from one queue to other. With Push one can start processing a torrent, stop active torrent or push from/to seeding state.

Changed lines 95-101 from:
to:

Files:

.info
status info
work/.watchdog
status file when watchdog is running
current.log
stdout output of btget run in background
current_error.log
stderr output of btget run in background
Changed lines 103-104 from:
  • ??
to:
  • slow listing is due to directory traversal and .info file processing
  • btget fails block checksum when empry directory is specified in torrent. Manual stop and move from work dir is required. Many other clients have the same issue and it is unclear if this is a bug in client or bad torrent. Most notable is eg. empty AUDIO_TS directory sprecified in .torrent file
Changed lines 107-115 from:
  • ??
to:
  • Add check for empty directory / patch btget
  • Really Quick view

Best practice:

  • Run two www servers for CGI interface. Public server should reside on some high port. eg 8008. I suggest busybox_httpd for this job as it lacks autodir features. For intranet I suggest thttpd with symlinks and autodir features on standard 80 port.

oleo July 05, 2006, at 08:57 AM

January 10, 2005, at 08:58 PM by perlguru --
Changed lines 1-16 from:

Describe {{Torrent}} here.

to:

Info on the torrent scripts.

Intro

  • torrent_daemon
  • torrent_get
  • torrent_admin

Issues:

  • ??

Feature requests:

  • ??
view · edit · print · history · Last edited by ralph.
Based on work by ralph and oleo.
Originally by perlguru.
Page last modified on March 29, 2008, at 07:51 AM