NSLU2-Linux
view · edit · print · history

This howto describes, how to enable traffic-shaping for the NSLU2, so you can limit the rate of outgoing (and even incoming) network traffic. The howto is devided into several steps, for some of them you can either choose the "hard way" (i.e. doing all on your own), or the "easy way" (i.e. trusting my compilation-capabilities). So lets start...

Step 1: The Kernel

Unfortunately the kernels of the recent Unslung-3.x-releases do not support fair queueing, an essential part of the Linux traffic-shaper. So, we'll have to activate some modules in Unslung's kernel and compile it.

1.1 The easy way

Get the Siddy provided flash-image with activated fair-queuing and gunzip it. The MD5 hash of the gunzipped image must be f6dd6036d0037e629b20519a0d17f5dc.

1.2 The hard way

  • Get bitbake + openembedded trees (see corresponding howto)
  • Modify openembedded/packages/linux/unslung-kernel-2.3r25/able/defconfig by appending:
 CONFIG_IP_NF_IPTABLES=m
 CONFIG_IP_NF_MATCH_LIMIT=m
 CONFIG_IP_NF_MATCH_MAC=m
 CONFIG_IP_NF_MATCH_PKTTYPE=m
 CONFIG_IP_NF_MATCH_MARK=m
 CONFIG_IP_NF_MATCH_MULTIPORT=m
 CONFIG_IP_NF_MATCH_TOS=m
 CONFIG_IP_NF_MATCH_RECENT=m
 CONFIG_IP_NF_MATCH_ECN=m
 CONFIG_IP_NF_MATCH_DSCP=m
 CONFIG_IP_NF_MATCH_AH_ESP=m
 CONFIG_IP_NF_MATCH_LENGTH=m
 CONFIG_IP_NF_MATCH_TTL=m
 CONFIG_IP_NF_MATCH_TCPMSS=m
 CONFIG_IP_NF_MATCH_UNCLEAN=m
 CONFIG_IP_NF_MATCH_OWNER=m
 CONFIG_IP_NF_FILTER=m
 CONFIG_IP_NF_TARGET_REJECT=m
 CONFIG_IP_NF_TARGET_MIRROR=m
 CONFIG_IP_NF_MANGLE=m
 CONFIG_IP_NF_TARGET_TOS=m
 CONFIG_IP_NF_TARGET_ECN=m
 CONFIG_IP_NF_TARGET_DSCP=m
 CONFIG_IP_NF_TARGET_MARK=m
 CONFIG_IP_NF_TARGET_LOG=m
 CONFIG_IP_NF_TARGET_ULOG=m
 CONFIG_IP_NF_TARGET_TCPMSS=m

 CONFIG_NET_SCHED=y
 CONFIG_NET_SCH_CBQ=m
 CONFIG_NET_SCH_HTB=m
 CONFIG_NET_SCH_CSZ=m
 CONFIG_NET_SCH_PRIO=m
 CONFIG_NET_SCH_RED=m
 CONFIG_NET_SCH_SFQ=m

 CONFIG_NET_SCH_TEQL=m
 CONFIG_NET_SCH_TBF=m
 CONFIG_NET_SCH_GRED=m
 CONFIG_NET_SCH_DSMARK=m
 CONFIG_NET_SCH_INGRESS=m
 # CONFIG_NET_QOS is not set
 CONFIG_NET_CLS=y
 CONFIG_NET_CLS_TCINDEX=m
 CONFIG_NET_CLS_ROUTE4=m
 CONFIG_NET_CLS_ROUTE=y
 CONFIG_NET_CLS_FW=m
 CONFIG_NET_CLS_U32=m

(fully enables iptables (needed to mark packages which need queueing) and fair queueing)

  • build the unslung-images ("bb unslung-image", again: read howto!)
  • If you have already built an image using that installation, you must first do (in order for the modules to be rebuilt):
    • rm -rf ${OEBUILD}/tmp/work/unslung-*-kernel*
    • rm -rf ${OEBUILD}/tmp/stamps/unslung-*-kernel*

Step 2: Flashing

Flash the (compiled or downloaded) image-file "unslung-able-nslu2-XXXXXXXXXXXX.flashdisk.img" to your slug, following the README of the normal unslung-distribution until you finished to it's very end, having completed all those reboots, unslings etc pp.

Step 3: Installing the modules

The next step will be to copy the compiled kernel-modules to your slug. For this purpose, you have to create a directory named "/opt/lib/modules". If you followed step 1.1, get the modules from http://nslu2.siddy.org/shaper/shaper_modules.tar.gz and untar them in "/opt/lib/modules". If you followed step 1.2, copy the modules out of your build-tree. They can be found in the subdirectories of /your/base/dir/build/tmp/work/unslung-able-kernel-2.3r25-r7/install and you need the modules

 cls_fw.o
 cls_route.o
 cls_tcindex.o
 cls_u32.o
 ip_tables.o
 ipt_DSCP.o
 ipt_ECN.o
 ipt_LOG.o
 ipt_MARK.o
 ipt_MIRROR.o
 ipt_REJECT.o
 ipt_TCPMSS.o
 ipt_TOS.o
 ipt_ULOG.o
 ipt_ah.o
 ipt_dscp.o
 ipt_ecn.o
 ipt_esp.o
 ipt_length.o
 ipt_limit.o
 ipt_mac.o
 ipt_mark.o
 ipt_multiport.o
 ipt_owner.o
 ipt_pkttype.o
 ipt_recent.o
 ipt_tcpmss.o
 ipt_tos.o
 ipt_ttl.o
 ipt_unclean.o
 iptable_filter.o
 iptable_mangle.o
 sch_cbq.o
 sch_csz.o
 sch_dsmark.o
 sch_gred.o
 sch_htb.o
 sch_ingress.o
 sch_prio.o
 sch_red.o
 sch_sfq.o
 sch_tbf.o
 sch_teql.o

(okay, we don't need all of them, but the more the better, right? ;) )

Step 4: Loading the modules

To simplify the process of loading the required modules, let's create a simple bash-script to do the work for us. So, create the file "/opt/sbin/load_shaper_modules", chmod 755 it and paste the following content (removing the leading spaces, as usual...)

 #!/bin/sh

 MODDIR="/opt/lib/modules"
 INSMOD="/sbin/insmod"

 $INSMOD $MODDIR/ip_tables.o
 $INSMOD $MODDIR/iptable_filter.o
 $INSMOD $MODDIR/iptable_mangle.o
 $INSMOD $MODDIR/ipt_REJECT.o
 $INSMOD $MODDIR/ipt_MARK.o
 $INSMOD $MODDIR/ipt_length.o
 $INSMOD $MODDIR/ipt_mark.o
 $INSMOD $MODDIR/sch_htb.o
 $INSMOD $MODDIR/sch_sfq.o
 $INSMOD $MODDIR/cls_fw.o

Execute /opt/sbin/load_shaper_modules and hopefully don't get any error-messages.

Step 5: Installing 'iptables'

To install the iptables userspace-tool, simply do an "ipkg update; ipkg install iptables".

Step 6: Installing 'tc'

In this step, we'll install the program named 'tc' (for "traffic control"). It is part of the standard iproute2-package, but unfortunately I failed to cross-compile iproute2 up to now, so there is no iproute2-ipk for the slug. So, here we go:

Step 6.1: The easy way

Get 'tc' from http://nslu2.siddy.org/shaper/tc.gz and gunzip it to /opt/sbin . Then make it executable by doing chmod +x /opt/sbin/tc.

Step 6.2: The hard way

Get the source of iproute2 from http://developer.osdl.org/dev/iproute2/ and compile tc. It can be compiled using a native compiler (see corresponding howto) or cross-compilation (again...). Cross-compilation will probably fail somewhere at arpd, but that's after tc, so who cares... ;-) So, somehow you will be able to get an arm-executable of tc and copy it to /opt/sbin.

Step 7: Install shaper-script

We're getting close to our goal now... it's time to configure the traffic-shaper! If you are a real sadist, you can do it manually with the help of tc and iptables and some hours of man-page-reading. Or... just copy and paste someone else's work (like I did). There are lots of scripts setting up a traffic-shaper out there (like wondershaper and others), so either you take the script of your choice or take the one of my choice and copy it to /opt/sbin/shaper :

 #!/bin/sh
 #
 # myshaper - DSL/Cable modem outbound traffic shaper and prioritizer.
 #            Based on the ADSL/Cable wondershaper (www.lartc.org)
 #
 # Written by Dan Singletary (8/7/02)
 # Edited by Siddy
 #
 # NOTE!! - This script assumes your kernel has been patched with the
 #          appropriate HTB queue and IMQ patches available here:
 #          (subnote: future kernels may not require patching)
 #
 #       http://luxik.cdi.cz/~devik/qos/htb/
 #       http://luxik.cdi.cz/~patrick/imq/
 #
 # Configuration options for myshaper:
 #  DEV    - set to ethX that connects to DSL/Cable Modem
 #  RATEUP - set this to slightly lower than your
 #           outbound bandwidth on the DSL/Cable Modem.
 #           I have a 1500/128 DSL line and setting
 #           RATEUP=90 works well for my 128kbps upstream.
 #           However, your mileage may vary.
 #

 DEV=ixp0
 RATEUP=100
 MINCLASSRATE=14 # should be RATEUP/7

 TC="/opt/sbin/tc"
 IPTABLES="/opt/sbin/iptables"


 #
 # End Configuration Options
 #

 if [ "$1" = "status" ]
 then
         echo "[qdisc]"
         $TC -s qdisc show dev $DEV
         echo "[class]"
         $TC -s class show dev $DEV
         echo "[filter]"
         $TC -s filter show dev $DEV
         echo "[iptables]"
         $IPTABLES -t mangle -L MYSHAPER-OUT -v -x 2> /dev/null
         exit
 fi

 # Reset everything to a known state (cleared)
 $TC qdisc del dev $DEV root    2> /dev/null > /dev/null
 $IPTABLES -t mangle -D POSTROUTING -o $DEV \
    -j MYSHAPER-OUT 2> /dev/null > /dev/null
 $IPTABLES -t mangle -F MYSHAPER-OUT 2> /dev/null > /dev/null
 $IPTABLES -t mangle -X MYSHAPER-OUT 2> /dev/null > /dev/null

 if [ "$1" = "stop" ]
 then
         echo "Shaping removed on $DEV."
         exit
 fi


 echo "Starting outbound shaping..."

 ###########################################################
 #
 # Outbound Shaping (limits total bandwidth to RATEUP)

 # add HTB root qdisc
 $TC qdisc add dev $DEV root handle 1: htb default 26

 # add main rate limit class
 $TC class add dev $DEV parent 1: classid 1:1 htb rate ${RATEUP}kbit
 # class for local traffic
 $TC class add dev $DEV parent 1: classid 1:2 htb rate 100mbit

 # add leaf classes
 #
 # We grant each class at LEAST it's "fair share"
 # of bandwidth. this way no class will ever be
 # starved by another class.  Each class is also
 # permitted to consume all of the available bandwidth
 # if no other classes are in use.

 ## create fair-share-classes, descending priority
 $TC class add dev $DEV parent 1:1 classid 1:20 htb \
   rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 0
 $TC class add dev $DEV parent 1:1 classid 1:21 htb \
   rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 1
 $TC class add dev $DEV parent 1:1 classid 1:22 htb \
   rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 2
 $TC class add dev $DEV parent 1:1 classid 1:23 htb \
   rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 3
 $TC class add dev $DEV parent 1:1 classid 1:24 htb \
   rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 4
 $TC class add dev $DEV parent 1:1 classid 1:25 htb \
  rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 5
 $TC class add dev $DEV parent 1:1 classid 1:26 htb \
  rate ${MINCLASSRATE}kbit ceil ${RATEUP}kbit prio 6

 # attach qdisc to leaf classes
 # here we at SFQ to each priority class.  SFQ insures
 # that within each class connections will be treated
 # (almost) fairly.
 $TC qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
 $TC qdisc add dev $DEV parent 1:21 handle 21: sfq perturb 10
 $TC qdisc add dev $DEV parent 1:22 handle 22: sfq perturb 10
 $TC qdisc add dev $DEV parent 1:23 handle 23: sfq perturb 10
 $TC qdisc add dev $DEV parent 1:24 handle 24: sfq perturb 10
 $TC qdisc add dev $DEV parent 1:25 handle 25: sfq perturb 10
 $TC qdisc add dev $DEV parent 1:26 handle 26: sfq perturb 10

 # filter traffic into classes by fwmark
 #
 # here we direct traffic into priority class according
 # to the fwmark set on the packet (we set fwmark with
 # iptables later).  Note that above we've set the
 # default priority class to 1:26 so unmarked packets
 # (or packets marked with unfamiliar IDs) will be
 # defaulted to the lowest priority class.
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 20 fw flowid 1:20
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 21 fw flowid 1:21
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 22 fw flowid 1:22
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 23 fw flowid 1:23
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 24 fw flowid 1:24
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 25 fw flowid 1:25
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 26 fw flowid 1:26
 $TC filter add dev $DEV parent 1:0 prio 0 \
   protocol ip handle 2 fw flowid 1:2

 # add MYSHAPER-OUT chain to the mangle table in iptables
 #
 # this sets up the table we'll use to filter and mark packets.
 $IPTABLES -t mangle -N MYSHAPER-OUT
 $IPTABLES -t mangle -I POSTROUTING -o $DEV -j MYSHAPER-OUT

 # add fwmark entries to classify different types of traffic
 #
 # Set fwmark from 20-26 according to desired class. 20 is highest prio.

 # Default for low port traffic
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --sport 0:1024 -j MARK --set-mark 23

 # Default for low port traffic
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --dport 0:1024 -j MARK --set-mark 23

 # ftp-data port, low prio
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --dport 20 -j MARK --set-mark 26

 # ICMP (ping) - high prio, impress friends
 $IPTABLES -t mangle -A MYSHAPER-OUT -p icmp \
   -j MARK --set-mark 20

 # DNS name resolution (small packets)
 $IPTABLES -t mangle -A MYSHAPER-OUT -p udp \
   -j MARK --set-mark 21

 # secure shell
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --dport ssh -j MARK --set-mark 21

 # secure shell
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --sport ssh -j MARK --set-mark 21

 # telnet (ew...)
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --dport telnet -j MARK --set-mark 21

 # telnet (ew...)
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --sport telnet -j MARK --set-mark 21

 # Local web server
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --sport http -j MARK --set-mark 22

 # https
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   --sport 10000 -j MARK --set-mark 22

 # small packets (probably just ACKs)
 $IPTABLES -t mangle -A MYSHAPER-OUT -p tcp \
   -m length --length :64 -j MARK --set-mark 2

 # hyper-speed for local traffic
 $IPTABLES -t mangle -A MYSHAPER-OUT -p all \
   --dst 192.168.1.1/24 -j MARK --set-mark 2 

 # redundant- mark any unmarked packets as 26 (low prio)
 $IPTABLES -t mangle -A MYSHAPER-OUT -m mark \
   --mark 0 -j MARK --set-mark 26

 # Done with outbound shaping
 #
 ####################################################

 echo "Outbound shaping added to $DEV.  Rate: ${RATEUP}Kbit/sec."

As usual, remove the leading spaces and take care of unwanted line-breaks... or simply get the script from http://nslu2.siddy.org/shaper/shaper.gz . Remember to make the script executable. You probably want to edit some variables in the script, especially "RATEUP" for the overall maximum upload-limit, and "MINCLASSRATE" for the minimal per-class rate. You might also want to change the iptables-mangle-rules to match your setup.

Now, execute "/opt/sbin/shaper start" and be happy if you do not see any error-messages! :-D

Step 8: Create init.d-script

You probably want to start the traffic-shaper each time you boot your slug. For this purpose, simply create a script named "/opt/etc/init.d/S05shaper" (chmodded to 755) and paste something like

 #!/bin/sh
 /opt/sbin/load_shaper_modules
 /opt/sbin/shaper start

into it.

Well, that's all! Your traffic-shaper should now be up and running! Easy going, wasn't it?!

If you do have any problems or suggestions, found an error or wanna flame me, just contact me via email/irc/whatever.

Oh... not to forget: I am not responsible if you break your slug by following this howto! This howto is just a documentation of what _I_ did to have traffic-shaping for my slug (with some shortcuts for you lazy guys). If it doesn't work for you (and I do not know why it shouldn't), it's not my fault!

Siddy (sidddy _at_ gmail.com)

view · edit · print · history · Last edited by tman.
Based on work by DrS, tman, bobtm, and Siddy.
Originally by Siddy.
Page last modified on July 22, 2005, at 02:01 PM