uShare Revisited

Quick update, My MyBook died before christmas so was sent away for a replacement. It took aaaaages for the replacement to come back and I've been using an older tower with Ubuntu in the meantime.

I'm finally getting round to transferring everything back to the new MyBook and for some reason, ushare isn't working, despite following my own instructions here. (The very thing this blog was for).

Anyway, not sure what all the problems were, but I've had to alter the ushare.sh script in /etc/init.d in the following ways:

USHARE_OPTIONS has been altered to be "x -f /etc/ushare.conf". Without the -f option, the config file is meant to be loaded as a set of environment variables which ushare sees and uses. This wasn't working, so the -f option forces the config file to be loaded by ushare.

I also had to change the way the check_shares function was called. Changed from $(check_shares) to simply check_shares, the CONFIG_FILE is now read (although that's slightly redundant since ushare stil didn't use the variables set). The original method just wasn't loading the values.

so my ushare.sh now looks like so:

[gordon]$ more /etc/ushare.conf
# /etc/ushare.conf
# Configuration file for uShare

# uShare UPnP Friendly Name (default is 'uShare').
USHARE_NAME="Anya uShare Media Server"

# Interface to listen to (default is eth0).
# Ex : USHARE_IFACE=eth1
USHARE_IFACE=eth0

# Port to listen to (default is random from IANA Dynamic Ports range)
# Ex : USHARE_PORT=49200
USHARE_PORT=

# Port to listen for Telnet connections
# Ex : USHARE_TELNET_PORT=1337
USHARE_TELNET_PORT=

# Directories to be shared (space or CSV list).
# Ex: USHARE_DIR=/dir1,/dir2
USHARE_DIR=/shares/internal/VIDEO

# Use to override what happens when iconv fails to parse a file name.
# The default uShare behaviour is to not add the entry in the media list
# This option overrides that behaviour and adds the non-iconv'ed string into
# the media list, with the assumption that the renderer will be able to
# handle it. Devices like Noxon 2 have no problem with strings being passed
# as is. (Umlauts for all!)
#
# Options are TRUE/YES/1 for override and anything else for default behaviour
USHARE_OVERRIDE_ICONV_ERR=

# Enable Web interface (yes/no)
ENABLE_WEB=

# Enable Telnet control interface (yes/no)
ENABLE_TELNET=no

# Use XboX 360 compatibility mode (yes/no)
ENABLE_XBOX=yes

# Use DLNA profile (yes/no)
# This is needed for PlayStation3 to work (among other devices)
ENABLE_DLNA=no

See that comment about my MyBook dying, coming soon: "How to do backups!"

Cross Compilation 04

It's been a while. My MyBook died, Western Digital took an age to get a replacement back, and when they did it has the new 2.x firmware which doesn't include a native gcc on the box. So I either have to install optware or... get a working cross compiler going. I've had some major insights into cross compilation, specifically regarding the difference between the BUILD, HOST and TARGET variables. I've also stepped back from buildroot for a moment to focus on going through the steps from the ground up. I'm now following this tutorial from IBM whih pretty much covers it all.

I've run into a problem at the moment where I'm trying to build uClibc instead of glibc, as covered in the guide. I have the error

ubuntu > arm-linux-uclibc-gcc -Wall ... -mlittle-endian ... -c arm/resolve.S -o arm/resolve.o
as: unrecognized option: -EL

It looks like I'm using the native (ubuntu i386) assembler rather than the cross compiler one. I think this is because at this stage I've only built a minimal gcc suite and this doesn't include a separate assembler (if any at all).

For tomorrow, I need to complete the tutorial, building glibc, and then using that to build a compelte gcc suite. Hopefully with that I can then build my uClibc library.

I'll then have two C libraries available andjust need to make sure the correct one is used when cross compilin for the MyBook.

MyBook uShare

I have movies stored on the MyBook streaming to the XBox and it works better than I could have imagined!

I was expecting stuttering (as happens when viewing films on my wireless PC connected to the telly (I suspect the USB wireless adapter on it just isn't up to the task) ) and it streams almost instantly, unlike on my laptop which seems to download the whole file before starting to play it.

Here's how.

Install ushare on the MyBook world edition. This depends upon the upnp library so get, build and install it first, then same for ushare. If anything run under sudo doesn't work then you can alternatively log in as root and run the command as root.

[gordon]$ mkdir usharebuild
[gordon]$ cd usharebuild
[gordon]$ wget http://dfn.dl.sourceforge.net/sourceforge/pupnp/libupnp-1.6.6.tar.bz2
[gordon]$ bzcat libupnp-1.6.6.tar.bz2 | tar xv
[gordon]$ cd libupnp-1.6.6
[gordon]$ ./configure
[gordon]$ make
[gordon]$ sudo make install

This will install linupnp in /usr/local/lib so we need to check this is in our libpath. Ensure the /etc/ld.so.conf file contains an entry for /usr/local/lib and run ldconfig to ensure it's added to the path.

[gordon]$ more /etc/ld.so.conf
/usr/local/lib
[gordon]$ sudo ldconfig

Now get, build and install ushare. NLS is disabled otherwise there is a dependency error on some library I don't have, possibly iconv? DLNA is disabled as it's not used for the XBox. If you have a PS3 you will need this and the dependent libraries.

[gordon]$ cd ..
[gordon]$ wget http://ushare.geexbox.org/releases/ushare-1.1a.tar.bz2
[gordon]$ bzcat ushare-1.1a.tar.bz2 | tar xv
[gordon]$ cd ushare-1.1a
[gordon]$ ./configure --disable-dlna --disable-nls
[gordon]$ make
[gordon]$ sudo make install

Simple! Technically that's it, ushare is installed and ready to run. Here's my config file after editing from the default location, /usr/local/etc/ushare.conf. Obviously you need to change your USHARE_DIR setting.

[gordon]$ more /usr/local/etc/ushare.conf
# /etc/ushare.conf
# Configuration file for uShare

# uShare UPnP Friendly Name (default is 'uShare').
USHARE_NAME="Anya uShare Media Server"

# Interface to listen to (default is eth0).
# Ex : USHARE_IFACE=eth1
USHARE_IFACE=eth0

# Port to listen to (default is random from IANA Dynamic Ports range)
# Ex : USHARE_PORT=49200
USHARE_PORT=

# Port to listen for Telnet connections
# Ex : USHARE_TELNET_PORT=1337
USHARE_TELNET_PORT=

# Directories to be shared (space or CSV list).
# Ex: USHARE_DIR=/dir1,/dir2
USHARE_DIR=/shares/internal/VIDEO

# Use to override what happens when iconv fails to parse a file name.
# The default uShare behaviour is to not add the entry in the media list
# This option overrides that behaviour and adds the non-iconv'ed string into
# the media list, with the assumption that the renderer will be able to
# handle it. Devices like Noxon 2 have no problem with strings being passed
# as is. (Umlauts for all!)
#
# Options are TRUE/YES/1 for override and anything else for default behaviour
USHARE_OVERRIDE_ICONV_ERR=

# Enable Web interface (yes/no)
ENABLE_WEB=

# Enable Telnet control interface (yes/no)
ENABLE_TELNET=no

# Use XboX 360 compatibility mode (yes/no)
ENABLE_XBOX=yes

# Use DLNA profile (yes/no)
# This is needed for PlayStation3 to work (among other devices)
ENABLE_DLNA=no

And here's my /etc/init.d/ushare.sh file used for starting and stopping the process. Call /etc/init.d/ushare.sh start to kick things off.

[gordon]$ more /etc/init.d/ushare.sh
#!/bin/sh -e
### BEGIN INIT INFO
# Required-Start: $local_fs $syslog
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop ushare
# Description:
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/ushare
NAME=ushare
DESC="uShare UPnP A/V Media Server"
PIDFILE=/var/run/ushare.pid
# Use default in /usr/local/etc/ushare.conf
CONFIGFILE=/usr/local/etc/ushare.conf
USHARE_OPTIONS=-x

# abort if no executable exists
[ -x $DAEMON ] || exit 0

# Get lsb functions
#. /lib/lsb/init-functions
#. /etc/default/rcS

[ -f /etc/default/ushare ] && . /etc/default/ushare

checkpid() {
[ -e $PIDFILE ] || touch $PIDFILE
}

check_shares() {
if [ -r "$CONFIGFILE" ]; then
. $CONFIGFILE
[ -n "$USHARE_DIR" ] && return 0
fi
return 1
}

case "$1" in
start)
echo -n "Starting $DESC: $NAME"
if ! $(check_shares); then
echo -n "No shares available ..."
# echo -n 0
else
checkpid
start-stop-daemon --start --quiet --background --oknodo \
--make-pidfile --pidfile $PIDFILE \
--exec $DAEMON -- $USHARE_OPTIONS
fi
;;
stop)
echo -n "Stopping $DESC: $NAME"
start-stop-daemon --stop --signal 2 --quiet --oknodo --pidfile $PIDFILE
;;
reload|force-reload)
echo -n "Reloading $DESC: $NAME"
start-stop-daemon --stop --signal 1 --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
;;
restart)
$0 stop
$0 start
;;
*)
N=/etc/init.d/$NAME
echo -n "Usage: $N {start|stop|restart|reload|force-reload}"
exit 1
;;
esac

exit 0

Finally, if you want ushare to start automatically on bootup, and die cleanly on shutdown then add the following softlinks in your /etc/init.d directory.

[root]$ cd /etc/init.d
[root]$ ln -s ushare.sh S36ushare
[root]$ ln -s ushare.sh K64ushare
[root]$ ls -al *ushare*
lrwxrwxrwx 1 root root 9 Aug 29 17:05 K36ushare -> ushare.sh
lrwxrwxrwx 1 root root 9 Aug 29 17:05 S36ushare -> ushare.sh
-rwxr-xr-x 1 root root 1615 Aug 29 17:00 ushare.sh

And the final thing I had to do was add a "broadcast" route to the mybook's routing path. without this, the XBox could browse the files, but not play any of them. I admit I've only added this on the command line for the moment so it my need re-added if the mybook is ever rebooted.

[root]$ route add -net 239.0.0.0 netmask 255.0.0.0 eth0

That's it! It worked. Now browsing files on the XBox prompted me to connect to XBox Live to download a media update (containing the DivX codecs an such) and following that I was watching MOVIES I RIPPED FROM MY OWN DVDS on my lovely 1080p LCD TV. Any upscaling is unnoticable (to me).

I guess the next step is to, um, "find" some hi-def ripped films and try those to check the wireless bandwidth and the MyBook disk controller are up to the task.

Cross Compilation 03

I was right. It was too good to be true. I mounted the .ext2 file created by buildroot.

> sudo mkdir /mnt/mybook

> sudo mount -t ext2 -o loop rootfs-arm_nofpu.ext2 /mnt/mybook

and was able to browse around. It has all the config files and scripts under /etc (woop) but the /bin directory is completely empty (meh).

So it looks like, at the very least, I haven't built the kernel or busybox.

On the plus side, the build_arm_nofpu/staging/dir/bin did seem to contain a set of tools prefixed with "arm-linux-" so I wrote a very simple Hello World c file and compiled it using arm-linux-gcc. (I added the staging_dir/bin directory to the PATH).

> arm-linux-gcc -o helloarm helloworld.c

Then scp'd the resulting executable onto the mybook. Then from the mybook, I was able to run helloarm!

So technically I've carried out my first cross-compilation. But this was the most basic file possible and I guess I could have achieved it using gcc directly on the command line just as easily (although I've linked to uClibc I think. Which is nice.)

Try again tomorrow to configure the kernel and busybox; cross-compile those, hopefully all from within the buildroot run.

Cross Compilation 2

Ha! I almost typed cross copulation there! That would have been funny?

Right this was too easy. After a few trip ups

1. Needed to install flex, bison and patch (via apt-get of course)
2. ubuntu links /bin/sh to the dash shell which was causing a syntax error: bad fd number(?) so I changed the softlink to use bash instead of dash.
3. Copied all the directories under buildroot-patches/packages and buildroot-patches/patches into buildroot/package
4. I was getting the following error message: "arm-linux-uclibc-ld: ERROR:/home/gordon/extract/buildroot/build_arn_nofpu/lib/gcc/arm-linux-uclibc/3.4.2/libgcc.a(_dvmd_lnx.o)uses hardware FP, whereas ld-uClibc-0.9.28.so uses software FP" so altered one of the FPU settings under make menuconfig to not use a hardware FPU. (It looked like my original config file from last night had been overwritten as some other settings were lost). A new call to make seemed to revuild everything.

and it worked!? I have a rootfs.arm_nofpu.ext2 file of type "Linux rev 0.0 ext2 filesystem data". Of course it can't really have worked. Can it? It was entirely too fast, I'm convinced my kernel config is still not done and... well it worked. I never got anywhere near this point on the last two attempts and was expecting this to take weeks!

Anyway, finding out how to mount this image as a filesystem is something I need to research so this seems as good a palce to stop for the night as any. I'm sure it's simple but it's a new step to start tomorrow. Oooh, the anticipation!

I bet it's empty.

Cross Compilation 1

Cross? I was bloody livid.

This blogging stuff is difficult to keep up with isn't it? One of the things I've been working on and not posting about is trying to carry out cross compilation for my Western digital myBook. So I'd like to be able to, for example, download the sourcecode for lightppd, compile it on a faster processer and then install on the MyBook. Now it may be possible to just download the code and try

make CROSS-COMPILE=arm

or something, but it seems the way to do things is to download buildroot and create an entire rootfs which matches that on the MyBook. Confused? I know I am.

I've attempted this twice already and achieved nothing. So I'm trying again and documenting my progress this time. This time I have identified the GPL code provided by Western Digital (as they seem to be obliged to do under the GPL) This includes the copy of buildroot they used to create the MyBook disk image, the linux header files they used and all the packages they used, including their busybox setup. They also include some of the necesary config files, but not all, and I think this is where the 'magic' happens and where I'm likely to continue cursing and swearing for the next few months.

So, progress so far.

1. Buy a new laptop. This isn't strictly necessary but I found a bargain. Acer Aspire 5630 with 2GB RAM, Core 2 Duo T5500 for £299.
2. Install VMWare server, free from vmware.
3. Download ubuntu server 8.04
4. With ubuntu running, ensure we have the latest versions of gcc and binutils and such.
5. Download the Western Digital myBook GPL Source Code

I've untarred the files and am now following instruction from this chap but may need create my own config files (for buildroot, the linux kernel and busybox). I'm up to make menuconfig and it's time to take a break...

You Say To-ma-to

I run a hacked Western Digital MyBook World Edition. It runs SSHD and with port forwarding of port 22 enabled on my router, I am able to login and access the 500Gb of storage from work.

Unfortunately this has meant opening port 22 on the router to the outside world and I was finding logs like this on the mybook recently

May 9 15:53:04 anya auth.info /usr/sbin/sshd[15612]: Failed password for root from 220.227.218.21 port 39837 ssh2
May 9 15:53:07 anya auth.info /usr/sbin/sshd[15614]: Invalid user fluffy from 220.227.218.21
May 9 15:53:07 anya auth.err /usr/sbin/sshd[15614]: error: Could not get shadow information for NOUSER
May 9 15:53:07 anya auth.info /usr/sbin/sshd[15614]: Failed password for invalid user fluffy from 220.227.218.21 port 40167 ssh2
May 9 15:53:09 anya auth.info /usr/sbin/sshd[15616]: Invalid user admin from 220.227.218.21
May 9 15:53:09 anya auth.err /usr/sbin/sshd[15616]: error: Could not get shadow information for NOUSER
May 9 15:53:09 anya auth.info /usr/sbin/sshd[15616]: Failed password for invalid user admin from 220.227.218.21 port 40408 ssh2

(If anyone wants to go visit 220.227.218.21, be my guest)

This is typical of a brute force, automated scripting attack on sshd; repeated attempts to login using a dictionary of usernames and passwords. The trick to stopping it is to recognise multiple incoming packets from the same source. A solution is described here:

Diary of a Geek

Unfortunately the solution requires iptables, the linux firewall mechanism, which needs to be compiled into the kernel. The mybook needs to be small and lightweight so does not have iptables compiled in.

However my router does. It runs Tomato which lets you do all kinds of nice linux/network related things. It's linux on a router, including iptables. Setup is via a very slick GUI, but the linux command line is there when you need it.

At first I had problems using the recent match as described in Andrew Pollock's solution as it didn't seem to be available in the Tomato setup. However a quick browse round the installation found the module was available, just not loaded. I then found that everything I needed to do (loading modules, adding new iptables rules) could be done from the Administration->Scripts section of the Tomato GUI

Figure 1. Init commands

Commands added to the Administration->Scripts->Init section are run when the router is booted up. I added

modprobe ipt_recent

to load the recent match module. Surprisingly easy. Rebooted and the previously missing module was now loaded. (use lsmod to see which modules are loaded)

Figure 2. Firewall commands

Little more complicated. I think I understand the basics of iptables now, but like quantum mechanics, anyone who claims they understand it all is lying. Here's what I added:

iptables -N wanrecent

iptables -A wanrecent -p tcp -d anya --dport 22 -m state --state NEW -m recent --name SSH_LIMIT --rsource --set
iptables -A wanrecent -p tcp -d anya --dport 22 -m state --state NEW -m recent --name SSH_LIMIT --rsource --update --seconds 60 --hitcount 4 -j LOG --log-prefix "SSH LIMIT"
iptables -A wanrecent -p tcp -d anya --dport 22 -m state --state NEW -m recent --name SSH_LIMIT --rsource --update --seconds 60 --hitcount 4 -j DROP

iptables -I wanin 1 -j wanrecent

Line 1 creates a new user table called wanrecent. this just means I can keep all this custom stuff in a single place.
Line 2 initialises a match which which keeps track of incoming NEW packets on port 22 (ssh) in a list called SSH_LIMIT.
Line 3... let's skip that as it does exactly the same as line 4 but creates a log entry in syslog instead of dropping incoming packets.
Line 4 checks if a 4th NEW packet destined for port 22 from the same source arrives, and if so simply DROPs the packet. That's the trick which is going to stop our scrip-kiddies from just firing request after request after request.
Line 5 just adds the new custom table to the existing wanin table. The wanin table contains the port forwarding rules so by inserting wan recent at position 1 it is guaranteed to execute before the forwarding. Ta-dah!

The biggest difference between what's actually happening here and in Andrew's suggestion is that wanin is part of the FORWARD table in my setup as opposed to the INPUT table. This is because the INPUT table is used when packets are destined for "this" host, the machine where iptables is running. In my case, due to port forwarding, the packets are routed through the FORWARD table.

Reboot the router it all just worked! I am hpapy.