Cross Compilation 1

Cross? I was bloody livid.

This blogging stuff is difficult to keep up with isn't it? One of the things I've been working on and not posting about is trying to carry out cross compilation for my Western digital myBook. So I'd like to be able to, for example, download the sourcecode for lightppd, compile it on a faster processer and then install on the MyBook. Now it may be possible to just download the code and try

make CROSS-COMPILE=arm

or something, but it seems the way to do things is to download buildroot and create an entire rootfs which matches that on the MyBook. Confused? I know I am.

I've attempted this twice already and achieved nothing. So I'm trying again and documenting my progress this time. This time I have identified the GPL code provided by Western Digital (as they seem to be obliged to do under the GPL) This includes the copy of buildroot they used to create the MyBook disk image, the linux header files they used and all the packages they used, including their busybox setup. They also include some of the necesary config files, but not all, and I think this is where the 'magic' happens and where I'm likely to continue cursing and swearing for the next few months.

So, progress so far.

1. Buy a new laptop. This isn't strictly necessary but I found a bargain. Acer Aspire 5630 with 2GB RAM, Core 2 Duo T5500 for £299.
2. Install VMWare server, free from vmware.
3. Download ubuntu server 8.04
4. With ubuntu running, ensure we have the latest versions of gcc and binutils and such.
5. Download the Western Digital myBook GPL Source Code

I've untarred the files and am now following instruction from this chap but may need create my own config files (for buildroot, the linux kernel and busybox). I'm up to make menuconfig and it's time to take a break...

You Say To-ma-to

I run a hacked Western Digital MyBook World Edition. It runs SSHD and with port forwarding of port 22 enabled on my router, I am able to login and access the 500Gb of storage from work.

Unfortunately this has meant opening port 22 on the router to the outside world and I was finding logs like this on the mybook recently

May 9 15:53:04 anya auth.info /usr/sbin/sshd[15612]: Failed password for root from 220.227.218.21 port 39837 ssh2
May 9 15:53:07 anya auth.info /usr/sbin/sshd[15614]: Invalid user fluffy from 220.227.218.21
May 9 15:53:07 anya auth.err /usr/sbin/sshd[15614]: error: Could not get shadow information for NOUSER
May 9 15:53:07 anya auth.info /usr/sbin/sshd[15614]: Failed password for invalid user fluffy from 220.227.218.21 port 40167 ssh2
May 9 15:53:09 anya auth.info /usr/sbin/sshd[15616]: Invalid user admin from 220.227.218.21
May 9 15:53:09 anya auth.err /usr/sbin/sshd[15616]: error: Could not get shadow information for NOUSER
May 9 15:53:09 anya auth.info /usr/sbin/sshd[15616]: Failed password for invalid user admin from 220.227.218.21 port 40408 ssh2

(If anyone wants to go visit 220.227.218.21, be my guest)

This is typical of a brute force, automated scripting attack on sshd; repeated attempts to login using a dictionary of usernames and passwords. The trick to stopping it is to recognise multiple incoming packets from the same source. A solution is described here:

Diary of a Geek

Unfortunately the solution requires iptables, the linux firewall mechanism, which needs to be compiled into the kernel. The mybook needs to be small and lightweight so does not have iptables compiled in.

However my router does. It runs Tomato which lets you do all kinds of nice linux/network related things. It's linux on a router, including iptables. Setup is via a very slick GUI, but the linux command line is there when you need it.

At first I had problems using the recent match as described in Andrew Pollock's solution as it didn't seem to be available in the Tomato setup. However a quick browse round the installation found the module was available, just not loaded. I then found that everything I needed to do (loading modules, adding new iptables rules) could be done from the Administration->Scripts section of the Tomato GUI

Figure 1. Init commands

Commands added to the Administration->Scripts->Init section are run when the router is booted up. I added

modprobe ipt_recent

to load the recent match module. Surprisingly easy. Rebooted and the previously missing module was now loaded. (use lsmod to see which modules are loaded)

Figure 2. Firewall commands

Little more complicated. I think I understand the basics of iptables now, but like quantum mechanics, anyone who claims they understand it all is lying. Here's what I added:

iptables -N wanrecent

iptables -A wanrecent -p tcp -d anya --dport 22 -m state --state NEW -m recent --name SSH_LIMIT --rsource --set
iptables -A wanrecent -p tcp -d anya --dport 22 -m state --state NEW -m recent --name SSH_LIMIT --rsource --update --seconds 60 --hitcount 4 -j LOG --log-prefix "SSH LIMIT"
iptables -A wanrecent -p tcp -d anya --dport 22 -m state --state NEW -m recent --name SSH_LIMIT --rsource --update --seconds 60 --hitcount 4 -j DROP

iptables -I wanin 1 -j wanrecent

Line 1 creates a new user table called wanrecent. this just means I can keep all this custom stuff in a single place.
Line 2 initialises a match which which keeps track of incoming NEW packets on port 22 (ssh) in a list called SSH_LIMIT.
Line 3... let's skip that as it does exactly the same as line 4 but creates a log entry in syslog instead of dropping incoming packets.
Line 4 checks if a 4th NEW packet destined for port 22 from the same source arrives, and if so simply DROPs the packet. That's the trick which is going to stop our scrip-kiddies from just firing request after request after request.
Line 5 just adds the new custom table to the existing wanin table. The wanin table contains the port forwarding rules so by inserting wan recent at position 1 it is guaranteed to execute before the forwarding. Ta-dah!

The biggest difference between what's actually happening here and in Andrew's suggestion is that wanin is part of the FORWARD table in my setup as opposed to the INPUT table. This is because the INPUT table is used when packets are destined for "this" host, the machine where iptables is running. In my case, due to port forwarding, the packets are routed through the FORWARD table.

Reboot the router it all just worked! I am hpapy.

TestCase HeadCase

I'm writing a reeeeeally simple rails application, but want to support User security; passwords, login, authentication and authorisation, that sort of thing so am using the SaltedHashLoginGenerator detailed here.

I think I need to tweak it. I don't like email addresses as usernames for example but those are minor. My big problem is Rails functional tests.

I used the Rails scaffold generator to set up a framework for my own model/view/controller and it very kindly generated a complete set of model/unit and controller/functional tests for me. Unfortunately my functional tests all failed because every method required a valid user to log in first, thanks to Salty and the use of before_filters.

The problem is described here. Lots of people trying to use the setup method of the TestCase to either login or assign a user object to the session. Unfortunately setup seems to be broken as of Rails 2.0.2. Ticket #10568 has it fixed, but I'm not sure it hasn't broken something else, and I don't yet know how to apply a changeset without waiting for 2.0.3 anyway.

Here's how I've fixed it for me. I'll just demo using one method. The original test looked like

def test_should_create_contact
assert_difference('Contact.count') do
post :create, :contact => { :firstname => "tester" }
end

assert_redirected_to contact_path(assigns(:contact))
end

I found out that the post (along with get, out, delete and head) can take an optional 3rd parameter, a hash of key/value pairs which populate the test's session object. Since SaltedHashLoginGenerator's user_controller simply checks for a key called 'user' defining a User model object we can write

def test_should_create_contact
assert_difference('Contact.count') do
post :create, { :contact => { :firstname => "tester" } }, { 'user' => User.find(1000001) }
end

assert_redirected_to contact_path(assigns(:contact))
end

This ensures that the session object access by the controller has 'user' key with a valid User object, which keeps the user_controller happy.

Note, at first I tried to alter the line, so:

post :create, :contact => { :firstname => "tester" }, 'user' => User.find(1000001)

just adding on the last parameter after a comma. This variously ended up with an empty session and/or syntax errors as I messed about with strings, symbols and braces. In the line shown, the session turned out to be empty because it though 'user' was the second entry in the params hash, which is the second argument of the post method, i.e. the one hash contained two keys, :contact and 'user'. You wouldn't believe how long that took to track down!

Put it all together and whaddaya got!?

After faffing about a bit trying to get everything to boot up on startup, it all ended up being simpler than I expected.

First, create your configuration file for wpa_supplicant. Because I started off using the network-tool, this is /etc/wpa_supplicant/wpa_psk.conf for me.

gordon> more /etc/wpa_supplicant/wpa_psk.conf

network={
ssid="Sunnydale"
scan_ssid=1
key_mgmt=WPA-PSK
psk="xxxxxxxxxxxxx"
}

Simple and I think typical for a basic home network. Sunnydale is the domain defined on my router and xxxxxxxxxxxxx is my 13 character ascii passphrase as setup on the router. Replace these with your own values.

The next pleasant surprise, after contemplating writing an init.d script to start wpa_supplicant was that the network interface file supports wpa options! The file /etc/network/interface is read by the ifup command when the system boots, telling it which interfaces to bring up. It originally only had an entry for the loop back adapter.

gordon> more /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The Netgear wg121 USB Adapter loaded via ndiswrapper
auto wlan0
iface wlan0 inet static
address 192.168.11.12
netmask 255.255.255.0
gateway 192.168.11.1
# I don't know why the next line isn't ndiswrapper but
# wpa_supplicant reports that it doesn't support WPA.
wpa-driver wext
wpa-conf /etc/wpa_supplicant/wpa_psk.conf

# The loopback network interface
auto lo
iface lo inet loopback

reboot and I'm posting this from the laptop wirelessly! Yay!

Tearing My Hair Out (ha ha)

Right, the last post details how to get the adapter working assuming you have a working ndiswrapper.

My girlfriend bought herself a new laptop a couple of weeks ago, thereby freeing up my very old Toshiba Satellite 1800 laptop which I hadn't managed to touch since she moved in. Since I'm not actually running a Linux desktop at the moment I thought this would be ideal to try stuff out on. Unfortunately Ubuntu Gutsy Gibbon (7.10) wasn't having any of it and the installer kept freezing with a corrupted display. The Ubuntu Alternate CD was an option but I found a couple of references to fluxbuntu, an Ubuntu based distro using the fluxbox window manager. I'd used fluxbox before so thought it was worth a try. The installation failed once when the laptop switch itself off (almost as if it timed out and powersave kicked in) but worked perfectly on a second attempt if I pressed the spacebar every now and again. (Who knows?)

Once installed the first thing is to get on-line and get any patches and whatnot. While there are some new drivers (madwifi) around since I last tried this, they don't support the Broadcom chipset in my Netgear wg121 USB wireless adapter.

So the accepted solution is to use ndiswrapper, a Linux implementation of Windows' Network Driver Interface Specification (NDIS) APIs. This will load Windows drivers for your wireless adapter and allow Linux to use them. The ndiswrapper project consists of two parts,

1. The ndiswrapper module. This is the runtime component of the project and loads the API wrappers into your Linux system so that they're available to the Kernel. (I think)
2. The ndiswrapper utils, which is a command line tool to help you configure stuff and load your windows drivers.

fluxbuntu doesn't come with the ndiswrapper utils, or not that I could find anyway. So I had no way to load my Netgear driver. So off to http://ndiswrapper.sourceforge.net/joomla/ to download the latest source (1.52). Of course the laptop can't connect yet so I have to use the PC, download the zip, copy it to a USB stick, move the stick across, manually mount it because hotplug isn't working either, then unzip it. Phew, right.

Now! This next bit below (in red) is WHAT I DID WRONG! This is for background and what to check if you're having similar problems. Don't do this!

It looked like the ndiswrapper util came pre-built so I just cd'd into ndiswrapper-1.52/driver and build the module

gordon> cd ~/ndiswrapper-1.52/driver
gordon> make

All worked fine. This generated ndiswrapper.ko which should be copied to /lib/modules//ubuntu/misc/ndiswrapper, taking a backup of the existing one just in case.

gordon> sudo cp ndiswrapper.ko /lib/module/2.6.22-14-generic/ubuntu/misc/ndiswrapper

And then, followed the steps as described in the last post. the only difference (ha!) was that ndiswrapper wasn't on the su path so I had to run it manually from ~/ndiswrapper-1.52/utils/ndiswrapper every time.

Everything seemed to work, except where the rest of the world saw their wlan0 interface enabled, I had nothing. modprobe reported ndiswrapper module had been loaded correctly. ndiswrapper -l reported the driver and device were both present. It wasn't until I checked my ndiswrapper version that I realised what was wrong

gordon> ndiswrapper -v
module version is too old!

I eventually realised that I should have compiled the ndiswrapper utils after all, particularly the loaddriver object file, There was a Makefile in the top level of the ndiswrapper-1.52 download which I'd missed.

gordon> cd ~/ndiswrapper-1.52
gordon> sudo make uninstall
gordon> make

BANG! Screens and screens of errors. it seems that Fluxbuntu comes with the gcc compiler, binutils and even the linx kernel header, but not the libc6-dev package which includes all the useful files like stdlib.h.

Well shit.

If I was on the internet then it would be easy via apt-get, but of course what am I trying to achieve today!?

Back to the PC and manually download the libc headers from http://packages.ubuntu.com. I needed

• libc6-dev
• libc6
• linux-libc-dev

These are .deb debian package files. Copy to USB stick, yadda yadda yadda, and install using dpkg.

gordon> sudo dpkg -i linux-libc-dev
gordon> sudo dpkg -i libc6
gordon> sudo dpkg libc6-dev

Aaaaand back to ~/ndiswrapper-1.52 to rebuild

gordon> cd ~/ndiswrapper-1.52
gordon> make
gordon> sudo make install

And it looks like we finally have a result! the step in the last post now work.

Sorry that was so long but it's covered a number of things including ndiswrapper gotchas, installing debian packages manually and discovering what's missing from a fluxbuntu basic install. don't get me wrong, fluxbuntu is in early stages and can' guess what setup and circumstances are going to affect everyone using their distro. Apart from these teething troubles it looks very nice otherwise. Maybe ensure a full set of build packages are n the installation CD would be nice though.

I Have The Power!

A few days absent due to

1. having a life at weekends
2. f%*$ing fluxbuntu, ndiswrapper, modprobe et al

In the process I (re)discovered a few things so I'll break them up into a couple of posts. First getting a USB wireless adapter up and running on fluxbuntu, a lightweight ubuntu variant, on an old Toshiba Statellite 1800 (circa 1999). These steps assume you have a working version of ndiswrapper (more later...) Ndiswrapper is a Linux implementation of certain windows APIS. Suffice to say it allows you to load windows drivers in Linux. It's particularly handy for loading wireless adapter drivers which rarely have native linux drivers available.

First of all, find the Windows drivers for your adapter. In my case this is a Netgear wg121 USB adapter with V2 of its firmware installed. I already had the latest files from Netgear; WG121V200.zip. Plug in the USB stick and mount it under a directory under /media. Unzipping the file created a directory under my home area.

gordon> cd ~/WG121V200/ndis5

In this directory is the inf file describing the driver, netwg121.inf. fluxbuntu is set up to do everything using sudo rather than logging in as root. We are going to use ndiswrapper to load the Netgear Windows driver.

First unload the ndiswrapper module just in case it's already doing something else.

gordon> sudo modprobe -r ndiswrapper

Check there are no other drivers loaded by ndiswrapper.

gordon> sudo ndiswrapper -l

If anything is returned use

gordon> sudo ndiswrapper -r <driver name>

to unload anything loaded. (Technically ndiswrapper might have other Windows drivers loaded. You'll just have to use good judgement to decide whether one of those is for wireless adapter that you're not actually using and might cause a conflict with what we're trying to do)

Now load the netgear driver (or your own .inf file)

gordon> sudo ndiswrapper -i netwg121.inf
installing netwg121 ...

Excellent, check we've loaded the driver

gordon> sudo ndiswrapper -l
netwg121 : driver installed
device (0846:4210) present (alternate driver: p54usb)

Get ndiswrapper to write a modprobe conf file. this will ensure that the driver is loaded as wlan0 every time modprobe loads the ndiswrapper module.

gordon> sudo ndiswrapper -m

Now use modprobe to load the ndiswrapper module into memory. Module are like dynamic libraries which can be loaded on demand to give your Linux installation extra capabilities.

gordon> sudo modprobe -i ndiswrapper

And DING! Power and signal LEDs on the adapter are both on!

I'm not all the way there yet but this is a significant jump forward over the sweat and tears of Tuesday and complete stalemate of Wednesday. I still have to set up wpa_supplicant to handle my WPA encryption and make sure everything is reloaded automatically on rebooting. I'll cover the problems I had getting to here in the next post.

(There's weren't really tears. If Linux ever really makes me cry then it's time to get a new hobby)

smb.conf

Phew!

Startup script /etc/init.d/S27-shares-cleanup was calling the removeExternalShares.pl perl script. It should probably be trying to be clever and only removing shares that it finds under /shares/external that aren't listed under /proc/mounts. It's sufficient for now just to comment out the removeExternalShares line.