I’m happy to announce the release of Spek 0.6 – a multi-platform acoustic spectrum analyser.
This version is about 3 times faster than 0.5 thanks to the lightning-fast FFmpeg decoders and the new multi-threaded analysis algorithm.
Spek 0.6 also features dramatically reduced size of the Windows installer (from 17.1 MiB to 9.8 MiB) and OS X bundle (from 10.5 MiB to 6.1 MiB)
Read the NEWS file for a complete change log.
Next version of Spek will use FFmpeg libraries to decode audio files. There are several reasons for the switch from GStreamer:
GStreamer is a fantastic framework for building complex multimedia pipelines, however what Spek really needs is a simple decoder and FFmpeg’s libavformat and libavcodec do just that.
To handle some audio formats (e.g. APE and DTS), GStreamer relies on FFmpeg anyway, so the switch will result in lesser dependencies. It doesn’t matter too much on GNU/Linux, but this will reduce the size of the Windows and Mac OS X installers.
Spek used GStreamer’s spectrum plugin to perform the actual spectral analysis, with FFmpeg I had to implement it myself. The code I ended up with is very compact and gives room for a lot of experimentation, from using different window functions (it’s still Hamming) and working on performance optimisations to switching to a faster FFT library.
The last bit is actually done, Spek now uses FFTW which in my tests is 1.5x to 2x faster than Kiss FFT used by GStreamer. Apart from that, FFTW can scale to multiple threads with near linear performance increase, future versions of Spek will take advantage of this.
UPDATE: As one of commenters pointed out, FFTs on small number of samples are not very parallelisable and my benchmarks confirm this. Also, I switched from FFTW to avfft which is built into FFmpeg. It’s a little bit faster than FFTW for my particular use case. Lastly, 1.5x to 2x speed up was actually caused by a faster decoder, not by a faster FFT library.
Another thing that would be hard with GStreamer is static cross-compilation using mingw-cross-env to produce a single Windows executable. Because FFmpeg doesn’t employ a plugin architecture, static linking is not an issue.
Last, but not least, the whole experience was very educational. I now remember why I loved C back in the days, the trick is to not even attempt to write any GObject code with it, that’s what Vala is for. Fast Fourier Transform and the maths behind it is much fun, and NR was very helpful here.
For the curious, the code is merged to git master and pushed to Gitorious. The next version of Spek will be released sometime in July.
I’m happy to announce the release of version 0.5 of Spek, a multi-platform acoustic spectrum analyser.
Changes since the previous release:
This is the first version of Spek that features a Mac OS X port. All binaries in the app bundle are compiled completely from scratch using a slightly modified version of Aaron‘s excellent bockbuild project (the same tool that is used to package Banshee for OS X).
If your project uses the GNOME stack and you want to port it to Mac OS X — give bockbuild a try. Check Spek’s bundle script for gory details.
I’ve just released version 0.4 of Spek — an acoustic spectrum analyser for GNU/Linux and Windows.
This version brings the following features and and improvements:
Grab Spek 0.4 while it’s hot, the source code tarball and the Windows installer can be downloaded here.
A special “thank you” to users of a popular private tracker, their great suggestions helped to shape this release.
P.S. Hello, Planet GNOME! :)
Spek 0.3 is out, just 2 days after the previous release. This version includes the following features and fixes:
--help and --version (in addition to the usual gtk+ and gst options.)Spek 0.3 source code tarball and the Windows installer can be downloaded here.
Spek now has a fully functional Windows port! The installer bundles GTK+, GStreamer and all their dependencies, you don’t need to install anything else.
I also moved project hosting over to Google Code so Spek now has a full-fledged bug tracker. The source code is still hosted on Gitorious.
If you are tired of starting up monstrous applications like Adobe Audition or Audacity just to check a file’s spectrogram, give Spek a shot!
I just released version 0.1 of Spek — a little program that shows spectrograms of audio files. Spek is written in Vala and uses the standard GNOME stack: GLib, GTK+, Cairo and GStreamer.
Spectrograms are used to analyse the quality of audio files, you can easily detect lossy re-encodes, web-rips and other badness by just looking at the spectrogram.
This version of Spek doesn’t do much apart from showing the actual spectrogram and allowing to save it as a PNG image. However, I plan to add a bunch of features before releasing version 1.0:
You can download the tarball from the project’s website. To build and run:
$ tar -xjvf spek-0.1.tar.bz2 $ cd spek-0.1 $ ./configure $ make $ src/spek
Or `sudo make install` to have it installed.
The code is available on Gitorious. I really need and will appreciate help in these areas:
I initially wanted to write Spek in Haskell to practice the language after reading The Haskell School of Expression (generously sent to me by Jorge). However, after writing a bit of code I realised that all the functional goodness of Haskell is not used at all because Spek is simply a bit of glue between GTK+ and GStreamer, with very little code of its own. Also, the prototype’s executable size of 14+ MiB didn’t help much in convincing myself that Haskell was a good pick for this project.
I was left with C and C#. The latter didn’t feel like a good idea for the same reasons: I wanted something lightweight for this small little app, C# would require a lot of dependencies. Also, while I really like C# as a language and Mono/.NET as a platform, I wanted to try something new for a change — I already use them full-time on my day job and when hacking Banshee.
I wrote very little C/C++ code since late 90′s, now I know why I didn’t miss it much — it’s so incredibly verbose! After a few hours I gave up, decided to do some research and found Vala :)
The impression so far is hugely positive. Vala still has a few rough edges but nothing too bad and not work-aroundable. If you are tired of using C for your GTK+ applications, definitely give it a try!
After playing with FreeNAS I ended up using Debian for my server. FreeNAS is a great distribution if you want an out of the box experience, but I found it hard to customise, mostly because I’m not very familiar with BSDs. Also, they are switching to Debian for the next version. So, Debian it is.
This post will explain how to set up a NAS server with Debian running essential services such as ssh, samba, nfs, cups, rdiff-backup and rtorrent with a web interface; and using two HDDs in RAID 1 mode with everything encrypted. It took me awhile to research all bits and pieces, hopefully it will save you time if you are going to do a similar set up.
I use a VIA ARTiGO A2000 barebone storage server. It’s powered by a VIA C7-D processor, which has a built-in encryption engine called Padlock — quite useful for our scenario. If you are unsure which server to get, I can highly recommend A2000.
Some parts of this walk-through are specific to A2000 or C7, but most of it will apply to any hardware as long as it includes two HDDs and is compatible with Lenny.
I assume that you know how to use the Debian installer, if not — check the documentation. Because A2000 doesn’t have a CD-ROM, I booted the installer from a memory stick, you might need to do the same.
The tricky part of the installation process is disk partitioning. I used the following layout, though there are many ways to do the set up.
First we create RAID 1 partitions. We need a separate partition for /boot, because it won’t be encrypted; and for /tmp, because it will have encryption settings different from the root partition. This means we will have three partitions on each disk:
Phew, that was quite a few steps! Now you will see three RAID1 devices in the list, let’s set them up:
* When creating a physical volume for encryption, you can select the encryption algorithm and the key size. I use AES, because C7 provides hardware support for it; and 128 bits instead of default 256, because I’m not paranoid. Do your research and preferably select what your hardware supports. Software encryption is likely to be slow unless you have a very fast CPU.
You should see two encrypted volumes now: md1_crypt is automatically set up to be used as swap (do it manually if it’s not); md2_crypt however needs more tweaking.
That’s it! With this scheme, data and root partitions sit on top of an LVM group, which sits on top of an encrypted volume, which sits on top of a multi-disk volume. Some people prefer to have separate encrypted partitions for root and for data, but then you will need to enter passphrases for each of them on start up.
The rest of the installation should be straight-forward. When you reach the “Software selection” screen, make sure you choose “Standard system” and “File server”; and unselect “Desktop environment” — you are not going to need it on a headless server. Also tick off “Print server” if you need (I do).
After everything is installed, boot your server, type your passphrase to unlock the encrypted partition, and login as root. Now, because the installer writes GRUB only to the first disk, we need to install it manually to the second. Without this, if your first disk fails you won’t be able to boot:
# grub grub> root (hd1,0) grub> setup (hd1) grub> quit
Let’s install SSH, otherwise we will need a spare monitor and a keyboard connected to the server:
# aptitude update # aptitude install ssh
Edit /etc/ssh/sshd_config, I suggest disabling PermitRootLogin and PasswordAuthentication and enabling PubkeyAuthentication. If you decide to use public key authentication, add your public key to ~/ssh/authorized_keys. Then restart sshd, install sudo, and edit the list of sudoers:
# /etc/init.d/ssh restart # aptitude install sudo # visudo # Add this line under root, <user> is your non-root login: <user> ALL=(ALL) ALL
This section is specific to VIA C7 CPU. As I mentioned, it includes the hardware encryption engine called Padlock. The engine is supported by the Linux kernel, but the support is not enabled by default.
First make sure you have it:
# modprobe padlock_aes # modprobe padlock_sha
If the modules load fine, these steps (thanks Google Translate!) will auto-load them:
These steps are needed because Padlock modules must be loaded at boot, to work with our encrypted partitions. If they are loaded at a later stage, the software encryption modules will not be replaced because they are already in use.
After rebooting, check if Padlock is used. If aes_i586 is in use instead of padlock_aes, you did something wrong:
# lsmod | grep -i aes
To enable hardware encryption for SSL, edit /etc/ssl/openssl.cnf and add this before the [new_oids] section:
openssl_conf = openssl_def [openssl_def] engines = openssl_engines [openssl_engines] padlock = padlock_engine [padlock_engine] default_algorithms = ALL
After the change, observe an enormous speed bump with:
# openssl speed -evp aes-128-cbc
If you selected “File server” during the installation, NFS should already be up and running. To share the entire /data partition, edit /etc/exports and add this line:
/data *(rw,sync,no_subtree_check)
Check NFS documentation if you want something different. After changing your exports, reload them with:
# exportfs -a
On the client computers, add this line to /etc/fstab, replacing <server> with the IP of your NAS:
<server>:/data /mnt/data nfs defaults 0 0
Then mount with `mount -a`. Again, check the docs if you need more control over how the NFS share is mounted.
As with NFS, Samba should already be running on your server. Append this to /etc/samba/smb.conf, replacing <user> with a non-root login on your server:
[data] path = /data browseable = yes available = yes public = yes writable = yes force user = <user> create mask = 0644 directory mask = 0755
Then restart Samba and you are set:
# /etc/init.d/samba restart
Check Samba docs for more options.
The set up heavily depends on the printer model. I have a fairly common Epson colour ink printer, its driver is included in the gutenprint package which gets installed if you select “Print server” during the installation.
You will need to edit /etc/cups/cupsd.conf to make the CUPS web interface accessible from another machine, then just add your printer from http://:631/. Also check /etc/samba/smb.conf, it should have these sections:
[printers] comment = All Printers browseable = yes path = /var/spool/samba printable = yes guest ok = yes read only = yes create mask = 0700 [print$] comment = Printer Drivers path = /var/lib/samba/printers browseable = yes read only = yes guest ok = no
Check CUPS docs if it doesn’t work or if you want to fine-tune permissions.
FreeNAS comes with Transmission BitTorrent client. It looks nice but the web interface is too simple to my taste, it doesn’t even support labels. On the desktop I used to run Deluge, which is great but probably a bit heavy for a small server. After a bit of research I ended up using rTorrent, which is what most blogs recommend for a headless server.
There are quite a few frontends for rTorrent, the one I liked was ruTorrent, its development also seems to be the most active at the moment. It’s an almost exact rip-off of a popular Windows-based μTorrent client, hence the name.
ruTorrent requires a recent version of rTorrent compiled with the XML-RPC support. The bad news is that Lenny doesn’t have all packages required to build it. This can be circumvented by temporarily switching to testing (aka Squeeze), installing rTorrent’s build-deps, then switching back to Lenny. Depending on your situation, switching to testing may not be the best idea, do it only if you are comfortable breaking your system.
After installing build-deps, get the latest tarball of rTorrent, ./configure it with `–with-xmlrpc-c` option, make and make install. Afterwards, copy an example .rtorrent.rc file to ~/ and edit it to suit your needs. Also follow the steps in the Starting rTorrent on System Startup section.
ruTorrent can work with any web server supporting PHP 5.0, I went for lighttpd. Install it from the official repo, then follow ruTorrent set up guide.
The tricky part is setting up XML-RPC, there are a few contradictions in the the rTorrent and ruTorrent docs but the following works for me™.
Add to ~/.rtorrent.rc:
scgi_port = localhost:5000 encoding_list = UTF-8
Edit /etc/lighttpd/lighttpd.conf as described here. Ignore instructions from rTorrent, they won’t work. Restart rTorrent and the web server after you are done:
# /etc/init.d/rtorrent restart # /etc/init.d/lighttpd force-reload
rdiff-backup is such a fantastic tool: it’s available on all major platforms, it’s ultra fast and efficient, it performs backups incrementally, it can work over SSH and also it allows to restore files at any point of time. If you don’t already use it to backup your home directories — give it a try!
On the server, there’s nothing special to be done to install it. Just get it from Debian repos and add your public keys to ~/ssh/authorized_keys — we are going to use SSH.
On Linux clients, invoke it like this, replacing <user> with your login and <server> with the IP of the NAS:
$ rdiff-backup /home/<user> <server>::/data/Backup/<user>
On Windows clients, install Putty and follow these steps to generate a compatible key. Then invoke rdiff-backup like this:
rdiff-backup.exe --no-hard-links --remote-schema "plink.exe -i C:\Users\<WinUser>\privatekey.ppk %s rdiff-backup --server" C:\Users\<WinUser> <user>@<server>::/data/Backup/<user>
Check rdiff-backup docs for more options, there are plenty!
Extremely unscientific tests, but they give an idea:
# bonnie++ -d /data/tmp
Version 1.03d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
server 2G 10862 91 63699 27 29931 12 11483 91 83455 22 196.7 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 2693 67 +++++ +++ 3467 37 2948 47 +++++ +++ 3131 38
server,2G,10862,91,63699,27,29931,12,11483,91,83455,22,196.7,0,16,2693,67,+++++,+++,3467,37,2948,47,+++++,+++,3131,38
#
# sync
# dd if=/dev/zero bs=16384 count=131072 of=/data/tmp
131072+0 records in
131072+0 records out
2147483648 bytes (2.1 GB) copied, 40.7823 s, 52.7 MB/s
#
# sync
# dd if=/data/tmp bs=16384 count=131072 of=/dev/null
131072+0 records in
131072+0 records out
2147483648 bytes (2.1 GB) copied, 23.5763 s, 91.1 MB/s
# hdparm -tT /dev/mapper/md2_crypt
/dev/mapper/md2_crypt:
Timing cached reads: 584 MB in 2.01 seconds = 291.10 MB/sec
Timing buffered disk reads: 222 MB in 3.03 seconds = 73.19 MB/sec
hda is a 1TB WD Caviar Green, hdb is a 640GB Seagate Barracuda. I know, using different disk models is bad for RAID 1, but that’s what I had. At some point I will get a second 1TB WD, read the next section to find out how to grow the mirror when upgrading drives.
NFS transfers are slower, but good enough for my needs: 49 MB/sec when reading from a NFS share and 23 MB/sec when writing to it.
When your RAID 1 mirror is filled up you probably want to upgrade the disks with bigger ones. This can be done by replacing the first disk, syncing the mirror, then replacing the second one, and syncing again. After that you need to grow your data partition.
So, shut down your NAS, replace one of the drives, boot up and SSH to it. Check the status the mirror, notice that only one drive is used:
# watch -n 2 cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 hda3[0]
622679296 blocks [2/1] [U_]
md1 : active raid1 hda2[0]
1951808 blocks [2/1] [U_]
md0 : active raid1 hda1[0]
497856 blocks [2/1] [U_]
unused devices: <none>
Here I assume that hda is used and hdb has been replaced, run `fdisk -l` to check which is which in your case. Now copy the partition table from hda to hdb:
# sfdisk -d /dev/hda | sfdisk /dev/hdb
Adjust the last partition on hdb: run `cfdisk /dev/hdb`, select hdb3 and delete it, re-create hdb3 to use the entire free space, change the partition type to “FD Linux raid autodetect”, and finally write changes to disk and quit.
Add new partitions to the RAID array and wait until the sync is finished:
# mdadm --add /dev/md0 /dev/hdb1 # mdadm --add /dev/md1 /dev/hdb2 # mdadm --add /dev/md2 /dev/hdb3 # watch -n 2 cat /proc/mdstat
Add grub to hdb:
# grub grub> root (hd1,0) grub> setup (hd1) grub> quit
If you replaced the drive with a bigger one, you need to grow the last partition to take advantage of all available space. Here’s how to do it (the steps are borrowed from here):
# mdadm --grow /dev/md2 --size=max
Reboot, then run this:
# pvresize /dev/mapper/md2_crypt # vgdisplay -A | grep -i free Free PE / Size X / Y GB
Note the number X, we will use it in the next command. Also replace MAIN-data with the name you used for the /data partition:
# lvextend -l +X /dev/mapper/MAIN-data
Finally, grow the filesystem:
# xfs_growfs /data
The previous command will only work for XFS, adapt it if you use ext3 or another file system.
Inspired by this forum post I replaced stock A2000 fans with Scythe Mini Kaze SY124010L 40mm fan on the CPU and Noctua NF-R8 80mm fan on the rear exhaust. This made A2000 even more quiet. Other than that, I cannot think of any other mod I would like to do, A2000 is a very nice piece of hardware.
UPDATE 2009-11-19: this post explains how to do the same in Debian.
I’m building a small NAS for the household. It will run FreeNAS and will be used as a file, rsync, BitTorrent and printer server. I want it to be reliable and secure so it will have two HDDs in RAID 1 (AKA mirroring) and their content will be encrypted.
But what if in the future I will want to upgrade the drives with larger ones? A common scenario with RAID 1 is to replace one of the disks with the bigger one, rebuild the mirror then replace the other one and rebuild it again. In theory it sounds like an easy process that will keep all your data intact.
In practice however it’s not, Mike explains how to do it under FreeNAS in his blog. Growing mirrored and encrypted drives is a bit more complicated.
Here is how, in case you might need it:
After replacing the HDD, boot your box and log into the FreeNAS web interface
Go to Disks/Management, edit the disk you have replaced and click “Save”. This will read the new disk’s size.
Go to Disks/RAID, the status will be DEGRADED which is normal. In Tools select the new disk, “forget” and then “insert” it. Wait until the rebuild process is finished.
Go to Disks/Encryption, attach the disk and make sure your data is fine.
Go to Disks/Mount Point and delete the mount.
Get to the NAS console, either directly or via SSH. Things will get more interesting now:
# geli backup /dev/mirror/raid1 bak # geli detach /dev/mirror/raid1 # geli clear /dev/mirror/raid1
geli is the command line tool to manage encrypted storage in FreeNAS. raid1 is the volume name I used, yours might be different. The first line saves the encrypted volume’s metadata to a file called “bak”, we will need it later.
Go to Disks/RAID again, delete and re-add the RAID. Use the same volume name and tick the “Create and initialize RAID” check box.
Now back to the terminal:
# geli restore bak /dev/mirror/raid1 # geli attach /dev/mirror/raid1
This will restore the metadata from our backup and re-attach the encrypted volume
Fix the partition table, re-create and grow the partition to fill the entire disk:
# gpt recover /dev/mirror/raid1.eli # gpt remove -i 1 /dev/mirror/raid1.eli # gpt add -i 1 -t ufs /dev/mirror/raid1.eli # gpt label -i 1 -l data /dev/mirror/raid1.eli # growfs /dev/mirror/raid1.elip1
Note that the device name ends with “.eli” – it’s our encrypted disk.
Finally go to Disks/Mount Point and mount the partition.
That’s it, your encrypted partition should be functional now!
NOTE: always do your backups, I can make no guarantees that it will work for you.
xmonad is an elegantly minimalist and lightning fast window manager for X written in Haskell. I wanted to play with it for a long time: I’m using two 24″ monitors and so have to spend a fair bit of time re-sizing windows and moving them around. A tiling window manager like xmonad takes care of this; in addition you can control all aspects of window placement with the keyboard alone.
The good news is: xmonad plays really well with GNOME. You can keep your GNOME panels, themes, desktop backgrounds, etc – xmonad just replaces Metacity leaving everything else intact.
The bad news is: I should have tried it earlier.
A few notes about xmonad set up and usage:
Recommended way to set up xmonad with GNOME is to
export WINDOW_MANAGER=xmonad
before starting gnome-session, but it didn’t work for me. I tried every suggested place: ~/.gnomerc, ~/.xsession, ~/.profile, ~/.xinitrc; but none of them worked – GNOME always started with Metacity.
What worked though is this:
Create a file in /usr/share/applications called xmonad.desktop:
[Desktop Entry] Type=Application Encoding=UTF-8 Name=Xmonad Exec=/usr/bin/xmonad NoDisplay=true X-GNOME-WMName=Xmonad X-GNOME-Bugzilla-Bugzilla=XMonad X-GNOME-Bugzilla-Product=xmonad X-GNOME-Bugzilla-Component=general X-GNOME-Autostart-Phase=WindowManager X-GNOME-Provides=windowmanager X-GNOME-Autostart-Notify=true
Change this GConf key from ‘metacity’ to ‘xmonad’:
/desktop/gnome/session/required_components/windowmanager
On startup, xmonad doesn’t set the usual left-arrow cursor but inherits an ugly X cursor which looks like it was created in the eighties. To change it, add this line to your ~/.xinitrc file just before you start gnome-session:
xsetroot -cursor_name left_ptr
You need to create a config file in ~/.xmonad called xmonad.hs and add this:
import XMonad import XMonad.Config.Gnome main = xmonad gnomeConfig
That’s right, the config file is a Haskell program that starts xmonad, which means it’s extremely customisable. Let’s modify it a bit:
1 2 3 4 5 6 7 8 9 10 |
import XMonad import XMonad.Config.Gnome main = do xmonad $ gnomeConfig { terminal = "gnome-terminal" , modMask = mod4Mask , focusFollowsMouse = False , borderWidth = 2 } |
All keyboard short-cuts in xmonad are in the form Mod-X or Mod-Shift-X, where Mod is by default the Alt key. Line 7 tells xmonad to use the Win key – Alt is heavily used by GNOME applications.
When touch-typing, some shortcuts are painful to use, Win-Shift-6 probably being the worst. What I wanted is to replace the Mod part with another shortcut, a bit in the Emacs fashion, so that instead of Win-Shift-6 I would have for example a sequence of Ctrl-m and Shift-6.
mauke on #xmonad was extremely helpful, he came up with this code:
import XMonad import XMonad.Config.Gnome import XMonad.Actions.Submap import Control.Arrow import Data.Bits import qualified Data.Map as M main :: IO () main = do xmonad $ gnomeConfig { terminal = "gnome-terminal" , focusFollowsMouse = False , borderWidth = 2 , keys = addPrefix (controlMask, xK_m) (keys gnomeConfig) } addPrefix p ms conf = M.singleton p . submap $ M.mapKeys (first chopMod) (ms conf) where mod = modMask conf chopMod = (.&. complement mod)
which worked as advertised. To re-load xmonad after you changed the config file just press Mod-q. It takes just a second to re-compile and leaves all open windows intact.
After using xmonad for 2 days I must say I’m a convert. The keyboard short-cuts feel very natural, it’s not difficult to see the influence of vi. Moving a window to another screen or to another workspace (did I mention workspaces are per screen, which is a really neat feature), switching between workspaces, switching windows, changing layouts, etc… is just a short-cut away.
And as a bonus point, I now have a good reason to become more familiar with Haskell – it’s a very nice language.
Subscribe
