Balau

Working to work less

  • RSS Shared Items

    • Rubén Romero: Nokia N900 – What do you want me to test? 2009/12/02
      Ok, so I did it again. I got a Nokia N900 for testing and own amusement. Let the a free experience drive you phone... Unlocks with 'sudo gainroot' And there are lots of things I want to try: Spotify: got libdespotify+gspotify running, but it halts and I have not gotten it to work. I want to find a way that works for humans beings: i.e. no compiling […]
    • PRODUCT HOW-TO: Increase embedded processor efficiency through the use of distributed processing blocks 2009/11/30
      How to incorporate distributed multiprocessing in an embedded design using the Cypress PSoC 3/PSoC 5, which incorporate a main 8051 or Cortex M3 core and many Universal Digital Blocks (UDBs) serving as an array of mini-processors.
    • VirtualBox 3.1 released 2009/12/01
      Sun has announced the release of version 3.1 of the VirtualBox virtualization platform. There's a bunch of new stuff in this release, including a more flexible snapshot mechanism, better 2D video acceleration, and more storage options, but the headline feature appears to be "teleportation," which allows live migration of guests between hosts.
    • Sprites mods - Optical mouse-cam - Optical mouse cam 2009/11/30
      People get bored sometimes, and when I get bored I sometimes open upstuff to find out what makes it tick. One time, I did this with an oldoptical mouse. Well, a mouse only has 2 chips inside: the optical sensorand a chip for the usb/ps2-interfacing. I looked up the datasheet of theoptical sensor (which is an ADNS2610) and it told me the sensor has a tiny 18x […]
    • YC-Funded WakeMate Helps You Kiss Groggy Mornings Goodbye 2009/11/24
      Ah, the curse of the groggy morning. You may have followed all the rules: no caffeine before bed, an early bedtime, and all the rest. But your best efforts are oftentimes for naught, foiled by the mysterious ways of sleep cycles. There may be an answer: WakeMate, a Y Combinator-funded startup that’s launching today, is looking to help you catch that ever-elu […]
    • Open-Sourcing the Global Warming Debate 2009/11/24
      The email and documents recently netjacked from the Climate Research Unit at the University of East Anglia raise serious questions about the quality of the research being used to underpin major public-policy decisions. In the open-source software community, we understand about human error and sloppiness and the tendency to get too caught up in a pet theory. […]
    • Google Chrome OS. Or, how KDE and GNOME managed to shoot each other dead 2009/11/24
      A lot of people at the moment are immensely intrigued by Google Chrome OS. I won’t hide that I am one of them. Google promises a much needed shift in the way small computers work. Problems like software updates, backups, installation, maintenance, viruses, have plagued the world for too long: a shift is way overdue. To me, however, the change about to happen […]
    • Powermat retrofitted into plank of wood, new world of opportunies discovered 2009/11/23
      The Powermat may have slung "inductive charging" into the mainstream (at least to some minor extent), but it's still far too niche to be taken seriously by any electronics aisle in Target. Take that same technology and shove it into wood, however, and you've got something that each and every table owner on the planet could appreciate. Jas […]
    • Pegatron's Ubuntu-equipped netbook spotted in the wild 2009/11/22
      It's been a hot, hot minute since we've seen anything noteworthy from the lads and ladies over at Pegatron, so it's not shocking that we're super tantalized by the prospect of yet another netbook flooding the oversaturated market in the near term. All unnecessary snideness aside, the sub-$200 price point that's being floated about de […]
    • Good karma: an in-depth review of Ubuntu 9.10 - Ars Technica 2009/11/20
      Ubuntu 9.10, codenamed Karmic Koala, was officially released last month. In this comprehensive review, Ars takes you under the surface for an in-depth look at the new features and major architectural changes.

Reinforce ssh security with Denyhosts

Posted by Balau on 2009/11/10

…For years I’ve been saying security consists of protection, detection and response–and you need all three to have good security…

March 2007, Bruce Schneier

I always read Schneier’s site carefully, because everything he says is precise, consistent and grounded in experience. When I stumbled upon this concept of three components of security, I realized that most of my systems have only the first component: protection. My ssh servers, for example, use authentication through RSA key pairs, do not allow root login and listen to a non-standard port. But I have no way of knowing whether these servers were under a brute force attack (for example), unless I check the logs each day when I come home. And even then, that means the attacker had an entire day to try passwords.

Enter Denyhosts.

Denyhosts is a service that detects failed login attempts and reacts to them by adding into the hosts.deny file the IP of the offending hosts that are trying to login. It can also be configured to send a mail when it detects a possible attack. With ssh+Denyhosts  the security of my systems contains all three components:

  • Protection: users cannot login to the servers unless they know the port of the service and possess a strong authentication key.
  • Detection: the login attempts are logged and constantly checked by the Denyhosts service.
  • Response: the Denyhosts service blocks the offending hosts and reports the attack to the administrator, that in turn can take other measures such as stopping the ssh service, adding a rule to the firewall or trace the IP from whence the attack comes.

This setup could also be taken as example for many other applications, and the added strength of the components can be evaluated. For example, in my case the protection against brute force attacks was strong anyway, because of the RSA key authentication, and the weakest link could be, for example, the location of the private keys. A strong detection system could then also audit any successful access and mail the user that is logging in, in order to alarm him if someone stole his account.

Security is as strong as its weakest link; for this reason it is important to have a good understanding of the system and its vulnerabilities, and to act accordingly.

Posted in Software | Tagged: , , , , , , | Leave a Comment »

Italian Linux Day 2009

Posted by Balau on 2009/10/24

The Italian Linux Society promoted today a nation-wide event called Linux Day 2009. I went to the event that was nearest to my home, organized by the local Linux User Group (the GalLUG). They spoke to an audience that was mainly made of 18yo boys of the local technical institute, promoting the Free Open Source philosophy and demonstrating the power of Linux.

  • Presentation of the LUG and its initiatives
  • Open Source, GNU and Linux
  • How to install Puppy Linux on CD or USB Key
  • Linux and virtualization
  • inux steel (a pun on inox steel): a lesson on security
  • #!/bin/bash
  • Ideas and projects with Linux: tips for the 5th year project that the students must complete

They also prepared a stand where anyone could try Ubuntu, and they distributed Live CDs.

Posted in Software | Tagged: , , , , , , | Leave a Comment »

Free relax in an open plan office

Posted by Balau on 2009/10/19

The phone rings. LOUD. It’s from the nearest cubicle. Nobody answers. It’s LOUD. I’m working on something that requires long and constant atten- LOUD -tion.  The phone stops ringing, and the colleagues’ voices return to be distracting again. There’s an informal meeting a handful of seats from me; I’m happy for them that they’re laughing, but I’m afraid the software bug I’m hunting will take the time to hide himself deeper while I’m not looking.

I think that some of you may relate to my experience. Actually, I’m sure. The greatest source of distraction for me is noise, sounds and voices; if you are like me, you will like this lecture about sound on TED:

Julian Treasure: the 4 ways sound affects us

Quoting his presentation, open plan office: productivity loss = 66%. Mr.Treasure also stresses the importance of the positive effect that some sounds have on us, even on an instinctive level; I decided to give the “birdsong therapy” a try at the office. I started searching for sounds to use, and I stumbled upon this social and free database of sounds:

The Freesound Project

Basically every user can contribute to add sounds that can be used freely as Creative Commons. The site has tons of different ambient captures of birdsong. I enjoyed very much, and found suitable for my situation, this 42 minutes long recording of birds. The file’s format is wav, and this means that the size is unnecessarily big. In order to compress the sound on my Ubuntu Linux computer, I used a context-menu driven utility for sound conversion that is a Nautilus add-on. You can install it with:

sudo apt-get install nautilus-script-audio-convert

With this utility I compressed the 420MB wav file into a 63MB good quality OGG/Vorbis audio file (Vorbis is a free open source encoding algorithm). The last problem is that at the office I have a Windows XP box, and I need a player that reads ogg files. For this reason I decided to install the codecs for Windows Media Player:

DirectShow filters – play back ogg files in Media Player

The next day I copied the birdsong inside my workstation, installed the codec, put on my headphones and jumped into the nature. It worked, somehow. The noise of the office was still present, but more distant and subdued under the melody of the forest. In order to completely annihilate the auditory pollution, I feel that classic orchestral music is more appropriate, since it gives a harmonic background covering everything else.

Give it a try: it won’t cost you a penny and it could greatly improve your productivity.

Posted in Working | Tagged: , , , , , , , , , , , , | Leave a Comment »

Secure remote storage with Dropbox and TrueCrypt

Posted by Balau on 2009/10/11

Dropbox is a service for backup and synchronization of files, and it runs on Windows, Mac OS X and Linux.  As I pointed out before, I’d like to be able to use Dropbox without security torments. I don’t think that the guys who run Dropbox really want to peek inside my files, but the risk that someone else does indeed gain access to my data, accidentally or intentionally, is not negligible. A malicious employee, a security breach, the company is sold… I want to feel safe; I need a solution that, on top of Dropbox, adds the security I need. One of the best things about Dropbox is the ability to run on most computer platforms, so a nice solution to the security problem should also possess this quality. The most portable solution up to now seems to be the addition of TrueCrypt. TrueCrypt is a cross-platform encryption software that, among other functionalities, creates files that can be used as encrypted volumes. The idea is to put these encrypted files (that can be considered as safety vaults) inside Dropbox, and to use TrueCrypt on the local copy of the files to decrypt and access the private data. In this way, the data that is stored inside Dropbox is completely unusable by everyone, except the ones who can decrypt it. The decryption can involve a password that a user must remember, a key file that a user must have in his computer, or both.  I like the idea of having both because then, in order to read my data, a potential spy must have:

  • The encrypted vault file (located in my Dropbox or any other computer linked to it)
  • The key file (located in my computers or inside a USB drive)
  • The password (located in my brain)

I think the only feasible attacks to read my data would then be aimed at reading it when I have decrypted it (other than beat me with a 5$ wrench to make me hand over my USB drive and spit out the password).

Installation steps in brief:

  • Install Dropbox
  • Install TrueCrypt (or use it in Portable Mode)
  • Create a TrueCrypt encrypted vault file (with optional key file)
  • Put the vault file in a Dropbox folder
  • The vault file is automatically synchronized by Dropbox

For each other computer that you want to use to access the vault, you need to:

  • Install Dropbox
  • Install TrueCrypt (or use it in Portable Mode)
  • Synchronize the Dropbox folder (to download the vault file)
  • Copy the optional key file

The common use case to access your private data will then be:

  • Mount the vault
  • Access or modify the files inside the vault
  • Unmount the vault
  • The vault file is automatically synchronized by Dropbox

Tips to Ubuntu users:

I created a simple script that opens/closes a vault. It can be easily added to the “Applications” menu.

#!/bin/bash

MOUNT_DIR="${HOME}/truecrypt"
VAULT_FILE="${HOME}/Dropbox/Vault.tc"
KEY_FILE="${HOME}/Vault.tck"

if mount | grep "${MOUNT_DIR}" >/dev/null; then
    truecrypt -d "${VAULT_FILE}" && zenity --info --text="Vault closed: ${VAULT_FILE}";
else
    test -d "${MOUNT_DIR}" || mkdir -p "${MOUNT_DIR}"
    truecrypt --keyfiles="$KEY_FILE" "${VAULT_FILE}" "${MOUNT_DIR}" && gnome-open "${MOUNT_DIR}";
fi

Another useful trick for Linux/Mac users is to keep the files in the Dropbox folder, and create a link where you need them using “ln -s target link_name“. For example, you can copy the “places.sqlite” file that is inside your Firefox profile, and contains your bookmarks and history, inside the Dropbox folder, and create a link to it in your Firefox profile folder. Doing so, you can synchronize your Firefox bookmarks for all your computers.

Posted in Software | Tagged: , , , , , , , , , | 3 Comments »

Ubuntu Live CD with LVM capability

Posted by Balau on 2009/09/26

I recently bought a new 1TeraByte hard disk, and tried to understand the possible ways to add it to my existing Ubuntu Linux system.  On a default system, even with separate home and root partitions, it is quite difficult to exploit the added space of a new drive: one could migrate the home partition to the new drive and expand the root partition to cover the whole old drive, but it is not a flexible nor scalable solution. On a Linux system with Logical Volume Management (LVM) instead, all I should do is add the new drive into the volume group to have the possibility to expand both partitions. LVM is a system that stays between the filesystems and the physical drives, and simplifies the management of separate physical volumes by adding abstraction layers (Volume Groups and Logical Volumes). All the information you might need on LVM can be found here: http://tldp.org/HOWTO/html_single/LVM-HOWTO/

One problem with LVM is that in order to resize the root partition, the root partition must be unmounted. The easiest way is to boot a Live CD and manage the partitions from there. The Ubuntu Live CD does not support LVM: in order to install an Ubuntu system with LVM, the alternate installer CD must be used. This CD lacks the live graphical environment of the default Ubuntu desktop CD, and instead offers a text-based installer; the alternate CD lacks also an easy way to manage partition without reinstalling. What I need is a CD that enables both LVM installation and LVM managing, and this tutorial will show how to create it.

There is a very handy and complete guide here to customize Ubuntu Live CD:

How to Customise the Ubuntu Desktop CD

In order to build the CD I used a VirtualBox virtual machine, but it is not necessary: you just need an Ubuntu machine and administration privileges. The hard disk occupation of the whole process, including the ISO images of the original Live CD and the custom Live CD, is around 3.5GB, so organize your hard disk space accordingly. I downloaded an Ubuntu Jaunty Jackalope desktop CD iso from the official site, and  I followed the guide literally for the sections:

Then, for the Customizations section, I uninstalled some secondary packages to free space for “lvm2″ package and then I installed it:

# apt-get purge gimp gimp-data
# apt-get purge gnome-games gnome-games-data
# apt-get install lvm2

I then wanted to install the graphical and user-friendly tool “system-config-lvm” package; the package is included under the “universe” section that is disabled by default. For this reason I opened the configuration file of the “apt” repositories with the command “nano /etc/apt/sources.list”, uncommented the deb lines of the “universe” section and saved. To have a glimpse of the functionalities of this GUI front-end for LVM you can take a look at the relative section in the redhat enterprise manual.

# apt-get update
# apt-get install system-config-lvm
# apt-get autoremove
# apt-get clean
# apt-get autoclean

I then recreate the filesystem image. Since I used Ubuntu Jaunty Jackalope, the kernel release version is 2.6.28-11-generic. You can discover your release using “ls /lib/modules”, because the name of that directory corresponds to the kernel version of the original Live CD;
change the following “mkinitramfs” command accordingly.

# mkinitramfs -o /initrd.gz 2.6.28-11-generic

Some commands to cleanup the Live CD…

# rm -rf /tmp/* ~/.bash_history
# rm /etc/resolv.conf
# proc ">umount /proc || umount -lf /proc
# umount /sys
# umount /dev/pts
# exit

Then, in the normal command prompt, I moved the filesystem image (initrd.gz) into the casper directory like the tutorial says:

$ sudo umount edit/dev || sudo umount -lf edit/dev
$ sudo mv edit/initrd.gz extract-cd/casper/

I then followed literally the “Putting the CD together” section, and at the end I had an iso image of a complete Ubuntu Live CD with LVM support.

I hope to find the time to do a screencast on how to install Ubuntu with this custom CD and how to modify an already installed LVM system.

Posted in Software | Tagged: , , , , , , , , , | Leave a Comment »

Improve Linux boot time

Posted by Balau on 2009/09/09

A few tips to speed up boot for your Linux box:

1. You need a tool to measure performance objectively: bootchart is very useful to track the lifetime of the processes, disk I/O and CPU load during boot. It needs no configuration whatsoever: you install it (for example using “apt-get install bootchart” in Debian-based systems) and then bootchart will create a PNG image at each successive boot in “/var/log/bootchart/” along with a compressed log. The image is very self-explaining and contains the lifetime of each process, coupled with disk and CPU activity. You can have a better understanding of which processes have more impact on the boot time in order to decide what to do with them.

2. Use readahead: if your distribution supports it (Ubuntu does, for example) there’s a kernel option (profile) that could drastically improve the performance at boot time. If you have Grub as your boot loader, follow these instructions:

  • Power on your box.
  • Enter in the Grub menu.
  • Select your default boot choice and press ‘e’ to edit.
  • Select the line starting with “kernel” and press ‘e’ to edit.
  • At the end of the line, append the word ‘profile’ and press Enter.
  • Press ‘b’ to boot with the temporary ‘profile’ option.
  • This particular boot will take longer to complete and you will hear your hard disk grinding furiously.
  • When you see the login screen, wait until the hard disk stops and then restart.

From now on, you will hear hard disk grinding only at the beginning of the boot sequence. This is called readahead, and it is improving the boot time by placing the files that should be accessed during boot in a comfortable place to read; the list of files is kept in “/etc/readahead/boot”. It is advisable to repeat this operation after each drastic change of configuration or extensive upgrade of the Linux box to have better results.

See also: http://ubuntuforums.org/showthread.php?t=565651 for more readahead tricks.

3. Exploit multiple cores parallelism: if you have a Core2Duo or any CPU with more than one core, you can speed up the boot by running processes in parallel. If you look in the directory “/etc/rc5.d/” you will find many scripts whose name start with ‘Sxx’, where ‘xx’ is a progressive number. This number indicates a sequence that takes into consideration the dependencies between the scripts: the scripts with lower numbers are executed before the scripts with higher numbers.  By default those processes are started sequentially one after the other; alternatively, it is possible to start together at the same time the scripts with the same number. To enable this, edit the file “/etc/init.d/rc” and change the first line containing “CONCURRENCY=none” into “CONCURRENCY=shell”. It is very important that the start order of the scripts is correct, since this option can expose dormant problems that are usually hidden by the fact that the scripts do not run concurrently. If you touched the initialization scripts, be sure to understand this. If you have never touched those scripts but a service fails to start, then the package of that service has probably a bug that should be reported.

Edit: the 3rd advice can be effective also on single-core CPUs.

Posted in Software | Tagged: , , , , , , , , , , , , | Leave a Comment »

Google Reader as a Twitter client

Posted by Balau on 2009/08/27

I always have a Google Reader tab open in my browser, and I predict I will spend more time on it now that it’s becoming more similar to FriendFeed; for these reasons I tried to use it as a Twitter client.

In order to see a Twitter timeline feed, I need to authenticate first. I found GTweet, a web utility that handles authentication and returns a link to an RSS feed enriched with user avatars and buttons. I followed the instructions on the site, created a feed URL and subscribed to it on Google Reader.

I noticed a slow refresh rate (hours before the next refresh); I think I’m too used to real-time conversations. Anyway, clicking Refresh inside the Google Reader subscription will pull the data from the feed and keep it updated.

This setup allows to reply and favorite a tweet. One problem is that the title of each entry becomes a right arrow; this implies that if I want to share an entry with Google Reader Shared Items I will share a nameless entry. The same if I wanted to use the new “Send To Twitter” feature of Google Reader. With “Send To” I can create custom recipients, so I tried to understand if I could send the text inside the entry. Unfortunately, the system lets you access the title of the entry (using ${title}), but not of the internal text.

GTweet, with great honesty, links to a similar service (FreeMyFeed) that performs a similar task but does not reformat the Twitter feed, thus preserving the tweet as the title of the entry. Importing the feed created with FreeMyFeed shows instantly that you lose the features of GTweet, but you gain the ability to share and use custom “Send To” commands. In particular, I created the “Reply” and “Retweet” functionalities quite easily:

Name: Reply
URL: http://twitter.com/home?status=@${title}
Icon URL: http://twitter.com/favicon.ico

Name: ReTweet
URL: http://twitter.com/home?status=RT @${title}
Icon URL: http://twitter.com/favicon.ico

Since the title of the feed includes the Twitter username, if I retweet “balau82: hello world!” my status will be “RT @balau82: hello world!”.

Conclusion

Pros

  • Integrated in a page with all the other sources of news.
  • Integrated with Google Reader internal search.
  • With GTweet:
    • Easy reply.
    • You can still identify the users with their avatars.
  • With FreeMyFeeds:
    • Easy reply and retweet with “Send To”.
    • More tweets on screen thanks to List view.
    • Integration with Shared Items

Cons

  • Refresh is not automatic and does not occur when you refresh the browser page.
  • With GTweet:
    • The title becomes an arrow.
    • Each tweet takes more vertical space than tweets on twitter.com
  • With FreeMyFeeds:
    • The best way to display it is List view, but the setting is global and I hate to see Wired feed (or any other except Twitter) in List view. Also, the embedded search defaults to “Search view” and you need to change to List view every time you search.
    • Ugly: no avatars, no background.
    • In order to follow a link, you need to click the entry and then the link inside it.
    • No links on @username or #tag.

I think that Google Reader is a bad Twitter client because it’s a good feed reader:  it’s not real-time and it’s good at displaying rich content. Anyway, if you check Twitter once a day and need to search specific words in your friends’ tweets, it is a nice alternative to twitter.com .

Posted in Working | Tagged: , , , , , , , | 6 Comments »

Idea: Google Reader sort by likes

Posted by Balau on 2009/08/24

When I have a 1000+ folder, I tend to hesitate before clicking “Mark all as read”, because I’m afraid I’m missing something very important. If I was able to sort the reader items by the number of likes plus the number of times it has been shared, I would  check the hottest items before trashing all the others, and I would feel safer.

Please Google: do it.

[Edit]

Actually, thinking about it, some precautions are needed. Each feed owner could create many accounts just to share and like their own entries and take them to the top… A possible solution could be to sort items by likes from people you follow, since it could also encourage to increase the number of contacts.

Posted in Working | Tagged: , , , , , , , , | Leave a Comment »

Secure remote storage using sshfs and encfs

Posted by Balau on 2009/08/23

I was particularly impressed by Dropbox and its ability to backup, retrieve and share files so easily, but I am quite concerned about the security issues deriving from having your files on a remote, closed-source system. I think that a good remote storage solution should prevent the owner of the storage server from accessing private data. Is it even possible? It is, using the right approach: the client computer must perform encryption when storing remote files, and decryption when retrieving them. I put together a solution for Linux systems using sshfs and encfs.

I used two computers: a server and a client, but for testing purposes just one computer is needed. The server acts as a storage provider, using an ssh server. It can serve multiple users, and each user has a box where he can put anything. A user that connects to the server creates an encrypted space using encfs, and fills the box with private data.

On the server

  • Add a user, for example remote_user1
  • Prepare a directory that will be the remote_user1 personal box. For example:

# mkdir /srv/boxes
# mkdir /srv/boxes/remote_user1
# chown remote_user1.remote_user1 /srv/boxes/remote_user1

  • Install ssh server. On Debian-based systems:

# apt-get install open-ssh-server

  • edit /etc/ssh/sshd_config to include the following options (this will prevent the users to use ssh to do anything but file transfer on their personal box):

Subsystem sftp internal-sftp
ChrootDirectory /srv/boxes
ForceCommand internal-sftp

  • Reduce read/write permissions on the root server directory. This is done to prevent the user to mount the root directory and list its contents.

# chmod 711 /srv/boxes

  • Restart ssh server, for example:

# invoke-rc.d ssh restart

On the client

  • You can add a user, for example user1, or continue with an existing user.
  • Install ssh client, sshfs and encfs. On Debian-based systems:

# apt-get install open-ssh-client sshfs encfs

  • Add user1 to the fuse group

# adduser user1 fuse

  • Log in as user1 (if you are already logged in as user1, log off and log in to render the fuse group effective)
  • Create a folder that will contain the encrypted file system mounted through sshfs

$ mkdir ~/.box_enc

  • Mount the remote directory, providing remote_user1 password (boxserver is the computer where we created the remote boxes before)

$ sshfs remote_user1@boxserver:remote_user1 ~/.box_enc -o uid=$(id -u) -o gid=$(id -g)

  • Create a folder that will contain the unencrypted file system mounted through encfs

$ mkdir ~/box

  • Mount the encrypted directory, creating an encfs environment encrypted with a password. After this step, the directory ~/box can be used as any other directory, but everything in it will be encrypted and stored in a remote folder, specifically the /srv/box/remote_user1 folder in your server. If you list the content of that remote folder (that is also the content of ~/.box_enc folder in the client computer) you will only see encrypted content and the xml file that encfs uses to keep the environment settings.

$ encfs ~/.box_enc ~/box

  • To unmount the directories, execute, in order:

$ fusermount -u ~/box
$ fusermount -u ~/.box_enc

Tips

  • You can setup ssh with key authentication instead of password authentication to avoid typing the password every time you want to mount the encrypted remote folder. To do so, on the client side:

$ ssh-keygen
$ cp ~/.ssh/id_rsa.pub ~/.box_enc #~/.box_enc must be mounted

On the server side, logged in as remote_user1:

$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ cat /srv/boxes/remote_user1/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
$ rm /srv/boxes/remote_user1/id_rsa.pub

  • You can setup encfs to read the password from a file, using for example:

$ read -s -p 'password>' pwd
$ touch ~/.encfs_pwd
$ chmod 600 ~/.encfs_pwd
$ echo "$pwd" >> ~/.encfs_pwd
$ pwd=
$ encfs --extpass='cat ~/.encfs_pwd' ~/.box_enc ~/box

  • You can use the mounted directory as a backup, and use rsync to synchronize a local folder and your backup folder. For example:

$ #To pull your files from your backup folder:
$ rsync -uav ~/box/ ~/Documents/
$ #To push your files to your backup folder:
$ rsync -uav ~/Documents/ ~/box/

  • Then you can schedule a job to keep them synchronized. To run a sync every minute:

$ crontab -e
* * * * * rsync -ua /home/user1/box/ /home/user1/Documents/ && rsync -ua /home/user1/Documents/ /home/user1/box/

  • To change the behavior of the server so that only a certain group of users are restricted to the file transfer functionality of ssh, you can use the Match keyword inside the sshd_config file, before the “ChrootDirectory” and “ForceCommand” lines. (see “man sshd_config”)

Security

The security implications of this setup are complicated. Mostly, this setup is as secure as the combination of encfs and your client computer. The remote server cannot see the content of your files unless it knows the encfs password, but it can see the number of files and the directory structure (see http://www.arg0.net/encfsintro and encfs man pages for more details). Theoretically, it could be possible for someone with access to the remote server to understand what kind of files you have on your box. For example, a big 400MB file could be a movie, a compressed file or a CD image; a directory could have a structure that resembles that of a Firefox profile folder.

Usability

The setup I just described is quite easy to follow if you know Linux well. It is tricky if you’re not expert. This solution can be automated in some aspects, for example an installation could setup the remote box to mount at login. Moreover, a nice GUI could be developed to do a quick start, to select backup options and security settings. The performance is not so great, since the data must be encrypted two times: one for encfs and one for ssh; it is however faster than I thought (I have not done benchmarks); moreover both sshfs and encfs have options (like cache and compression) that could speed up the file transfer. About the Operating System: this setup works on Linux, but I think it should work also on Mac OS X thanks to the macfuse project that enables, among others, sshfs and encfs. On Windows sshfs is available through Dokan libraries, but I can’t find anything for an encfs port.

Final considerations

This experiment is satisfying from a technical point of view, but it lacks the usability to be considered a full-blown alternative to Dropbox; I think that a better alternative could be Novell’s iFolder. Currently, from what I found, iFolder server kinda works on non-Novell distros, but it needs some hacks and the installation is not straightforward (see http://ubuntuforums.org/showthread.php?t=1163192&page=3), but it’s promising. I think I will try to install a complete setup when I have the time.

[Update]

I found a blog that reports stability problems with sshfs + encfs setups. I haven’t found the timestamp problem he mentions, but the two computers I used are tightly connected. I didn’t do a “heavy load” stress test either, just syncronizing a directory containing the sources and binaries of a small project.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in Software | Tagged: , , , , , , , , , , , , , , | 3 Comments »

Social Earthquakes

Posted by Balau on 2009/08/11

The users of the Web 2.0 rely on many sites to discover, share and publish content. But these days I think that many people felt the earth crumble under their feet, when they found out that the sites they rely on are in an unstable situation. For example:

  • The URL shortener http://tr.im/ is closing.
  • Friendfeed has been bought by Facebook.
  • Twitter has been attacked by hackers.

The consequences of the changes could be great on the end user. If tr.im closes and an user has always used tr.im on his site, then all of a sudden all his links would be broken. If Facebook decides that Friendfeed’s “automatic post to Twitter” feature is not welcome, people relying on it would need to find another solution fast. People relying on private Twitter timelines to do “private” tasks from todo lists to turning on the heat at home remotely, they feel insecure and could stop to use their solution.

I think that two things are required in order to survive social earthquakes:

  1. Redundancy
  2. Easy data backup, manageable data restore

Redundancy means that if something fails, you have something to fall back to. I have both a twitter and an identi.ca account that can be used in the same way. The Friendfeed “automatic post to Twitter” feature can be replicated by Twitterfeed (that can also post to identi.ca, by the way).

Backing up your data is not always easy and not always automatic. WordPress.com allows to backup your entire blog and restore it later, even on a home server with WordPress.org installed. The restoring process can’t be prepared for every kind of situation, but at least it should be possible to restore by hand. For example, one could find a script that, for each shortened link on a page, writes on a text file the shortened URL and its expanded address. If the URL shortener closes, the links on the pages can be replaced, even one by one. It could be cumbersome to restore the original situation, but the links are not entirely lost.

Meanwhile, many tools come to help. Recently Google Reader became more like a small Friendfeed,  with a “Note this” bookmarklet, friends to follow, “Likes” and access through RSS. The Internet is always moving; to think that it won’t change is a fallacy, to act like it won’t change is a recipe for failure.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in Software | Tagged: , , , , , , , , , , | Leave a Comment »