FreeBSD jail on embedded Nas4Free install

Setting up a FreeBSD jail on embedded Nas4Free install

As most DIY computer geeks i have a server at home, more specifically a DIY Nas. It is basically an old p4 mini atx motherboard i had laying around with a raid controller card and a couple of hdd’s. The Nas runs an embedded FreeBSD distribution called Nas4Free,

Since the distro is an embedded install this means that any changes you make to it are gone when the server is restarted. So how can you extend its functionality and for example add a subsonic server to it.

The answer lies in Freebsd jails.Jails, sometimes referred to as an enhanced replacement of chroot environments, are a very powerful tool for system administrators, but their basic usage can also be useful for advanced users.

Jails improve on the concept of the traditional chroot environment, in several ways. In a traditional chroot environment, processes are only limited in the part of the file system they can access. The rest of the system resources (like the set of system users, the running processes, or the networking subsystem) are shared by the chrooted processes and the processes of the host system. Jails expand this model by virtualizing not only access to the file system, but also the set of users, the networking subsystem of the FreeBSD kernel and a few other things.

A jail is characterized by the following characteristics:

  • A directory subtree — the starting point from which a jail is entered. Once inside the jail, a process is not permitted to escape outside of this subtree.
  • A hostname — the hostname which will be used within the jailm usualy a descriptive one for the service that is running inside the jail.
  • An IP address — The IP address of a jail is usually an alias address for an existing network interface, but it is not an requirement.
  • A command — the path name of an executable to run inside the jail.

All of this means that this is the correct way to go when adding functions to an Nas4Free embedded install. So after some extensive googling and reading about FreeBSD jails i was confident enough to try setting up an jail.

Configuring Nas4Free

  1. Go to this page: http://wiki.nas4free.org/doku.php?id=documentation:setup_and_user_guide:services_ssh
  2. Check so that ssh is enabled and check the port number and also check that the option “Permit root login”is enabled.(The root password is the same as the WebGUI password but the login name is always “root”)
  3. Go to the Nas4Free webgui and navigate the menu like this: System->Advanced->sysctl.conf
    Add there:
    Name: security.jail.chflags_allowed
    Value: 1
    Comment: can be whatever you want.
  4. Now navigate in the webgui like this: Advanced->File Editor
  5. In the file path textbox write “/etc/rc.conf”
  6. Click load
  7. Add to the file jail_enable=”yes”
  8. Click the save button next to the textbox where you wrote the path to the file and then restart the nas4free server.

And now the fun starts ssh via putty or some other equivalent to the server and follow the following steps.

Create the folders and mount points

Remember to change all reference to /mnt/data to the mountpoint on your Nas where you are going to store the jail.

  • mkdir /jail
  • mkdir /mnt/data/jail
  • mkdir /mnt/data/jail/{work,plugins,conf}
  • mount_nullfs /mnt/data/jail /jail

The mount_nullfs command points /mnt/data/jail to /jail for ease of installation and use.

/jail/work is used for downloads,temporary files.
/jail/plugins the jail itself, this is where we are going to install subsonic.
/jail/conf contains the configuration and run-time files.

Download and extract the FreeBSD base system

The base system has to be downloaded to make sure you get all the necessary binaries, config files and scripts. To download it you can just copy paste the following commands into the ssh shell.

  • cd /jail/work
  • fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/`uname -m`/`uname -m`/`uname -r | cut -d- -f1-2`/base.txz
  • fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/`uname -m`/`uname -m`/`uname -r | cut -d- -f1-2`/lib32.txz

The last command for fetching lib32 is not needed if you are running Nas4Free on an 32bit system.

The following two commands extract the base system into the plugins folder inside the jail.

  • tar xvf /jail/work/base.txz -C /jail/plugins/
  • tar xvf /jail/work/lib32.txz -C /jail/plugins/

Installing the plugins jail binaries

  • cd /jail
  • mkdir -p conf/root/{etc/rc.d/,usr/bin,usr/sbin}
  • cp plugins/etc/rc.d/jail conf/root/etc/rc.d/
  • cp plugins/usr/sbin/{jail,jexec,jls} conf/root/usr/sbin/
  • cp plugins/usr/bin/mktemp conf/root/usr/bin/

The commands above create the file structure for the runtime files, also copy the nescessery rc script and binaries to conf.

Configuring the jail

  • cp /etc/resolv.conf /jail/plugins/etc/
  • cp /jail/plugins/usr/share/zoneinfo/Europe/Stockholm /jail/plugins/etc/localtime

The commands above copy the resolv.conf file from the Nas to the jail and also the timezone file. Obviously exchange Europe/Stockholm for your own timezone. Next we will configure the mounts that the jail is going to be able to access

  • touch /jail/conf/fstab.plugins
  • mkdir /jail/plugins/mnt/DataDisk1
  • nano /jail/conf/fstab.plugins

Copy into the fstab file the following lines:

/mnt/data/DataDisk1 /jail/plugins/mnt/DataDisk1 nullfs ro 0 0

Of course exchange DataDisk1 for the mounts that you have on the Nas that you want to be accessible in the jail. The next part of the configuration is to create the rc.conf file.

  • touch conf/rc.conf.local
  • nano conf/rc.conf.local

Copy into the rc.conf.local the following lines:

jail_enable=”YES” # enable jails YES|NO
jail_list=”proto” # name of the jail to start, it can be basically whatever you want “proto www…”
jail_proto_rootdir=”/jail/plugins” # path to our jail
jail_proto_hostname=”plugins.domain.local” # hostname
jail_proto_ip=”192.168.2.201″ # ip of the jail, replace with a ip in the same subsystem as your NAS
jail_proto_interface=”fxp0″ # Network Interface to use, replace with your NAS interface name
jail_proto_devfs_enable=”YES” # use devfs
jail_proto_mount_enable=”YES” # mount YES|NO
jail_proto_fstab=”/jail/conf/fstab.plugins” # File with Filesystems to mount

And the last step is to create the jail start-up script

  • nano /jail/conf/jail_start

Copy into jail_start the following lines:
#!/bin/tcsh -x
#mounting to /jail
mkdir /jail
mount_nullfs /mnt/data/jail /jail
# copy jail binaries to /usr, not needed if N4F is 454 or up
# because Daoyama include needed files, uncomment if you use low .454 version
# cp -r /jail/conf/root/ /
# link config files to /etc
ln -s /jail/conf/rc.conf.local /etc
#start all jails
/etc/rc.d/jail start

For the startup script to be executable we have to make it executable via the following command:

  • chmod 755 /jail/conf/jail_start

And to make it run each time the Nas server is started we add it via the webgui under: System|Advanced|Command Scripts.

Command: /mnt/data/jail/conf/jail_start
Type: PostInit

Save and apply, and reboot your server. After a successful reboot you can check your new jail via SSH using the jls command. If everything went as it should you should see something like this:

JID             IP Address                   Hostname                      Path
1                192.168.1.201             plugins.domain.local       /jail/plugins

If the output of the jls command is different, type the following command: rehash and then try the jls command again. If the output is still different then go over the steps and verify that you didn’t miss a step.

P.s. to enter the jail you use the jexec command in the case of the plugins jail you would type in the ssh console ” jexec 1 csh “.

So basically that’s how you set up a FreeBSD jail on a Nas4Free embedded install.

About these ads

21 thoughts on “FreeBSD jail on embedded Nas4Free install

  1. Thanks for this guide. I’m glad you did this so well that I colud follow it.

    I’m having trouble completing this, mainly because it seems I can’t do the first “mkdir /jail”. When I ssh in I get something like “Operation not permitted”. Do I have to login as root? If so, how do I do that? Or add that to the user I’m sshing with?

    • This is an error on my part and i appologize for that, i kind of skipped the part on how to setup ssh and one important part about a setting that has to be added via the webgui.

      Heres a quick howto for those steps:
      1. Go to this page: http://wiki.nas4free.org/doku.php?id=documentation:setup_and_user_guide:services_ssh
      2. Check so that ssh is enabled and check the port number and also check that the option “Permit root login”is enabled.(The root password is the same as the WebGUI password but the login name is always “root”)
      3. Go to the webgui and navigate the menu like this: System->Advanced->sysctl.conf
      4. Add there
      Name: security.jail.chflags_allowed
      Value: 1
      Comment: can be whatever you want :)
      5. Now navigate in the menu like this: Advanced->File Editor
      6. In the file path textbox write “/etc/rc.conf”
      7. Click load
      8. Add to the file jail_enable=”yes”
      9. Click the save button next to the textbox where you wrote the path to the file and then restart the nas4free server and then you should be able to make the jail setup without any problems. :)

      i will update the post during the weekend now and add the steps i wrote above to the post, sorry for this and hopefully this should solve your problem. :)

  2. I have followed the instructions a couple of times now, with no joy. jls shows nothing.

    Can you direct me to logs to check for errors etc? dmesg shows nothing.

    Thanks for your help, very insightful.

    • I suspect the error is maybe because i forgot to add to the post the initial nas4free config that i wrote in the comment field, i have added it now to the post. It helped another user that couldnt get the jail install to work. If its not that then there could be several things where there is an error,let’s start with some basic things where there could be an error. Do you have the required settings in the nas4free sysctl.conf? And also do you have jail_enable=”yes” in the nas4free rc.conf? Are the settings in sysctl.conf and rc.conf persistent? Also what output is there when you run the jail_start script?

      • So the jail appears to have installed correctly, however jls results in no information being presented, just the headers. I saw the additional steps in the comments later and added the sysctrl.conf information afterwards. And, if by persistent you mean the settings are there after a reboot – then yes.

        One major difference I may have from you is that my drives are encrypted – so I would expect the script to fail until I enter the password to decrypt them.

        When I run the ./jail_start it echos all unconnected lines, the states configuring jails, then starting jails.

        Any ideas?

      • So I noticed a couple of issues…I think…in your steps:

        You create /jail/plugins, but reference proto in several areas…instead of, I assume plugins. After changing everything to plugins, I tried to start jail and receive the following:

        /etc/rc.d/jail: WARNING: $jail_enable is not set properly – see rc.conf(5).
        Cannot ‘start’ jail. Set jail_enable to YES in /etc/rc.conf or use ‘onestart’ instead of ‘start’.

        This doesn’t make any sense, as jail_enable is indeed set to YES:
        nano /etc/rc.conf shows the following (excerpt)

        fuppes_logfile=”/var/log/fuppes.log”
        jail_enable=”YES”
        mtdaapd_logfile=”/var/log/mt-daapd.log”

      • Please read all my comments, I think your post might have some errors.

        My problem was copying nd pasting some of your code. The quotes didn’t copy correctly. Messed a bunch of stuff up.

        Thanks for the info though, now onto your next post to install subsonic.

    • Hi again, sorry for not writing back that fast, but i have been really busy with moving and everything. The reason why i referenced proto in several places is because proto is just the name that i use for the jail, you can name it anything you want and i went with proto for “prototype” :)

      And for the :
      “/etc/rc.d/jail: WARNING: $jail_enable is not set properly – see rc.conf(5).
      Cannot ‘start’ jail. Set jail_enable to YES in /etc/rc.conf or use ‘onestart’ instead of ‘start’.”

      That is an interesting problem, i actually have no idea why it would say that it is not enabled when it clearly is :/ I have however read on a couple of forums that sometimes using lowercase letters for rc.conf settings helped other people get rid of this kind of problem. When i nano my /etc/rc.conf on my nas4free install i get the following:

      fuppes_logfile=”/var/log/fuppes.log”
      jail_enable=”yes”
      mixer_enable=”yes”
      mtdaapd_logfile=”/var/log/mt-daapd.log”
      powerd_flags=”-a adaptive -b adaptive -n adaptive -M 638 -m 600″
      rsync_client_logfile=”/var/log/rsync_client.log”
      rsync_local_logfile=”/var/log/rsync_local.log”

      So you could try writing yes in lowercase letters, i know it seems kind of irrelevant if its written in capital letters or lowercase letters but it seems to make a difference for some setups.

      Is the jail functioning properly now? I presume that it is because you said you were moving on to installing subsonic. The subsonic install post that one is pretty straightforward so hopefully you are not going to get any problems there.

      • Everything went fine in the end. It was copying the ” character – they were wrong.

        Subsonic install went fine. However I cannot decode flac or AAC, any ideas?

        I believe it is related to the lame, ffmpeg, and flac packages.

      • Maybe you misunderstood me when I mentioned I noticed some issues in your code. You reference plugins and proto. I believe it should be one or the other.

      • Unfortunately this is related to Jukebox only. I think the real problem is the ffmpeg package I install/add does not have –enable-libmp3lame set. By default –disable-libmp3lame is set. I am not sure how I would compile this port? myself with the flag enabled. Are you familiar with this process? Thanks for all your help.

      • It should be pretty straight forward, you could try to log in to the jail, grab a portage tree, cd to the port directory and then run “make config” to enable libmp3lame and then run “make install clean”. For a more detailed instruction you should check out this post at the nas4free forum http://forums.nas4free.org/viewtopic.php?f=79&t=1796 its for how to compile ffmpeg for servio so not everything in the post applies to your situation but it gives a good description of how to go about compiling the port.

      • I finally got it to work, following your suggestions. I needed to install lame 3.98.4 before complete the ffmpeg compilation.

        Thanks for all your help!

    • Hi, glad to hear that you found it useful. But since i wrote this post a user on the nas4free forum named fsbruva has made an extension for nas4free that sets up a jail in the mater of minutes just with the help of 3 simple commands that you run via ssh. The user raulfg3 on the nas4free forum has written a really simple easy to follow tutorial for installing “thebrig” on a nas4free setup at http://forums.nas4free.org/viewtopic.php?f=79&t=3894&p=21209&hilit=thebrig#p21209. So now with the help of “thebrig” it’s even simpler to setup a jail on a nas4free setup. :)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s