Wednesday, 16 November 2011

openSUSE 12.1 released - spread the word!

Congratulations to everyone who has worked hard on openSUSE 12.1 for another successful release. Can't wait to get it running.

In the meantime, spread the word!

Stories are appearing on Slashdot and HackerNews. Upvote, comment, discuss! Suggest to the websites and magazines you read to run a review. Zonker over at Linux.com has already written an intro to 12.1 piece.

Tell your friends on Twitter and Facebook! Let everyone know about the work openSUSE contributors have put in for the latest release - 12.1 has an amazing feature list.

openSUSE is one of the major distributions in existence, both in terms of number of users and in contributions back to the rest of the community, we should all be extremely proud to be part of it, I know I am.

Saturday, 13 August 2011

redshift and backintime in Factory

Well, it's been a long while since my last news ... but I've got some nice news to report - my submissions of redshift and backintime were accepted into openSUSE Factory. So these useful programs will be part of the next openSUSE release, hooray!

Redshift is a little command-line (and GTK) utility which reduces screen brightness at night (via colour temperature, so your screen becomes more red). Given your location in the world, it automatically calculates sunset and sunrise and gradually ramps up/down the brightness at the right times. I find it makes working at night easier on the eyes, and wrote a little plasmoid to control it easily from the desktop, which I will also release when I get some time.
For previous openSUSE versions one can download and install redshift from its OBS devel project, X11:Utilities.

Backintime is a backup program, effectively a rsync frontend, with GUIs for KDE and GNOME. Once configured with which folders you want to back up, and where you want to back up to, it can automatically take backups based on a schedule. Also a very nice feature is that on filesystems that support it (not FAT), it uses hardlinks to the previous backup, so each backup is a full backup but only changed files take up space on the backup disk. Each backup is just a normal copy of all the folders, so no special software is needed for recovery, but backintime offers a GUI that allows you to easily navigate through all your backed up versions of a folder. IMO it's the best userfriendly but complete backup software for Linux desktops. backintime used to be packaged by packman and now lives on the OBS in Archiving:Backup.


Enjoy! If you have other interesting programs that aren't in openSUSE remember that anyone can contribute to Factory now, so if you are willing to maintain them, go ahead and submit them on the OBS! My next target is byobu, a set of preconfigured GNU screen profiles (I never figured out how to write hardstatus lines!)

Wednesday, 20 April 2011

Development and build environments: schroot on openSUSE -- part 2

So in part 1 I detailed my search for something that would allow my development and productive environments to coexist on the same machine, and how I discovered schroot.

Schroot is a "chroot manager" - it allows configuring chroots so that users on the system can run shells and processes in them, and it takes care of all the setup/tear down I described at the end of my previous post. It is a part of the Debian buildtools, for building Debian packages in a safe and repeatable environment, just like openSUSE uses build.

Advantages of using schroot to setup development environments
Vs Full Virtualization:
  • low hardware requirements
  • easy network setup (just copy /etc/resolv.conf into the chroot)
  • can run X apps directly into running host X display
  • easy to share - host always has access to all guest files
  • can just run out of a directory
Vs plain chroots
  • can easily setup multiple environments
  • can run out of disk images or actual devices
  • takes care of all the setup work
  • manages sessions
  • good cleanup of any leftover processes and mounts on exit
Disadvantages of chroots vs full Virtualization
  • No real process isolation - no difference between a process inside the chroot and outside the chroot. Even 'ps' shows processes both inside and outside the chroot (/proc is shared)
  • No real security - trivial for a malicious program to break out of a chroot, full host hardware access by mounting all of /dev (unless you spend the time to only allow through specific devices)
  • OS is never actually booted (no init) or shutdown, just specific processes run, so harder to run daemons

First step was packaging schroot, no problem there, except that it doesn't seem to exist anywhere except in the Debian package system. I just grabbed the source from their source dpkg. I found that flichtenheld also had schroot packaged in his home project on OBS, so I grabbed some patches that looked useful (thanks!). The package is currently in my home project, it's been SR'ed to devel:tools, but is also waiting on a security review (since it runs setuid)

Next was configuring it. Read the schroot manpages, they are quite complete. schroot comes with three chroot types: default, desktop, and minimal. Since I wanted to run X apps, I used the desktop type, with a few modifications. All you have to do is create a configuration file in /etc/schroot/chroot.d/. For example, here's my development environment.

[devel]
type=directory
description=Development 11.4 installation
groups=wheel
root-groups=wheel
script-config=desktop/config
directory=/home/chroot

Note that I've allowed anyone in the "wheel" group (just me atm) to use this chroot, as a user or as root.

Then I setup the directory /home/chroot. Zypper has a very nice feature that allows bootstrapping openSUSE installs in situations like this: the "-R, --root" option where zypper behaves as if that is the root directory.

#first add a repository - I had my 11.4 dvd handy so I decided to save some bandwidth
sudo zypper -R /home/chroot ar dvd:///
#then install needed packages. base or minimal_base pattern is a good idea. sw_management installs zypper
sudo zypper -R /home/chroot in -t pattern base sw_management

Now you can chroot in with
schroot -c devel

A nice chroot is all set up for you, you can do your work, and when you leave it's all cleaned up. I was quickly able to run X apps, screen, and all my development tools - so far I've been able to do everything I need to. Excellent!

Schroot also comes with some pretty cool support for LVM and brtfs-based chroots - it can automatically create temporary sessions based off a "source chroot", which I thought was pretty cool and might use that in the future.

schroot is quite an extensible program; for example, it runs the scripts it finds in /etc/schroot/setup.d/ to create the chroots - you can easily add to this. I added tweaks to change the bash prompt so I know I'm in a chroot, allow ssh-agent access from the chroot, setup X11 authorizations properly to run X apps, and allow 'screen' usage.

If you want dbus system bus to work in your chroot, add to /etc/schroot/desktop/fstab
/var/run/dbus   /var/run/dbus   none    rw,bind         0       0
I also commented out /home since I didn't want my home dir shared into the chroot.

In /etc/schroot/desktop/config, I added these lines to give some config values to my custom script:
#Environment variables and values to copy over
ENVVARS="DISPLAY=:0"

#Display to extract authorization from
DISPLAY_AUTH=":0"

And I created the file below that is run every time a chroot is setup by schroot. Note it runs as root, outside the chroot. Anything that needs to be setup inside the chroot can be done via for example /etc/bash.bashrc.local, as I do for setting the prompt below. The variables used are all documented in the schroot man pages.
/etc/schroot/setup.d/75setupsuse contents:
#!/bin/sh
# Copyright © 2011 Tejas Guruswamy <tejas.guruswamy@opensuse.org>
#
# schroot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# schroot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see
# <http://www.gnu.org/licenses>.
#
#####################################################################

set -e

. "$SETUP_DATA_DIR/common-data"
. "$SETUP_DATA_DIR/common-functions"

if [ -f "$CHROOT_SCRIPT_CONFIG" ]; then
    . "$CHROOT_SCRIPT_CONFIG"
elif [ "$STATUS" = "ok" ]; then
    fatal "script-config file '$CHROOT_SCRIPT_CONFIG' does not exist"
fi

if [ $STAGE = "setup-start" ] || [ $STAGE = "setup-recover" ]; then

  info "Setting chroot environment and display"

  info "Grabbing .Xauthority from ${AUTH_USER} $DISPLAY_AUTH"
  su ${AUTH_RUSER} -c "xauth -f /home/${AUTH_RUSER}/.Xauthority extract ${CHROOT_PATH}/home/${AUTH_USER}/.Xauthority $DISPLAY_AUTH"

  info "Finding running ssh-agent"
SSH_AUTH_SOCK=$(find /tmp -user ${AUTH_RUSER} -name "agent*")
SSH_AGENT_PID=$(pidof ssh-agent)

  info "Adding ssh-agent variables to environment"
ENVVARS="SSH_AGENT_PID=$SSH_AGENT_PID SSH_AUTH_SOCK=$SSH_AUTH_SOCK $ENVVARS"

  vars=''
  for var in $ENVVARS; do
    info "export $var"
    vars=$(echo -e "$vars\nexport $var")
  done

  info "Setting chroot bash prompt"
  cat >"${CHROOT_PATH}/etc/bash.bashrc.local" <<'EOF'
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then
debian_chroot=$(cat /etc/debian_chroot)
fi 
PS1="${debian_chroot:+($debian_chroot)}$PS1"
EOF

  echo "$vars" >> "${CHROOT_PATH}/etc/bash.bashrc.local"

  info "Creating temporary directories necessary for screen"
  mkdir -m 0755 ${CHROOT_PATH}/var/run/screens
  mkdir -m 1777 ${CHROOT_PATH}/var/run/uscreens

fi

if [ $STAGE = "setup-stop" ]; then

  rm -f ${CHROOT_PATH}/home/${AUTH_USER}/.Xauthority
# rm -f ${CHROOT_PATH}/etc/bash.bashrc.local
  rm -rf ${CHROOT_PATH}/var/run/screens
  rm -rf ${CHROOT_PATH}/var/run/uscreens

fi

Development and build environments: the search -- part 1

One of the problems I often face is keeping my productive system I use for work, separate from my development environment with broken versions of programs and loads of extra packages installed.

For a while my solution was to create virtual machines in VirtualBox - which works great but has a very high overhead for the task. Especially when I am away from my desktop and only have my slightly underpowered laptop (no VT-x extensions, only 2GB RAM), running an entire virtualized system just to try out some packaging changes was painful.

So recently I started experimenting with some other solutions. My requirements were
  • low system requirements
  • easy full network access in the guest
  • able to run X apps (I like to have a development environment where I run KDevelop, plus most of the apps I develop are graphical)
  • process isolation (don't want a bad command ruining my host system)
  • file-level isolation (don't want my host polluted with loads of debuginfo, devel, and unstable packages/files)
  • at the same time, it should be able to share files if necessary
  • minimum of modification to the host system 
Probably the ideal solution would be something like FreeBSD jails, but I'm running Linux :) Of course I could just run many installations on the same computer, but rebooting all the time is very tiresome.

My first experiment was with Xen. Since my laptop doesn't have hardware virtualization extensions (even though it is a relatively recent Intel cpu, it's a cheap one), I had to use paravirtualization. Xen dom0 (host) needs a different kernel, but this was easy in openSUSE (Novell have traditionally supported Xen) install and setup went fine - in fact YaST has a module "Install Hypervisor and Tools". I was able to create and install a VM via virt-manager. But even paravirtualization was really, really slow on this machine - and interacting with the VM via vnc was quite awkward. All in all no better than the full virtualization I had before.

Next I tried LXC - Linux Containers. They are basically super-chroots, using the power of the new kernel cgroups. Initially it looked like exactly what I needed - similar to FreeBSD jails. Overall it is comparable to OpenVZ (I didn't try OpenVZ because it needs a special kernel), almost but not quite virtualization (also called "operating system level virtualization"). It is a very new project though, and it showed, as there were no easy frontends to configure and run them. libvirt / virt-manager was supposed to have support for LXC, but I couldn't get it to work quite satisfactorily. Also, the OS running inside it needs heavy modification as LXC doesn't present any hardware by default (just a chroot, remember). To get the installed guest OS to boot, all the init scripts need to be gutted to remove anything trying to interact with hardware directly (HW clock, console, display, udev, hal). This was a painful process, and I couldn't get it quite right even with some extensive (but out of date?) documentation on the openSUSE wiki and elsewhere. On the plus side the hardware requirements of this were very low, as you would expect for a chroot. Perhaps in the future, when easy frontends exist, and distro's have versions suitable for installation in LXC, this would be an excellent solution.

So the chroot-based LXC was almost what I needed - I looked around for other chroot-based systems. The openSUSE "build" script sets up chroots for build environments but is too focused on that, it doesn't give you much help for using the chroots for anything else. I found that ssh can automatically chroot users on login, so I tried setting up a "devel" user, configuring appropriately, and ssh'ing into localhost. But the purpose of ssh's chroots is security, so it doesn't help much with the task here either.

I tried writing my own chroot management scripts. What is needed is to mount various directories (/dev, /sys, /proc) into the chroot so applications have access to the kernel/hardware, and possibly copy over some settings (e.g. /etc/resolv.conf for network access). It should also have some cleanup functions so that when the work in the chroot is done, it removes these mounts, ensures there aren't any stray processes, and leaves the host system in a consistent state. I had a few attempts at this, but it turned out to be harder than it seemed. I figured someone else must have experienced this issue, this is Linux after all, there must be something that does what I need. After spending most of my time looking at RPM-based distro solutions to this problem, I had the idea to look at Debian - and they seemed to have exactly what I needed. One of the Debian buildd-tools is a program called schroot.

In part 2 I cover how I got schroot working on openSUSE and my experiences with it so far.

Sunday, 27 March 2011

openSUSE 11.4 - first steps

Finally term is over and I'm on my Easter break, which means I can step up my contributions again.

First step was to install the new release on my desktop - openSUSE 11.4 looks really, really good!
No hassles during installation at all, took about 20mins from DVD.

Here's what I did after installation:
  • I prefer using sudo, so setup sudo via visudo and tell KDE to use sudo

    kwriteconfig --file kdesurc --group super-user-command --key super-user-command sudo

  • Add yourself as a PolicyKit administrator
    Open krunner (Alt+F2) -> type "global policy" -> open "Global Policy Configuration", add users/groups as required. Now you'll be asked for your password instead of root's, just like sudo.

  • Install subpixel hinting, and tell everything to use DejaVu fonts as defaults

    sudo zypper ar -f --repo http://repos.opensuse-community.org/subpixel/openSUSE_11.4/Subpixel.repo
    sudo zypper in fontconfig-feature-subpixel-hinting freetype2-feature-subpixel-hinting

  • Get Yakuake working and enable autostart - can't live without it!

    sudo zypper in yakuake
    cp /usr/share/applications/kde4/yakuake.desktop ~/.kde4/Autostart

  • Install nvidia proprietary drivers (though I was impressed how well nouveau was working this release)

    sudo zypper ar -f http://download.nvidia.com/opensuse/11.4 nvidia
    sudo zypper in x11-video-nvidiaG02

  • Setup ssh-agent for passwordless SSH logins (take note of bnc#682897)

    echo >~/.kde4/Autostart/ssh-add.sh <<EOF
    #!/bin/sh
    /usr/bin/ssh-add </dev/null
    EOF
    echo >~/.kde4/shutdown/ssh-agent.sh <<EOF
    #!/bin/sh
    /usr/bin/ssh-agent -k
    EOF
    chmod +x ~/.kde4/Autstart/ssh-add.sh ~/.kde4/shutdown/ssh-agent.sh
    

  • Disable 32bit Flash player and install 64bit preview

    sudo zypper al flash-player
    sudo zypper rm -u nspluginwrapper
    cd Downloads
    wget -c http://download.macromedia.com/pub/labs/flashplayer10/flashplayer10_2_p3_64bit_linux_111710.tar.gz
    cd /usr/lib64/browser-plugins
    sudo tar xzf ~/Downloads/flashplayer10_2_p3_64bit_linux_111710.tar.gz
    sudo chown root:root libflashplayer.so
    sudo chmod 755 libflashplayer.so

  • Add Packman Essentials and install needed codecs, and switch to Xine Phonon backend for amarok equaliser.

    sudo zypper ar -f --repo http://packman.inode.at/suse/11.4/Essentials/packman-essentials.repo
    sudo zypper in libxine1-codecs w32codec-all phonon-backend-xine
    Note the .repo file always points to packman.inode.at, no matter which mirror you get it from - you can change to a different packman mirror by editing /etc/zypp/repos.d/packman.repo.

  • Install the Firefox 4.0 theme Oxygen KDE and fiddle with the many settings. Firefox 4.0 is awesomely fast, and looks really slick.


Finally we have this beautiful result!

Monday, 10 January 2011

Winter updates

Phew. long time no news! It's been a busy start of winter ... on that topic a happy new year to everyone! Welcome if you're reading this for the first time on PlanetSUSE, have a look at my intro if you're curious who I am.

A few updates on what I've been working on this month at least.


libopensync-plugin-akonadi 0.22.1 released
Released an update to try and make it work better with the Google Akonadi resource (thanks ares)

Package updates on the OBS
  • Packaged krazy2, the KDE code quality checker, straight from git in KDE:Unstable:Playground
  • Updated get_iplayer, the BBC iPlayer download tool, in home:MasterPatricko. It now works!
  • Packaged powertop2 beta, the power consumption monitor, in home:MasterPatricko; interesting improvements compared to powertop. Needs kernel 2.6.37 (i.e. Factory or Kernel:HEAD) to show off its full power, but still works in 11.3.
  • Update liquidwar6, the particle simulation game, to 0.0.9beta in games
  • Update maxima, the Computer Algebra system, in science to 5.23.0
  • Update unshield, the .CAB file extractor from the SynCE project, in Archiving (and eventually Factory) to 0.6



  • Fix build of denyhosts, the SSH brute-force protection tool, on x86_64



  • Update redshift, the auto screen-brightness adjuster in Factory:Contrib to 0.6



  • Created a package vncserver-autostart in home:MasterPatricko which adds an init script to start a tightvnc vncserver on bootup. Complete with sysconfig-style configuration. Completely insecure of course but perfect for a local network.



Started using my hard-earned openSUSE member benefits; tejas.guruswamy AT and masterpatricko AT opensuse.org are operational; got the blog syndicated on planet.opensuse.org

Plus, as always, I have a couple of new project ideas I'd like to get started on ... more on that soon. Comments welcome as always

Thursday, 23 September 2010

openSUSE Feature - push OBS repository state to users

Yesterday I wrote up a new feature request for openSUSE and the OBS on openFATE. The general idea is to make the Build Service more useful to users by giving them more information about the repositories available.

The major complaint that people using the OBS have is simple. If you search on openSUSE Software Search or webpin you may find five different packages of the software you want; What's the difference between them? Which one is stable? Which one will still be working in a week's time? On the opensuse-kde mailing list, we constantly get asked to explain the difference between KDE:Distro:Stable, KDE:Distro:Factory, and KDE:Unstable:SC - even after pointing them to the wiki pages here and here.

There is even disagreement on what "Stable", "Unstable" etc. means - people ask is it KDE that is Factory - or openSUSE? And then whenever a repository rename occurs, users are left surprised when their zypper refreshes suddenly return errors. And how do you tell users about important changes in repositories like the upcoming shift from KDE SC 4.5 to 4.6 that is about to happen in KDE:Distro:Factory?

So having thought about all this, I wrote up my suggestions on openFATE:310604. Basically I propose a NEWS file, or similar, that gets pushed to users and updated with the metadata, along with a repository stability flag where each state is very clearly defined and visible in software search results. Please vote and comment, and lets get this done for openSUSE 11.4!
Creative Commons Licence
This work by Tejas Guruswamy is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.