For the last year I have been hanging out on a local Windows user group mailing list just to see how the other half lives. I have been on the local Linux user group mailing list for at least 12 years. The difference is night and day. The Windows user group mailing list has invites to various free lunches paid for by vendors who want to you to buy their stuff and the occasional chatter about some problem with SQL Server or IIS or something. When someone complains about Windows a rep from Microsoft shows up on the list to handle the situation. The Linux group is full of passionate discussion and debate about all manner of technical and political issues plus the usual advice giving and newbie helping. The Linux people are clearly in it for the love of it and the Windows users just seem to be there for a paycheck. Just look at the times people post to the lists. The local Linux folk are posting day and night and weekends. The Windows people are mostly 9 to 5'ers Monday through Friday. The Windows group meets at the local Microsoft office and watch a vendor presentation and then go home. The Linux group meets at a local school, put on their own presentation from 7 to 9 pm every second Thursday and then a dozen or so of them go to an after-meeting meeting at a nearby Denny's restaurant for conversation until midnight. If any of you are in the San Diego area check out http://kernel-panic.org/

And on top of it all, tonight I hacked a new feature into Kudzu (detecting AoE disks so that Anaconda can install RedHat/CentOS onto them which is something I *really* need) made possible by Free Software and some help from my LUG friends so I'm pretty darn happy with the GNU/Linux community right now. I'll be sending that code to RedHat for sure because that's how I give back and we as a community keep improving our Free Software.

 geeks vs 9-5'ers
 by TechGeek (3.49) on Mon 14th Jan 2008 03:07 UTC
 TechGeek
 Member since:
 2006-01-14
 
 Working at a university I see a lot of trends that people talk
 about. Its easy to spot these when you have seen thousands of
 students cycle through your program. The attitude that Open Source
 users are more capable looks like its true on the surface. But its
 just an illusion. The real problem is that there are two types of IT
 people, as I like to label them, the geeks and the 9to5'ers.
 
 We have students who just went in to IT because they thought it
 would be a good career. These are the 9to5'ers. They do the minimum
 learning to skate by. And nothing is easier to look good at running
 than Windows. Thats not to say it IS that easy to do well, but with
 a little knowledge its pretty easy to look like you know what your
 doing. These people don't touch linux or unix because it requires a
 lot more effort to learn it to any degree of proficiency.

SCO has lost their epic legal battle. The judge ruled against them. Time to party! No more FUDing about Linux. At least not that same old FUD which we have had to tolerate for the last 4 years. And hopefully MS has learned a lesson from SCO's foolishness. Once again I feel vindicated.

Here is an excellent review of the $100 laptop of the One Laptop Per Child laptop project which is a small laptop designed to be used as an educational tool for children in poor third world countries who already have food and a roof over their heads but still lack education.

An Open Letter To Everyone: I know JUST how this guy feels. Why can't I ever get free legal or car work done? From the article:

So you're having a problem with your computer, eh? Yes, you're right, I do know something about computers and yes, I was a computer science major. Will I fix your problem? I'll certainly try. I'm more than happy to help you to the best of my ability. You're a friend and I'm glad I can help you. I know you'd do the same for me and in all likelihood, I'll be hitting you up for free financial/legal/medical/automotive advice in the near future. I happen to have an area of expertise and I genuinely don't mind helping you out.

All of this combined with Dell shipping Linux pre-installed on their hardware and report having received 30,000 requests for Linux pre-installed on their machines makes this a pretty good week. People are starting to get it!

I haven't really done much Linux advocacy in recent years since Linux really doesn't need nearly as much as it used to. Linux is mainstream, very successful, etc. Companies with any significant computer infrastructure that don't use Linux are becoming rare. Of course, most of them still make the right choice for the wrong reason but at least they are choosing Linux. But every now and then I still have to step in and advocate for Linux on the desktop. Most of the practical problems with it have gone away leaving us with the perceptual and habitual issues. And those will be even harder to solve and you won't win those arguments on technical grounds. There has got to be another way.

I was reading the Fedora mailing list today and came across this excellent comparison:

 The difference between Windows and Linux is like the
 difference between a whore and a long term relationship.
 It needs some time, it does not allow everything and is
 not always easy, but it is worth it. - Adalbert Prokop

How true!

To which someone else replied:

 I like your comparison and your explanation. A person
 that I knew divorced after 20 years of marriage
 because the wife went with another man. This person
 went to a place and met a whore and got married with
 her. He says that she is the best women a man can
 have. She does not have the urge for other men because
 she has been there, done that. She is faithful and
 treats him like a king. However, this is the
 exception and not the rule.

 The way you state it is the way it is, you stick with
 linux and in the end, you will be better off. It will
 take time, but it is a good decision.

 Regards,

 Antonio

Very interesting point of view, no?

And to look at the long term relationship aspects of it: I have been using Linux for over 13 years now. And everything I learned back then still works today. The tools work the same, the environment works the same, the fundamentals are unchanged. The time spent learning it has been an investment which now pays dividends every time I sit down in front of my computer. Some important capabilities have been added since then that make things much better but they are all pretty much invisible to the desktop user. From a desktop point of view things are a bit prettier and we can read/write Word files etc. Not a big difference. The biggest differences are hidden from the user in the form of better driver support, more auto-configuration of hardware, etc. In fact most of the improvements in Linux as a desktop in recent years seem to have come in the form of getting most of that stuff out of the users face.

On the other hand, Windows users have needed retraining on how many different revs of software since then? And they have had to pay again every time. Windows techs have to get re-certified every time too. It is a never-ending upgrade treadmill. Have fun with that.

#!/usr/bin/python
"""

Get list of full volumes from the bacula database, upload the volumes
to S3, delete the volumes from the local system.

Tracy Reed
March 15, 2007

"""

import mimetypes
import os.path
import sys
import MySQLdb
import bitbucket
import time
import fcntl

AWS_ACCESS_KEY_ID = 'myaccesskey'
AWS_SECRET_ACCESS_KEY = 'mysecretkey'
BUCKET_NAME = 'mybucket'
BACKUP_DIR = '/home/treed/media/backups'

def daemonlock(progname, timeout=2):

    """Test lock for daemons to prevent multiple instances
 
    @param progname: The name of the caller. This name is used for the lock and
        should be unique on the system. Alternatively, this may be an absolute
        path for the lock file, which is useful for programs that do not run
        as root.
    @param timeout: Seconds to wait for the lock.
    @return: 
         0 - success
        -1 - another daemon is still active
        -2 - some other error
 
    The lock will be released when the caller terminates. No unlock is
    required.

    This is much slicker than the old way of writing our pid out to a
    file and then reading in that file and parsing ps output to see if
    that pid still exists.
    """
    
    if progname.startswith(os.path.sep):
        lockfile = progname
    else:
        lockfile = "/var/run/%s.pid" % (progname.replace(os.path.sep, "_"))
 
    try: 
        fd = os.open(lockfile, os.O_CREAT | os.O_RDWR, 0644)
    except OSError, e:
        print e
        return -2
 
    timed_out = True              # Assume failure
    t_end = time.time() + timeout
    while time.time() < t_end:
        try:
            fcntl.lockf(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
            timed_out = False
            break
        except IOError, e:
            pass
        time.sleep(0.3)         # Wait a little to minimise CPU load
        
    if timed_out:
        return -1
    else:
        # Got lock, write the program's current pid
        os.write(fd, "%d\n" % (os.getpid()))
        return 0

def get_full_volumes():

    """ Connect to the database and get a list of all of the full
    volumes. Return the list. """
    
    db = MySQLdb.connect(host="localhost", user="root", passwd="mypassword", db="bacula")
    cursor = db.cursor()
    cursor.execute("select VolumeName from Media where VolStatus='Full'")
    result = cursor.fetchall()
    return result

def upload_volumes(full_volumes):

    """ Use bitbucket to establish a connection to S3. Iterate over
    full_volumes and upload each one.  Bitbucket is cool in that it
    makes the files to upload keys in a dictionary. The bucket is the
    dictionary, the keys are the files, the values associated with the
    keys are the contents. So to put data into file on S3 we just
    assign the data to a key and it gets uploaded. It automatically
    checks if the contents are the same using md5sum in the
    implementation of the bucket object and if they are the same
    already it does not upload the data.  """

    connection = bitbucket.connect(access_key=AWS_ACCESS_KEY_ID, secret_key=AWS_SECRET_ACCESS_KEY)
    bucket = connection.get_bucket(BUCKET_NAME)
    for record in full_volumes:
        file=("%s/%s" % (BACKUP_DIR, record[0]))
        print file
        if os.path.exists(file):

            # This while causes us to loop which retries the
            # upload. We increment attempts each time through and if
            # attempts is less than 3 we raise ValueError which gets
            # caught with a pass which causes us to hit the bottom of
            # the loop so we start over. If attempts is 3 or greater
            # we don't raise ValueError which causes us to hit the
            # else clause which contains a break which gets us out of
            # the loop.

            attempts = 0
            while 1:
                # This try for implementing the retry mechanism
                try:

                    # This try for ignoring empty files
                    try:
                        if bucket.has_key(record[0]):
                            bits = bucket[record[0]]
                            bits.filename = file
                        else:
                            bits = bitbucket.Bits(filename=file)
                            # Here's where we assign bits (read in
                            # from the file) to record[0] (the
                            # filename in S3) to a key in the bucket.
                            bucket[record[0]] = bits
                        os.unlink("%s/%s" % (BACKUP_DIR, record[0]))
                    except bitbucket.BitBucketEmptyError:
                        print 'sync_dir: Empty File - Ignored %s' % fullpath
                    attempts = attempts + 1
                    if attempts < 3:
                        raise ValueError

                    
                except ValueError:
                    print "Retrying...Retry number ", attempts+1
                    pass # retry
                else:
                    break # done


if __name__ == "__main__":

    # Check our lockfile and see if another copy is already running
    if daemonlock("s3-upload") != 0:
        sys.exit()
    # Connect to the bacula db and get our list of full volumes to
    # upload
    full_volumes = get_full_volumes()
    # Do the uploads to S3
    upload_volumes(full_volumes)

It passed in the US with hardly a mention but for the rest of the world September 16th was "Software Freedom Day" and events were held all over to celebrate and promote Free Software.

http://www.softwarefreedomday.org/

This year I helped out LinuxBaja (formerly known as TJLUG, they merged) in whatever way I could with their event and served as a "Wikiman" which is basically a person available to answer questions about free software. Linux in Mexico is spreading and there are a number of businesses there who use it but it is usually a secret of the guys who run the servers. I would estimate around 250 people showed up and enjoyed 6 hours of presentations and demos. You can find Vladimir's writeup of the event here along with pictures:

http://floss.linuxbaja.org/index.php?option=com_content&task=view&id=63&Itemid=75

Today there was an article about how the US Department of Defense (that's the military) is sometimes afraid to use "Open Source" software which is what some people call Linux:

DoD Wary of That "Open" Word

They say that some military types do not like the word "open" because it sounds insecure and dangerous. So I replied that maybe we should be using the phrase "Free Software" to describe what Linux is which was the original term for it until maybe 7 years ago when a guy named Eric Raymond made the phrase "Open Source" popular. Many people really liked my comment and gave it a very high score. Many people have replied to it. But the best reply so far is this one:

"Free Software" intentionally invokes Cold War

He talks about Free Software, freedom, the Cold War, communism, etc. Very interesting discussion and ideas.

I think a lot of people misunderstand ReiserFS and filesystems in general. ReiserFS (3 and 4) acknowledges the fact that cpu is very fast and disk IO is slow. If you can do anything at all in cpu as far as calculations or optimizations to avoid having to make disk accesses it is a win. This is why ReiserFS takes more cpu. Overall it should be faster. It also assumes that your hardware is reliable. If your hardware is bogus you are going to have problems with any fs but particularly ReiserFS. The on disk and in-memory data structures are much more complicated than ext2/3/4. All designed to provide better performance. If you have a memory problem or disk controller problem or really any hardware problem at all you are in deep shit. Want good performance and data integrity? Use quality hardware and implement redundancy!

Journalled filesystems like ReiserFS easily handle power-out problems, accidental reboots, etc. These are not data corruption issues. But once some bogus piece of hardware starts causing random bits to be scribbled to the disk all bets are off. I don't even see the lack of an fsck program as a problem. If you ever get to the point where you need to do an fsck you really should just restore from backup. When I hear these stories about how people lost all of their data because their filesystem "crashed" I have two reactions: 1. Skepticism that they didn't have bogus hardware or didn't somehow screw themselves up. It is extremely rare that anyone can actually prove it was a bug in the fs that burned them. 2. Total lack of sympathy because they didn't have a backup.

Here's what I do:

I value my data so I spent an extra $100 to get another 250G disk and I mirror. $100 is DIRT CHEAP insurance against hard drive related failures. Disks are so cheap and big there is no excuse for not mirroring important data. Plus you get a bonus on read performance. If I offered you $100 to let me delete 250G of data from your machine right now would you let me? Then your data is worth more than $100 also and worthy of a mirrored disk. But a mirrored disk is not a backup. You need backups too.

I have Bacula setup to run every night. It makes a backup of my data to an external USB2 attached 80G drive. I don't back up all of my data as there is some stuff I really don't care about. But all of my email, source code, and vacation photos etc get backed up every night. I probably have 30G of data I really give a care about. I have two of these drives. I do a full backup once a month and incrementals every night after. At the end of the month I take the drive over to my storage unit (or a friends house would do, or even my desk at work) and swap it with a second drive which I have stashed there.

I think I paid around $80 for each of the external drives plus $100 for the extra disk for the mirror. So I have a really great, fast, reliable backup solution for $260 plus some time to set it up. Is it worth it? HELL YES! While writing this I just thought to do a test restore of some data. It worked. Yeay! My backup is solid and there if I need it.

If any one of you offered me...say, $1000 to come over to my house in San Diego right now to boot your own super-destructo CD which did a military grade erase of my HD's I would let you. RIGHT NOW. I have the data backed up. I figure my time to do the restore is worth $1k to me. And I'll have everything back up in 24 hours or less. If you can't do the same right now your data better not be important to you because that's how disasters happen: Completely unannounced.

Remember kids: If it wasn't backed up it wasn't important!

A couple of nights ago I gave a presentation on Xen at the Kernel Panic Linux User Group meeting. There were 30 people in attendance. I won 4 books in the raffle (which I need to write reviews on). A book on podcasting, one about RFID, and a set of two Samba books. Aside from the projector not working so I couldn't show anyone anything it went pretty well. I temporarily moved my server to the meeting site to demo a real working Xen system. My new server is an AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ with 2G of RAM and 250G of SATA disk. I gave away or sold several other servers I had laying around and consolidated everything onto this one box and so far it is working out great. Notes from my presentation can be found here. My Xen box runs my personal workstation, webserver, mailserver, and firewall all in completely separate virtualized environments each with its own complete instance of the Linux OS called a "domain" in Xen terminology.  I may set up a new domain with M0n0wall to be the firewall in the near future. Someday I hope to set up some AoE disk and play with Xen's process migration.