January » 2005 » Always Where

Sysadmins: It's the people, stupid!

Over at O’Reilly they have an article by a sysadmin who recently did some rescue work using a Knoppix CD. (Knoppix is a distribution of Linux that runs entirely off the CD without touching the hard drive.)

Let’s look at his first paragraph:

As a sysadmin, I wear many hats. Some days I’m the janitor–I clean up discarded files on the file server and clear spam from the mail server. Other days I’m the maintenance man–I make sure all the servers are running smoothly and that any holes have been patched. Some days I’m the architect–I plan, organize, and design systems to suit our needs. Some of my favorite days, however, are the days I put on my rescue hat. When a machine is in trouble, the whistle sounds, I grab my rescue gear, and I run down the beach with my life preserver.

Wonderful! This sysadmin has given me the opportunity to say something I’ve wanted to say for a long time. You see, he’s given us a very common (I think) view of what a system administrator’s job is. And, in a way, this is what sysadmins do. More specifically, it’s the extent of what mediocre sysadmins do.

While at Indiana University Bloomington, I discovered that the Computer Science department there has some of the best systems administrators in the world. Well, I must admit, I’ve only sampled three academic institutions to date (and have word from a number of others through friends and colleagues), and so far noone believes me when I tell them what our systems staff were like.

Practically speaking, the machines never went down. It wasn’t an issue. There was the occasional downtime for the big servers when memory or hardware upgrades took place, but here “occasional” means “yearly” and “scheduled really far in advance.” When machines went down on schedule, downtime was typically arranged to take place early in the morning on a weekend. So the main departmental fileserver would go offline on (say) a Saturday at 7 AM. For an hour.

When something catastrophic happened, things were fixed fast; a pager number was publicly posted for all to use in the event of systems going down. I called once on a weekend when an NFS handle went stale. Occasionally, someone would let a process run away on the webserver, killing things for all other users (OK, OK, that was me). These were often remedied in minutes after the call was made. Likewise, when someone discovered they had deleted a file or email they shouldn’t have, file restores were handled in minutes or, at worst, hours.

The largest failure I recall was on the undergraduate fileserver. It lost a drive in the RAID, which was replaced. Noone noticed. Not, anyway, until a second drive failed before the new drive had finished being introduced into the RAID. RAID5 dies completely if it looses two drives. As it turns out, it was a bad controller, which in turn was destroying the drives—hard to diagnose. But, they had the parts on hand to rebuild the array (that’s two spare drives and a controller in place), and we lost something like 5 hours of data (between midnight and 5 AM) on a subset of our machines. In otherwords, the losses were minimal, and the machine was back up after a complete restore of over a million files from backup—a number of hours, yes, and our admins were there through the night to see that things were back up as soon as possible. (We were discouraged from using the pager for backup requests, mind you.)

But do you know what? The integrity of the machines was secondary. The systems group was, first and foremost, a group committed to supporting the productivity of the faculty, staff, and students in the department. Their commitment was to the people, and the smooth-running operation of the systems was just part of how they manifested that commitment.

Software requests were promptly handled, and often pro-active. “I’ve installed both the stable and unstable versions, as it looks like the unstable tree had the features you need; they’re in /l/blahblah-(stable|unstable)” might be a common reply to a request for an application or set of libraries. Once, when I found that my BerkeleyDB files were no longer accessible, not only were the apps for the older files restored to a part of the tree, but the admins (quickly) investigated likely problems, and were able to give me a direct solution: “Run x on your databases, and it should bring them up to the current version; I tried it on y, and it looks like it does the trick.” I didn’t know why my CGI application couldn’t access the DB files; they not only discovered why, but proactively found the tools I needed to get my app up-and-running again.

So when I encounter sysadmins who are chuffed about how great their systems run, I think I’ve met a sysadmin who doesn’t really understand what it takes to be great at what they do. Why? Because when the systems are running smoothly, it’s not time to fire up a video game and play some Civ or Counterstrike all afternoon; that’s when you start finding out what your users are doing, what their needs are, and how you can best support those needs.

And that’s what lets your users not just get their jobs done, but do really cool stuff and really great things.

Philosophy of Teaching: A Conflict?

My philosophy of teaching is concise. I describe where I first encountered (and still fondly recall) the kinds of education I most enjoyed, and how my own explorations and formal studies have led me to the believe that education is most effective when it is constructive, authentic, and fun. This is in keeping with the work I began at Indiana in Introduction to LEGO Robotics (TeamStorms), and continued at Kent with CSCS.

However, my teaching on the IT Certificate this last year has challenged this in some ways. Or, my experiences teaching non-traditional students (largely adults coming back to education) have also dropped me at the other end of the educational spectrum, where my statement of research lies. That is, I believe that we do not understand the fundamental mechanisms by which students learn to read code—or, if it is analogous to how students learn to read natural languages, those connections have not been empirically made. This has manifested itself in my own classrooms through the BlueJ Trails project, where we’ve attempted to provide some measure of “drill-and-practice” learning where students can hone the basics of syntax and, in some cases, semantics, before we attempt to tackle larger creative and design issues in programming.

I don’t know if this represents a conflict or not? Personally, I don’t see it as one, as I believe that there is no such thing as one right way to teach. I prefer to provide my students with constructivist learning environments where they engage in as close to a real-world activity as possible—and have fun learning in the process. However, I also know that different students and situations merit different approaches, depending on the desired outcome.

So I don’t think I see it as a conflict. It did give me pause, though, while working through some positivist/utilitarian critiques of educational research from the past half-century. And mercifully, my post is not nearly as detailed as the critiques.

PDFpen: Cool

I decided to pick up a Graphire3 tablet; it’s a toy. But, it’s a cool toy, and it’s already changing the way I do some things.

I also purchased PDFpen; this app is simple and sweet. It lets me open a PDF, write on it freehand, and resave the document as a PDF. It also lets me do a few other things as well; you can see for yourself.

All day I’ve been going through articles on grounded theory and evidence-based research, and normally this is process involves printing an article, reading + making notes on it, and then wondering how I’m going to keep track of my notes. Often, I end up writing up some annotations after a couple of articles, ending up with an annotated bibliography. However, this looses a lot of the context of the note.

Today, I’ve been reading on-screen; perhaps less good for the eyes, but better for the environment. And, with the pen and PDFpen, I’ve been able to underline pieces of text, circle things that matter, and even write notes in the margins; I can flip the page 90 degrees, write something, and flip it back without any difficulty.

The result?

I have the original article, but with the parts I think matter marked. I can even insert blank pages directly into the PDF and type or hand-write more extensive notes if necessary. And when I’m done, I still have one file, with all my notes and the original source.

Now, I know that XML/RDF and all kinds of other Semantic Web stuff is going to make these kinds of things magically happen, but the truth is, you can’t separate information from it’s context and have it still be usable. In this case, the papers I download and read, and the notes I make on them, remain portable and usable to me.

It’s a step, anyway. I’m fairly confident I like it. Time will tell, but it certainly seems to have been a productive day, and the biggest difference between printing and making notes and making notes on-screen is that I don’t have to archive and truck the paper around; instead, I have a searchable (come on OSX 10.4!) version of my papers that I can archive and catalogue in any number of ways.

Nifty.

Concurrency in the media

I’ve noticed more and more mainstream publications making a reference to the slowing of Moore’s law and the increasing trends towards concurrency. I’d like to have a list of a dozen of these references from the last year or two (if I can dig up that many) before heading to SIGCSE in February. It would make some nice motivational material for the work we’ve been doing with the Transterpreter.

I’ve killed comments on the site; I’ll probably just edit this post as I go digging for more. Drop a note to mcj4 at kent plbbt ac plbbt uk if I miss any.

List as of Jan 7, 2005

The Free Lunch is Over: A fundamental Turn Toward Concurrency in Software
Herb Sutter, to appear in Dr. Dobb’s Journal, 30(3), March 2005, and a short version titled “The Concurrency Revolution” in C/C++ Users Journal, 23(2), February 2005.

There was an IEEE publication w/ an interview w/ Sir Professor Tony Hoare, and another that had quotes from Intel execs saying they were going to produce “slower” and “more parallel” chips in the near future. I thought there was an IBM Developer Works article as well.

It’s something to keep digging for.

Applications

Over the last few months I have been gathering up material for my CV, thinking about a statement of research and philosophy of teaching, and it is now available to the public (Bio and CV, right).

This evening, I sent out my first application for employment after the PhD. It feels like it means so many things, while meaning nothing at all. I have committed to being done before the end of the summer—to potential future colleagues as well as myself. That alone is something; I’ve been in school for quite a long time. The end is near.

Professor David Wise would say to me over and over “Keep your eyes on the prize.” When he first said this to me, I wasn’t ready to hear it—because I wasn’t even a PhD student yet. Or, I didn’t think I was; he clearly saw things I didn’t.

If I had developed a mantra for this last year, it would have been “keep your eye on the prize.” I don’t know if I’ve always done a good job of adhering to it, but when the end comes into sight, it’s hard not to get excited. Well, excited and scared. But it does give me something to work for and look forward to.

Photobooks and OmniGraffle

I may have found the application that solves my photobook problem: OmniGraffle. The solution was, interestingly, discovered accidentally a few weeks ago. It happened because I tried to drag-and-drop of an image from the Finder into OmniGraffle, and guess what: it worked.

That surprised me. What surprised me more was my poorly aimed drop put the image directly into a shape I already had on the canvas. What I didn’t expect was that the shape was automatically filled by the image. This didn’t really matter to me at the time, but it’s come back with a vengance now that I’m trying to lay out attractive photobooks for publishing on lulu.com (see yesterday’s post).

Let’s look at what my “requirements” are for a quick/good solution to laying out photobooks. The solution…

MUST be simple; my mother should be able to use it.
MUST be fast; I don’t want to take lots of time on this, I just want my photos printed, in a book.
SHOULD be flexible; I don’t want to be stuck with only three page layouts for all time.
SHOULD be automatable via a script; I don’t require this at the moment, but I’d like to be able to use a markup language in the future to generate large photobooks quickly and/or from a database of images and text.

Yes, I know iPhoto does photobooks, but they want (something like) $30 setup and $3 per page beyond 10. Lulu, on the other hand, is offering a $4.53 setup, and $0.15/page fee for each full-color, 8.5″x11″ page. (They’re asking $0.02 per black-and-white page.) Once my photobook is printed, Lulu provides an instant storefront from which I, my parents, and friends can buy copies of the book. This means that when I put together a book of my three years living in Canterbury, friends there can purchase their own copies, as that book will represent shared memories, not just my own. Yes, they could get copies of the digital pictures, but then what? If you think about it, this really is a simple, attractive, and powerful solution to sharing photos in print.

So, let’s look at how I’ve used OmniGraffle, and how each of my needs are addressed. I think it’s an interesting exercise in understanding one’s tools, and how the KISS philosophy really does work.

Can my mother use it?
At the moment, the solution involves a folder full of pictures and a running copy of OG. She needs to drag-and-drop her images into some shapes. (My mother is actually becoming quite savvy with the Mac, so I’m confident she can manage this solution.)
Is it fast?
Quite, and it could be faster. It’s almost keyboard-free, but quite quick just the same. In pictures, my solution looks like this:
1. I create a new canvas in my document
2. I select a layout from my scripts menu (we’ll see where those come from). Here, I’ve chosen a layout for two photos that are horizontal, not vertical.
3. I drag my images into the template.
Overall, this meets my “fast” criterion.

Is it flexible?

This is the best part (from my point of view). Not only can I easily add new templates, I have a solution by which my mother could add new templates. That’s cool.

Creating layouts could be tedious; however, OmniGraffle is AppleScriptable. This means you can write programs to tell it what to do. So, I created programs that create the layouts; for example, the script to create the layout in the above example looks like:

tell front document of application "OmniGraffle Professional"
  tell the last page
    make new shape with properties {name:"RoundedRectangle", Â¬
                                    origin:{102.666, 398}, Â¬
                                    size:{370.667, 278}, Â¬
                                    rotation:0}
    make new shape with properties {name:"RoundedRectangle", Â¬
                                    origin:{103.666, 54}, Â¬
                                    size:{370.667, 278}, Â¬
                                    rotation:0}
  end tell
end tell

Where did those numbers come from? How the hell did I know where to put those rectangles? The secret: I didn’t.

OmniGraffle saves it’s documents in an XML format; this means a lot of things, and I won’t go into them here and now. What it means for me, though, is I can write a program that:

Takes in an OmniGraffle document (by dragging-and-dropping)
Processes it in technical ways
Outputs the AppleScript you see above

So all of my templates are created by laying them out in OmniGraffle, running the program I wrote on them, and then moving the script it outputs into my OmniGraffle folder. Then, I have an AppleScript that will recreate all the objects I had in my OmniGraffle document.

Or, put another way, to create a new template, all I have to do is lay out some shapes in OG, save it, and run my program. Instantly, I have a new template for my photobooks pages!

(Come to think of it, I could put all of this in an AppleScript of it’s own. That way, I could have a script in the OG script menu called “Create New Template.” All in good time, I suppose.)

I think my solution meets my criteria. That, and it’s relatively simple, and fairly robust.

Once I’m done with my photobook, I choose “Print” and “Save as PDF.” The question is, will it work? I’ve sent my PDF off to the printer, and if the fancy-schmancy digital printer they use doesn’t bork on the PDF (that’s a technical term), I’ll post my software that does the conversion of an OmniGraffle document to an AppleScript. Perhaps someone else will find it useful.

EDIT:

I just received my first photobook from lulu.com. It’s nice; it’s not the same quality as a photographic print, and is probably lower in quality than an iPhoto book. However, I now have 110 pages of pictures from a trip Carrie and I took to Greece in one place, and the total cost was $26 (with shipping). That’s hard to beat with a stick.

My problems with PDF generation so far must be due to bad font embedding. The book that I just received was produced using TeXShop and Gerben Wierda’s installation of teTeX for OS X. The book that bombed was produced using a tiny teTeX installation that, most likely, didn’t have fonts in the right places.

My suspicion is now that everything is going to work just fine; that is, I have confidence that the “Save as PDF” function in OS X produces a “good” PDF, and that I shouldn’t have any problems with generating photobooks as I describe above. Time will tell, but I have faith.

Digital Photos and Photobooks

I have a lot of digital photos. Not as many as some, but enough that it’s tough to share them with others. In particular, it’s hard to share them with people who don’t have high-speed Internet connections.

You see, I quickly adapted to the digital philosophy—take lots of pictures; some will turn out. But on the flip side, I don’t want to pay $0.20 (round-number estimate of all charges involved) per picture to get hard copies printed. Besides, then I have a stack of 4″x6″ prints to deal with.

Enter the photobook. In the age of digital, print-on-demand services, this is the only way to go. My photos can be arranged in one or more attractive layouts, I can annotate the images, and they’re always in one place. Furthermore, I can easily share a photobook. If I make up one book of photos of me and my friends over the past three years, we can all purchase one from a print-on-demand supplier.

iPhoto allows end-users to easily assemble their content into an attractive, hard-bound photobook. The providers of this service also charge $30 for setup and $3 for each page past 10. The interface is good; my (ideal) needs are:

The ability to choose from one or more standard layouts
Drag-and-drop images into that layout
Export to PDF and/or upload direct for publishing

The ability to create new layouts would be nice as well.

However, I don’t want to pay iPhoto prices. I have found other print-on-demand suppliers who are less expensive (lulu.com), but currently my PDFs are failing when they go to print. The program I wrote to create photobooks lets me write something like

(book
  photo1.jpg
  (two photo2.jpg photo3.jpg))

which would give me a two page photobook (PDF)–the first page with a single picture, and the second page with two pictures. The markup language gets converted to LaTeX, and then converted to PDF. These PDFs, however, are not “clean” enough for the printer; personally, I suspect a bad (or absent) embedding of CMR, a common font in the TeX world.

I may figure my problem out. I may not. I don’t really want to take time on this right now. I’d rather pay $50 for a tool that lets me lay out my photobook quickly, easily, and visually, and then I can dump it to PDF and submit it to a photobook maker of my choice.

At the end of the day, I just want a book of my pictures that I can share with my family. Really. That’s it. And I’d rather not screw around with prints.

Currently, lulu.com is the most affordable print-on-demand service I’ve found. I hope they make it in what must be a competitive space; they’re offering a (potentially) great product at a great price. Now, if I can just get my PDFs cleaned up.

Print on Demand with lulu.com

I just created and published my own photobook in 30 minutes.

I haven’t seen the product yet, but when it arrives, I’ll post pictures.

The day before yesterday I discovered lulu.com, a print-on-demand website that is insanely easy to use. I created an account, create a PDF of my content, uploaded it, whipped up a quick cover, and BLAM I was able to purchase a printed version of my very own book. I just have to wait 3-4 days for production, and then shipping (another 2-3 days). By the end of this next week I should have my book.

The reason I think this is so exciting is because I just assembled a 112 page, full-color photobook for $20. While I can order photobooks directly from iPhoto on my Mac, I don’t want to pay $30 for setup and $3 per page for every page past 10. I want a photobook with several hundred pictures in it, and I want it to be affordable. The book I created is spiral-bound with full CYMK color throughout on #50 white paper. Each page contains one or more photos from the trip Carrie and I took to Skiathos; if things turn out well, I’ll make the tools available that I used for creating the book, as they are (I think) quite usable.

What is great is that setup costs $4.53, and Lulu has a per-page cost of $0.15 for full-color printing ($.02 for black and white). Compared to getting photos printed online or at a photo kiosk (where a single 4×6 print might cost me $.25), in a photobook I can print two 4×6 pictures on a US Letter page for 15 cents. Furthermore, if I want to share the book with my friends, they can purchase their own copies—or I can purchase multiple copies and give them as gifts. At $20 for a 112 page book (around 200 pictures or so), I’m just amazed.

I feel like I’ve got diarrhea of the blog here; however, the site makes publishing content so easy they’re like the MovableType of the print-on-demand world. It really is way, way to easy.

(If things go well, you may find me ranting about how cool PDFPen is. It looks like a sweet little Mac app that lets me doodle on PDFs; can you say “Annotate pictures in the photobook”?)