May » 2004 » Always Where

A new weblog

It’s always fun when someone you know starts a new weblog. Stuart Kent, formerly at the University of Kent (no relation), now at Microsoft, has started a weblog regarding his work on tools infrastructure.

Reading Microsoft weblogs regarding software design and development practices is a very educational experience. I must admit, I don’t read many of them, but I would imagine that there is a lot of ephemeral knowledge to be gained by tracking a handful of key weblogs. I suspect Stuart’s weblog will be more on the design side rather than implementation, focusing on larger issues regarding DSLs and how to express those ideas in tools. We’ll see.

Hyalophora cecropia

A Cecropia moth (Hyalophora cecropia) has taken up roost outside the window of my father’s office for the last two days.

I don’t think I’ve ever seen an insect that large/colorful “in the wild.” Not that my home town is all that wild, mind you… but still. Nifty.

cecropia

16-bit emulated CPU?

Christian and I are working on an interpreter (written in C) for the Transputer instruction set. We are currently wrestling with porting the run-time to the LEGO Mindstorm. More precisely, we have cross-compiled our interpreter and run it successfully on the LEGO—no question about that—but what we have no way of doing is debugging on the Mindstorm.

When testing on Linux, we can use printf statements, or (more often) we use GDB (the GNU debugger) to explore how our application is running or how it has failed. On the LEGO, we have no output to speak of, and as we are currently debugging platform-specific communication, it is hard to… well, use the limited communication capabilities of the LEGO for expressing what is going on inside of it.

Is this where we’re supposed to buy a development board for the H8/300 series microprocessor and do things the “right” way? Or, is there a software-only solution that would allow us to develop C, cross-compile to an emulated CPU, and test/debug our 16-bit code on that? We’ve had no luck w/ Google finding something that answers this question adequately. It is possible we don’t know what we’re looking for.

Comments or emails on the subject (mcj4 at kent dot ac dot uk) are appreciated.

Scheme takeaway

It’s 11 PM, I’m going to be up most of the night, and I haven’t had dinner yet.

I wrote a Scheme program to place the order for our office.

(define order
  `((veg-rolls 160)
    (veg-rolls 160)
    (spring-pancake-roll 140)
    (crabmeat-sweet-corn-soup 180)
    (special-fried-rice 380)
    (chicken-lemon 450)
    (chick-hot-sauce 440)))

(printf "Cost: ~a~n~n"
        (apply + (map cadr order)))

(printf "Items: ~n~a~n"
        (apply string-append
               (map (lambda (s)
                      (format "~a~n" s))
                    (map symbol->string
                         (map car order)))))

My office mates were… disturbed.

I just realized that my symbol->string is redundant with the use of format for building strings. Ah well. It was a quick hackjob.

WebCT: scriptable?

Perhaps some of the educators/education support types out there will have some insights into this. And, generally speaking, may be interested in the evolution of a summer project I’m now working on.

This summer, I’ll be working on a project to develop Java programming tutorials and supplementary content to support the next run of the course I’ve just finished teaching. The tutorials will be rather “smart,” in that they will interact directly with the pedagogic programming environment “>BlueJ, allowing the tutorial to check the state of the student’s code before progressing. My role is to assemble the software (through development or reuse) and author content. In theory, this system should run in and interact with WebCT.

The details will, no doubt, make themselves more apparent in this space in due time. For the moment, the question I have is this: how “scriptable” is WebCT? Is the backend open to programmers and developers at all, or is it assumed that I’ll interact with the app entirely through the web-based interface?

For example: I would like to auto-generate a large number of fora, and ideally generate RSS feeds off of each one of them. The point is to have a forum for each step in a tutorial, so students making their way through can ask questions in a space that can be explicitly watched (and contextualized) to where they are.

Another example: students are engaged in an online tutorial that is not built using the WebCT quiz framework. I want to record in WebCT that they have completed the tutorial. Can I access the gradebook functionality of WebCT without going through the WWW interaface?

My initial explorations of WebCT tell me it is a very limiting framework that provides very little flexibility in terms of applications deployment. In short, it is designed as a platform for WebCT to deploy its own applications, and for me to use the applications I’m given. It is not intended to be end-user extensible in any meaningful way.

In short, they’re selling support contracts and will provide custom development at a healthy, if not exorbitant fee.

Thoughts? Anyone with substantial (or any?) WebCT development experience is also welcome to drop me a note at m c j 4 at k e n t dot a c dot u k, as I may have one or two more questions. If WebCT won’t meet our needs (if I can make a reasonable case), then I believe we can leave it (mostly) behind as a framework.

Update, 7:39 PM
From the WebCT sales documentation:

Scenario: An instructor wants students to take quizzes within a third-party assessment tool and would like the students to be able to access the quiz tool from WebCT Vista. The instructor wants to avoid having the students log-in separately to the third-party assessment tool and wants the students to be returned to WebCT Vista automatically upon completing the quiz. Lastly, the instructor would like the WebCT Gradebook to be automatically be updated with the student’s grade on the assessment.

This is exactly the scenario I’m looking at constructing. And it looks like you have to upgrade to Vista to have a reasonable application development environment.

I stand by what I had said to someone earlier this week: if you want to build a good educational CMS, start with a good CMS (or other content-driven middleware layer), and build on top of it. WebCT started off crippled, and will probably show signs of that early hamstringing for a long time to come. Unless we have Vista, we’re going to have a hard time developing the software we want to develop.

MovableType Pricing

mt-pricing

All of CS-ED.org no longer works under this pricing scheme. I have

one student,
Cool Stuff in Computer Science,
the main CS-ED.org page,
my weblog, and
course weblogs for a question/answer system I’ve developed

where there are multiple authors involved in some of these cases. I’m already into the $120 price bracket, with little room to grow and/or expand.

My options seem to be to either

stay at version 2.66, or
move to another blogging platform.

For CS-ED.org to work, I need the ability to easily add weblogs and authors, full stop.

I wonder if MovableType will change their position on their pricing scheme. It created a poo-storm in the weblogging community, and they might react. We’ll see.

Filed under: things that make you go… ahwww, bugger!

Useless Java

I have a lot of things to say about this one piece of code. It illustrates so many wonderful things, I can’t even begin to tell you all about them. Part of the reason I can’t go into it is because I’m actually in the middle of teaching a class (we’re working in teams on some problems), and I don’t have time for a full post. The other reason is there are some relatively big ideas here (not new, but relatively big) that I want to spend time on.

So, for future reference:

  int largestValue() {
        Iterator it = ls. iterator();
        int temp = 0
        while (it hasNext()) {
            Integer intTmp =(Integer) it.next:
            int num = intTmp. intValue ();
            if(num >temp); temp = num;{
                return temp;
            }
    }

Ah. It’s sooooo good!

London for two days

I’ll be up in London tomorrow and Thursday for some meetings, the Scheme UK users group meeting, and a day kicking it with a friend.

The question is: if I were to visit a museum on Thursday, which museum should I go to? I haven’t seen the Brancusi yet, which is supposed to be a good exhibition. My favorite to date covered the life works of Bridgit Riley.

Hm. Maybe the V&A? Tate Britain?

XML-RPC for PLT Scheme, version 2.

Documentation (in HTML and PDF), as well as the PLT file.

DrScheme users should be able to select File -> Install PLT file…, paste the url for the URL for the PLT bundle in, and things should ” just work.”

Update, 20040511 (Today) 5:12 PM
I feel a bit guilty releasing this software today; my only reason is because I have someone (apparently) desperately (?) attempting to make use of XML-RPC in PLT Scheme. The reason I’m particularly not keen is that I do not handle exceptions gracefully; faults from the underlying libraries are not handled well. In fact, I have no idea how faults are handled on the server-side of the library at this time.

This isn’t great software engineering practice, I don’t think. The evolution of the second version is

I needed a version of the library that could run under Apache.
I chose reasonable libraries for parsing and extracting data from XML (SSAX and SXPath).
I kept what I could from version one.
I developed version two, running against a small suite of test cases.
I expanded those test cases into a larger suite of unit tests, focusing on covering the internal methods of the library, exploring all the possible methods of failure for each of the components; I do not have complete coverage yet.
I tested version two against two other standard implementations (Apache’s XML-RPC for Java and SOAP::Lite for Perl), adding error-handling code for each error I encountered (but did not systematically search for).

Certainly not Waterfall, and definitely not Extreme Programming. Perhaps closer to “Advanced Hack.”

I don’t expect that I’ll develop my code in a black-box, but I do expect better testing and error handling. Certainly before I release the code for others to use. As it stands, the documentation says nothing about how the library might fail, and under what conditions it will fail.

Am I proud of this? No. Why does that matter? Because it is my software, and other people might build on top of it. I’d rather that my software didn’t cause other people problems. Hell, I know that I’ll build on top of it, and I would rather not discover uglies a year from now that I could have handled when I originally wrote it. But, it’s out now, and I’ll just have to update the documentation and software as quickly as is reasonable to address these problems.

Who should be the judge?

I have, more than once, faced criticism from peers in Computer Science that my research might have a better home… well, elsewhere. To be fair, the sciences have long wrestled with the question of whether the study of pedagogical issues within the domain is a valid avenue of research and reflection, or whether this kind of inquiry belongs “outside” the domain.

Bouncing through a few weblogs after finishing a movie, I found a brilliant bit of navel-gazing-avec-criticism in the writings of Glenn Fleishman. I want to take a moment here to address this, simply because it was an excellent example of how easy it is for people to misunderstand what it means to do research in a humanistic paradigm. Much like some of my close-minded colleagues in Computer Science.

The navel gazing:

For instance, search Google for Maine laptop program, and my post is the fifth result, and third site listed. After posting an item a few weeks ago about the consistent lack of hard data to be coupled with the soft data that the program was collecting, someone involved in the analysis of its success emailed me to take me to task and pointed me to published research.

Unfortunately, in my analysis, the published reports actually demonstrated much more clearly than any of the reportage of the program that they had no hard data on absenteeism, improvement in test scores, or other measures that could be correlated.

First, a high Google rank does not make one right. And secondly, Glenn could not have carried out an analysis unless he

understood the research question being asked,
had access to the original research data, and
had human subjects approval to view that data.

Since none of these things are the case, Glenn has actually gone about asking a completely different set of questions than the researchers were attempting to address. From these questions (which he does not actually explicate), he proceeds to proclaim that the research was inappropriate and inadequate (to answer his questions). This is an easy mistake to make when one is not accustomed to actually dealing with real research data and reports on it.

Glenn’s analysis is less than brilliant. I’ve provided some of his bullet points to save you having to open another browser window:

I’ve read the reports, and I’m still seeing the same problems that I write about before:

Significant, objective, quantitative results are absent from the reports (see below)
Subjective surveys are useful as a touchpoint, but they don’t address the actual performance changes, just the subjective experiences; these must be correlated. If 90% of teachers who report that laptops have made a significant difference in students’ attendance can be correlated to show that yes, in 75% of those 90% reporting, absenteeism was down by a significant factor (5%? 10%), that means something.
Obvious correlation apepars to be missing. Couldn’t a random sampling of student essays be taken from classes in which there are high and low laptop use over the period surveyed and evaluated using standard tools for language skills? If laptop-using classes showed an average improvement in grade level of writing and thinking by a half a grade, or any significant amount, that would be phemonenal.

Point by point, Glenn stumbles into classic misunderstandings of qualitative and quantitative research, as well as a general misunderstanding of the role of statistics in evaluating real-world data:

Glenn wants the results of this work to be
1. significant,
2. objective, and
3. quantitative.
First, it is nearly impossible to obtain objective data in any humanistic regime; clearly, Glenn has a notion that “good” research involves putting people in the equivalent of a Skinner box (rats in cages), where we can control the entire environment around them. In the real world, with real people, we cannot provide objective; instead, research in the social sciences attempts to triangulate one or more qualitative observations to paint a full and robust picture of what was going on.
Glenn’s (desired) hard, numerical data should not to be confused with “subjective surveys … useful as a touchpoint”, which only address “subjective experiences”, which “must be correlated.”
Glenn seems to feel we are dealing with a difference between quantitative data and subjective data; most researchers I know would call this qualitative data, but I have encountered many people who feel the way Glenn does about research that does not involve numbers. Now, while I’m sure 3 out of 4 doctors would agree with Glenn, the truth is that we use different research methods to address different questions. Inappropriately used, numbers are no better than words in attempting to answer a research question.

The word must is also inappropriate; it is not true that we must correlate surveys and interviews with grades (or any other metric) to “prove” or “demonstrate” success. This is simply the easiest way for someone who has been fed bar graphs and pie charts their entire life by FOX News to understand things; it does not make it the right way to analyze the research, no matter what Glenn might wish was the case.
“Obvious correlation apepars to be missing.”
First, there might not be any “obvious correlation.” I’ve spent months digging through my research data; an “obvious correlation” would be like pennies from heaven; I believe there is no such thing. Real data, in the real world, is real messy. Nothing in Glenn’s background indicates he has ever done any (academic) research of any sort, from his undergraduate degree in graphic design on through his career writing columns and books for end-users about how to use drawing packages and desktop publishing software. Real research is not a matter of taking a few numbers and tossing them into Excel; properly done, studying education is a time-consuming, expensive, and tiring process that typically requires hundreds of person-hours to be done well.

I don’t know what this man wants; below is a figure summarizing one response on a survey of middle-school special-ed teachers. It accounts for responses from over two-hundred and ninety-three (293) teachers.

special-ed

There is no lie in this report; it is a report on the perceptions of the teachers. In this report, 269 (of 293) teachers reported the use of the laptops improved their students engagement and interest in the material and instruction they were receiving. Does that necessarily turn up in a student’s final grade? No. We should not expect a student’s grade to necessarily indicate how engaged or interested they were. Furthermore, are we to distrust the judgment of all 269 of these teachers? I’m sorry—they’re all professionals, probably having spent many, many more years being trained to teach and teaching students with special needs than Mr. Fleishman has ever spent in a classroom or doing research of any kind.

Real research is real hard. And it is a sad world where people think that numbers tell the story better than the story itself. Given the scope of the work, any numerical results would be difficult (at best) to trust, given the huge number of schools and diversity of environments involved.

Honestly. I think I’m growing tired of this qualitative vs. quantitative crap.