The Agile Hacker Blog

by Bill Burris
www.componentsnotebook.com
wburris@telusplanet.net


This is my web log for documenting the journey along the road to becoming an Agile Hacker.  The idea is to combine techniques from Hacking, Agile Development and Cognitive Psychology.  See my Agile Hacker page which documents my current thinking about what it takes to be an Agile Hacker.  I have no qualifications for teaching Hacking, Agile Programming or Cognitive Psychology, but I am beginning the journey to see what I can learn from those three fields. The focus here will be on the software development tools and techniques.

Here is an index to my Blog, with most recent at the top.  Below the Blog entries are in the order that they were written, newest at the bottom.  All entries will be subject to extensions, revisions, and clarification as the need arises.

June 26, 2004 Moving to Open Source
June 6, 2004   -  
Compiling and using CppUnit on Windows
June 4-6, 2004   -   Linux for Windows users


June 4-6, 2004   -   Linux for Windows users


It is a good idea to document what you are doing so that you can repeat or refine the steps at a later time.  Since I never did that when playing around with Linux, I will have to go from memory here.

To fully immerse yourself in the Hacker culture you need to use Linux.  Most open source code is written for Unix and/or Linux.  If you are working on Windows you will be severely limited in what open source code you can use or play with.  If you have only used Windows, don't fret, you can make a start there, but plan to migrate over to Linux as soon as you feel adventurous.  In fact many of my initial log entries will be documenting techniques on Windows.

Make sure you understand what I am telling you to do before trying it out yourself.  Following these instructions blindly will likely result in loss of data or render you computer unusable.

Have you ever installed an Operating System?  If you have never done this, give it a try.  Start with your Windows CD and an old useless computer.  Since you are working with an old surpluss computer, you can safely erase the hard drive, losing all its current contents.  At he beginning of the install process, delete all existing partitions.  Create a new partition using all the space available on the hard drive.  This process works best with Win 2000 or XP, since you can boot off the CD.  For older versions of Windows you might have to create a boot floppy.  I have successfully installed Win 2000 on a 200 MHz Pentium with 64 Meg of RAM.  I used that configuration as my connection sharing machine for 3 years.

Whenever installing Windows, keep the machine disconnected from the network or behind a firewall.  If you connect directly to the network before installing all the critical updates, you are sure to be infected by a worm.  If you have a DSL or Cable connection, get a router, even if you only have one computer.  Having a router between your computer and the Internet, will typically give you enough time to install all the critical updates without getting infected.

If you don't have a surpluss computer, here are some things you can try with your main machine.  You don't want to do this if you have important work which needs done in the near future.  If you screw up you could loose all your important files or damage your hardware.  Back up your important files.  Buy a new hard drive.  Take your working hard drive out of the computer and put it on the shelf.  Install the new hard drive.  Proceed with doing a test system install on the blank hard disk.  Once you are done playing around with system installs, remove the new drive and put your old one back in.  You should be back to normal.

When swapping hard drives around, make sure you have the jumpers on the drives set correctly.  When you have two drives on the same cable, one should be set as master and the other as slave.  This include CD drives as well.  The master on the first IDE connection is usually the one that gets booted on system startup.

If you have more then one dead computer, start swapping parts around, to see if you can create a working system.  If you are going to spend a lot of time using computers, its a good idea to become familiar with the major components and be able to keep you system running and do your own hardware upgrades.

When purchasing new computers, make sure you are buying generic hardware that you can repair and upgrade yourself.  I only buy from stores that let me specify each component and have the system custom built for me.  I often purchase motherboards, hard drives, and memory to upgrade systems myself.

If you take the time to learn how to juggle hardware components and install the OS, you can greatly extend the life of your system.  You will also have a better understanding of how the system works.

Windows systems rot, and the only way to rejuvenate these systems is to erase the hard drive and re-install everything.  Windows XP and some versions of Office need activated.  If you erase your hard drive and re-install you may have problems re-activating your software.  I have re-activated Windows XP several times on the same hardware without problems.  With an MSDN version of Office XP, I have run into problems re-activating after a clean install.

If you have no further need for Windows you can skip all that practicing with installing Windows and start with Linux.

If you have an MSDN subscription, download Virtual PC and give it a try.  I installed Fedora Linux in Virtual PC.  On a 1 G Hz Athlon with 500 M RAM, Fedora Linux running in Virtual PC is rather slow.  On a 1.9 G Hz Athlon with 1G RAM, Fedora Linux running in Virtual PC works well enough for playing around to get some experience with using Linux.

I tried to install Fedora Linux on a 200 MHz Pentium with 64 M of RAM.  The install program hung part way through.  I don't know if this was due to defective RAM or just not enough of it.  The reason I bought a router to replace this computer as my connection sharing system, was that it was becoming flaky and required booting occasionally.  After replacing the 2 32 M modules with a single 128 M module I was able to install Fedora Linux and use it.  The performance was worse then running it in Virtual PC on the Athlons.  After several days trying to update the system using Red Hat update the system died and would no longer boot.  I was also unable to get Red Hat Update to work reliable with Fedora running in Virtual PC.

I decided to try Ximian Red Carpet to see if it would work any better.  At this time it doesn't support Fedora Linux.  I downloaded the Red Hat 9 CD images and installed it in Virtual PC.  Red Hat 9 would hang during startup in Virtual PC.  I then installed Red Hat 9 on a 1 G Hz Athlon.

Since I have 3 hard drives on this system.  I moved all the data from the 3rd drive to CD s then re-partitioned the drive under XP, having 40 G as a Windows NTFS partition and 40 G unused space.  I then booted off the Red Hat CD and began the install process.  I selected automatic partitioning, being careful to select the 3rd drive.  For some reason it would not use the empty space at the end of the drive.  Since I was too lazy to back up and partition it manually I told it to delete all partition on the drive, so ended up using the full 80 G for the Linux installation.  Actually manually partitioning under Linux is fairly simple.  That is what I did back in 95 when using Slackware.  If you have ever successfully played with fdisk in MSDOS or Windows you should have no problems partitioning your hard drive under Linux.  If you do happen to select the wrong drive, you will wipe out your Windows installation.

When installing Linux for the first time you probably want to select a typical desktop system install.  To get a look at what will be installed select a custom install.  After looking through the list, proceed without making any changes.  I always scroll to the bottom and select the install everything button.

Red Hat update running on Red Hat 9, successfully took me through the process of registering with Red Hat and upgrading my installation.  You get a free trial to start with, so you have access to their high speed server for a few days.  This worked very well, was quick and didn't require constant rebooting like you do with Windows update.

I then installed Ximian Red Carpet.  There were no updates on the Red Hat 9 channel since I had just obtained all the updates from Red Hat.  I then tried the Mono channel, since that is what initially got me interested in working with Linux again.  (Reading The Cathedral and the Bazaar is what convinced me to go all the way).  It took several iterations and much trial and error to get most of Mono installed.  There were still a few items that would not install due to conflicts.  I was able to compile and run a simple C# Hello World program as well as a simple Windows Forms program.

These articles will help you get started with Mono:

Installing Mono: Running .NET on Linux and Windows by Charlie Calvert
WinForms: How to Use Mono to Get Microsoft .NET GUI Based Applications Running on Linux by Charlie Calvert
Running Code Behind ASP.NET CSharp Projects on Mono by Charlie Calvert

Other links:

Linux Distributions at ibiblio, the publics library and digital archive

Links which appear in the text:

Agile Hacker
Fedora Linux
Ximian Red Carpet
The Cathedral and the Bazaar

Books to read:

The Spike: How Our Lives Are Being Transformed by Rapidly Advancing Technologies by Damien Broderick, amazon.ca, amazon.co.uk
- This book helps set the context for the future of Open Source.

Free as in Freedom by Sam Williams, amazon.ca, amazon.co.uk, online
- This book helps with understanding the origins of free software.

The Cathedral and the Bazaar by Eric S. Raymond, amazon.ca, amazon.co.uk, online
- This books helps with understanding what Open Source is, how it leads to better quality, and how it contributes to the bottom line.

June 6, 2004   -   Compiling and using CppUnit on Windows
updated June 12 2004


One of the corner stones of Agile techniques is Unit Testing.  I plan to mostly work in C#, so Nunit would be the unit testing framework to use.  I have been using C++ for many years, still write and maintain C++ code, and many Open Source projects use C or C++, therefore I will start by getting up to speed with CppUnit.

These are just my quick notes while playing around.  A re-write is in the works.

Read about CppUnit at the CppUnit FrontPage.
Download CppUnit from SourceForge.
Other Unit Test frameworks can be found at XProgramming.com

Downloaded cppunit-1.8.0.tar.gz
Launch Visual Studio using \cppunit-1.8.0\CppUnitLibraries.dsw
I am using Visual Studio .NET 2003, but CppUnitLibraries.dsw is for Visual Studio 6.
Use Batch Build to build the works.
Not all the project build because the object model for Visual Studio add ins has changed since Visual Studio 6.
When I was playing with this before I removed the projects that didn't compile, from the solution and played with what was left.
This time around I will do things a little differently.
Download cppunit-1.9.12, from the Development Snapshot
Use cppunit-1.9.12\src\CppUnitLibraries.dsw.
Use Batch Build to build the works.
Same problem as before, not all projects compile.
Using \cppunit-1.9.12\examples\examples.dsw, batch build the works.
Some errors during compile and link as expected.
Download documentation cppunit-docs-1.9.12.tar.gz
Extract into \cppunit-1.9.12\doc
Open index.html with your web browser.
Spend some time browsing around to see what documentation is there.
On the index page, follow the link to the CppUnit CookBook.
Skim through the page, the code should make sense, but you likely have no clue how to use it.
The first clue is more then half way down the page, where you will find the TestRunner section and a listing for the main function.
Create a new work directory for your test code.  I am using \cppunit-1.9.12\bill.
With Visual Studio, create a new Blank Solution called Test inside your work directory.
Right click on Solution Test in the Solution Explorer and select Add Existing Project.
The project to add is \cppunit-1.9.12\src\cppunit\cppunit.dsp
Delete the contents of \cppunit-1.9.12\lib, since it has a lot of stuff from the previous build attempts.
Rebuild the Solution.  It should be successful and the file cppunitd.lib should appear in the lib directory.
To verify that the next step works, delete the contents of the cppunit-1.9.12\examples\simple\Debug directory.
Add the \cppunit-1.9.12\examples\simple\simple.dsp project to your Solution and rebuild.
The contents of cppunit-1.9.12\examples\simple\Debug should reappear.
Open the command prompt and cd to cppunit-1.9.12\examples\simple\Debug.
Run the simple.exe program.
It will report errors in the exampletestcase.cpp file, which were put there intensionally by the example creators.
See if you can correct the errors, and re-run simple.exe until it reports no errors.
Spend some time looking through the code in the simple and cppunit projects.

Add project \cppunit-1.9.12\examples\money\money.dsp to your solution.
Build then run moneyd.exe from the Command Prompt.
Should print OK (4).
Add the following line of code to the private section of class MoneyTest in MoneyTest.h

  CPPUNIT_TEST( testJunk );

Add the following line of code to the public section of class MoneyTest.h

  void testJunk();

Add the following function to MoneyTest.cpp

void
MoneyTest::testJunk()
{
    CPPUNIT_ASSERT( 1 == 0 );
}

Build.
An error : Assertion should appear in the Task list.
Double click the error in the Task List.
This will take you to the failed assertion in the code.
Run moneyd from the Command Prompt again and take a look at the error message.
Change the ASSERT to:

    CPPUNIT_ASSERT( 1 == 1 );

Build and run again.
Now all we have to do is study the code in the money project to get started writing test for our own code.
The CppUnit Cookbook should now make sense.

How does that piece of magic with the Task List work?
Take a look at the Property settings for the money project.
Look at the Post-Build Event in the Build Event section.
This line of code in main, and the CompilerOutputter class contributes to the magic.

  runner.setOutputter( new CPPUNIT_NS::CompilerOutputter( &runner.result(),
                                                       std::cerr ) );

MFC


I noticed a new version on SourceForge, so I am using ver 1.9.14 for the following:

If you want a GUI for you unit tests, there is the MfcTestRunner.

My suggestion is to not write any new code in MFC unless you have to, but that is another story, lets get back to the topic at hand.

I am starting with a new version, so I have none of the baggage hanging around from the test mentioned above.
Create a work directory.
Create a new blank Solution and call it MfcTest in that work directory.
Add the cppunit.dsp project from the src directory to your project and build.
I received an error indicating that it could not write to the lib directory, so I created this directory manually and rebuild. Ensure that cppunitd.lib appears in the lib directory. Doing a build instead of rebuild did not work.
Add the CppUnitTestApp.dsp project from the examples\msvc6 directory, and build.
The build fails, since it can't find the testrunnerd.dll.
Add the TestRunner project from the src\msvc6 directory, and build.
There was an error, likely caused by the post build step in TestRunner, but hitting the run button caused a rebuild which succeeded this time. Make sure to set CppUnitTestApp as the startup project.
You now have a running UnitTest GUI program to play with.

June 26  -  Moving to Open Source

You are interested in using Linux but you don't want to give up all those Windows applications that you have grown to depend on.  One solution is to use both systems.  If you have a spare computer use it for Linux and set up XWindows on your Windows box.  This way you can easily move back and forth between Windows and Linux.  I haven't tried this, since I don't have a spare computer.  You could use Virtual PC for running Linux, but this has limitations which will reduce your ability to do serious work on Linux.  The most common solution is to set up your system as dual boot.

The dual boot solution has its problems, mainly having to reboot your computer constantly to switch between your favorite applications and Linux.  I will now talk about some alternative Open Source applications that you can try on Windows, as replacements for your favorite applications.  If these applications serve your purposes, then why not run them on Linux instead of Windows.  Mixed in with this I will have some of my history.

You are likely using Internet Explorer, just because it is there, or you have tried other browsers in the past and didn't like them.  Try out the FireFox & Mozilla browsers which you can find at www.mozilla.org.

I stopped using Netscape back in ver 4 days, because it rendered my system unbootable.  The combination of Windows 95, Visual Studio, and Netscape was a deadly combination.  Just weeks before Win 98 hit the shelves, I purchased a new system, with Windows 95, to be used for software development.  After receiving new computers, I alway erase the hard drive and re-install everything to ensure that it is done right.  After spending an afternoon, installing Windows & Visual Studio, I got down to the business of witting software.  A couple of weeks later I decided to install Netscape.  This rendered my system unbootable.  I erased the hard drive and started over.  This time the first application to be installed after Windows was installed and updated, was Netscape.  Everything worked fine.  I then installed Visual Studio, and the system was unbootable.  Since the purpose for this system was to write code using Visual Studio I left out Netscape on the 3rd system re-install.

For the last few weeks I have been using Firefox as my main browser.  I also keep Mozilla around and I am using Mozilla Composer for witting this.  I tried the HTML editor in Open Office but didn't like all the extra formating it put in automatically.  Both the Open Office editor and Mozilla Composer occasionally mess up the relative links.  I have uploaded this page to my ISP a couple of time only to discover later that some of the links had absolute paths from my local hard drive, instead of relative paths.  I think this is one of the reasons I switched from Netscape Composer to FrontPage, many years ago.

I haven't been using Frontpage lately because it wasn't included with Office 2003.  As it turns out purchasing Office 2003 was a waste of money.

A while back I converted my website to ASP.NET.  The ideas was to automate page composition. I was getting tired of editing 60+ pages every time I wanted to update my sidebar menu.  The pages with book listings were also tedious to edit in FrontPage.  I did start writing VBA Macros in FrontPage to automate some of the work.  After working with C#, I wanted to use it for all my coding.  The idea here was that if I use one language for everything, I won't have to spend so much time learning the tricks in several different languages.  This same reasoning is why I used C++ almost exclusively between 1990 and 2001.  Since 2001 I have been trying to focus on C#, but find that I often still work with C++.  I was distracted by Java & Visual Basic for short time periods, but soon discovered that they didn't work well for what I do.

My book listings are now entered into a database.  Since I don't want to pay my ISP extra for using a database I dump the database contents into XML files before uploading.  A simple utility written in C# extracts the data from the database and writes it to XML files.  Simple C# code on the web page loads the XML & XSLT to generate HTML and insert into the page.  The problem I still haven't solved is editing large amounts of text such as this page, and storing it as XML.  This is why I purchased Office 2003.  Since Microsoft is making a big deal about the XML capability in Office and about using ASP.NET for separating content development from display code, I had assumed that it would be easy to use Word for editing my web content.  This didn't work out of the box, and I haven't spent a lot of time working on it, but it looks like the only way is to write the XSLT to transform the WordML into HTML.

I could just edit the text in Visual Studio, since that is where I write the C# and create the ASP.NET tags, but Visual Studio has no spell checker.  Also the goal is to have the content separate from the layout code.

After downloading OpenOffice, I discovered that its native file format is XML files inside of a jar.  You can use your favorite unzipper to extract the files.  The file with the text looks very simple compared to a WordML file.  This should require a much simpler XSLT program then what is needed for WordML.  Maybe I can find the required XSLT on the web.

Download OpenOffice and give it a try, you may no longer need Microsoft Office.  Use it on Windows & Linux.  You probably already have it on your Linux installation.

If you use Access or MSDE, take a look at MySQL.  I haven't spent any time using it, but when I was doing the tech review for Beginning C# Databases, I download MySQL installed it and tested the example from the book in less time then it took to download the CD images for Oracle from the Oracle website.  As it turns out I never did install Oracle and try it out.  Wrox only give me 1 evening per chapter and no advance warning about what tools I needed to test all the code.

For email, chances are that you are using Outlook Express or Outlook.  I have used Outlook Express for years and it works very well.  Anyone who has not kept the patches up to date has probably had some grieve as a  result.  I have also used Outlook for a while.  The main problem with Outlook is that it has a lot of extra crap which never gets used, also the newsreader looks like it was tacked on as an after thought and not integrated with the email part of the program.  Since 90% of my email these days is spam, I was using SpamBayes with Outlook.  To start with I didn't train SpamBayes, but just used the Delete As Spam and Recover From Spam buttons, as I received mail.  This started to work, then began to deteriorate, I was spending a lot of time clicking the Delete As Spam button.  It turns out that SpamBayes didn't have enough information about Ham (good messages).  After several weeks there was only about a dozen ham messages that I had recovered but a couple thousand messages marked as Spam.  After downloading and installing the upgrade, I trained it with a few thousand ham messages as well as the 2000 or so messages in my Spam folder.  It worked much better after this.

For about 3 weeks now I have been using Thunderbird.  You can find at Thunderbird at mozilla.org.  Thunderbird has a built in spam filter which you train by clicking the Junk or Not Junk button.  During the first 2 or 3 days, I found some good messages in the Junk folder.  After 3 weeks some spam still gets left in the Inbox, but most spam automatically moves to the Junk box.  The news reader is well integrated with the email program, and works much like Outlook Express.

Now that all my email, web browsing, word processing, spreadsheet, and database applications can be solved by Open Source programs that work on both Windows & Linux, there is still the question of software development tools.

I was willing to live with the occasional system crash because there was nothing like Visual Studio on Linux.  After using QuickC & Borland C++ for years, reverting to command line tools on Linux felt like a throw back to my CP/M & early MSDOS days.  My desire to have the computer handle the tedious details so that I could concentrate on code is what led me down the path of rejecting Linux and becoming a Windows developer 8 years ago.  Without taking the time to immerse myself in the Unix culture at that time, I missed out on the concept of a little up front learning and using scripting to automate tedious tasks.  Visual Studio still has some advantages over what is available on Linux, but that gap is closing rapidly.  The only way to find out is to immerse myself in Linux development for a while.

Back when I started using Windows 95, I was still writing MSDOS code.  It was very easy to crash the system with bugs in MSDOS code.  Installing software on Windows 95 was also a hit and miss proposition, it would often render the system unbootable.  Switching to NT 4 at work and Win 98 at home as well as developing Windows code, almost eliminated the system crashes.  The move to Win 2000 & XP was a great improvement over Win 98, & NT.  Win NT was very reliable but it was a long drawn out process to install, upgrade, and configure.  Replacing Pentiums with Celerons and NT with Win 2000, made it a lot easier to install, update and configure systems before sending them out to the field.  With our application a Linux box was used to dial up the Windows boxes once a day and download the data.  When we first started deploying Win 2000, the Linux box was unable to connect to the Win 2000 boxes.   After bashing our heads on Win 2000 configuration for 2 weeks, the Linux sys admin discovered that he had missed a patch on the Linux box.  The problem here is that Linux sys admins always assume the Windows box is to blame.

This also showed the ugly side of Windows.  Typically you can install Windows, run Windows update to apply all the patches then get down to work.  This lets me concentrate on the code development and not spend a lot of time on learning sys admin.  The problem arises when you want to do something that is not a typical use for Windows.  In this case it was using a workstation system as a dial in server.  It was a little difficult to find all the check boxes in various setup panels that needed checked, as well as all the utilities for monitoring and logging only exist in the server versions of Windows.  It was only after I installed Windows 2000 Server and logged the details of the connection attempt that the Linux sys admin decided to look at the details of what was going on, on the Linux side.

Another aspect of Windows which is a real pain, is getting data into the system.  Actually this problem can be overcome to some extent by throwing money at the problem.  If you purchase expensive data acquisition hardware, it typically comes with device drivers and libraries.  When you work on projects where they are counting the pennies spent on hardware, but not on developer time, we often build custom hardware. Then they expect me to make the software work.  I was able to avoid the need to learn device driver development by purchasing DriverX from Tetradyne. When attempting to move to USB, the ugly side of Windows, again rears it head.  In order to embed a USB micro controller in our external electronics I need to learn Windows device driver development as well as 8051 programming.  The prospect of writing 8051 code didn't bother me to start with, since I have written my share of machine and assembly code in days gone by.  What turns out to be the problem is the impossible price tag on development tools from Keil.  My plan was to learn how to use open source tools, like the Small Device C Compiler.  Where I ended up wasting all my time was trying to get Windows to talk to the Cypress development board as I have written about elsewhere.  Since I was making no progress with the USB connection we decided to try a Digital I/O card from Measurement Computing.  This card comes with device drivers and libraries.  It took me about a day of coding to integrate code into my application so that it worked with the Digital I/O PCI card.  The next attempt at using USB will be using QuickUSB from Bitwise Systems.

The real problem here is that Microsoft puts all their effort into commonly used features, or whatever their marketing department is all gungho about.  The result is that you have thinks like web services, and data base applications that can be built in seconds with a few mouse clicks.  When it comes to doing odd ball things like writing code to access your home built hardware, you are on your own.  The solution, is you don't have to be a loner, that is what SourceForge is for.  You are more likely to find others on SourceForge, working on similar projects if you are using Linux.

Ok, but I still like the computer to do my project management, give me intellisence prompts, show me the contents of a variable by hovering the mouse over it, and let me visually single step through the code.  Using unit testing does tend to reduce the need for the debugger, but I am still not good at writing and using unit test.

The plan is to keep Windows around for running Visual Studio, spend time learning the development tools available on Linux, and write portable code where possible.  The next version of Visual Studio will give me .NET 2, generics in C#, and refactoring.  Beyond that I see Microsoft taking Visual Studio & Longhorn in directions that are of little use to me.  The idea of .NET becoming the OS appeals to me, but I have the feeling that Longhorn will be so over burdened by what they perceive as the needs of corporate users that it will become useless for the rest of us.  While continuing to use Visual Studio on Windows, the trick is to no longer waste time learning the inner workings of Windows and only write code which also compiles on Linux.  All new Windows software should target .NET, since win32 is about to be tossed out the window.  All native code should be as generic as possible, don't mix in win32 calls, and don't use MFC.

It looks like Mono will give me what I want.  Combine this with SourceForge and the long traditions of the Hacker and Unix cultures I should no longer have to struggle in the dark fighting with Windows to get the hard stuff done.  Over time MonoDevelopSharpDevelopEclipse, and other tools will eliminate the need for using Visual Studio.

I must be a slow learner.  After 20+ years of trying to turn myself into a software guru, I still haven't got very far.  I figured that after reading enough books and writing enough code, I would cross the threshold to enlightenment.  It never happened.  The reason it never worked is because the only measure of my success was the compiler accepting my code and the program accomplishing what it was suppose to do.  The only way to move beyond this is to have others review my code, and to review the code of others.  This is where Open Source comes in.  I imagined some time in the not to distant future when I would have semi-intelligent software to assist me with witting code.  Why wait for AI, when the real thing is there with a click of the mouse.  The Internet is a buzzing hive of minds ready to use, abuse, & criticize your source code.