A few software developers in Portland, Oregon learning Extreme Programming by creating an open-source VB/VBA add-in.
Just a quick update in case anyone else is checking up on us: what with the holidays and Steve's packed schedule, the group is on hiatus through the end of the year. We'll meet again to finish the current release in progress and see what people want to do from there.
That is all. Happy holidays!
Twas just Steve and I for the first couple of hours. We decided to forgo the planning session until Kristi arrived in the afternoon, so our first order of business was to try to speed up the tests.
Running the entire test suite was taking a considerable amount of time, which is bad because it creates a situation where we'd want to avoid running them during development (for example, "well that was just a tiny change -- we can probably get by without doing all the tests"). So we did some time trials. It took just over one minute to run the 65 or so tests (split up among 7 or 8 test classes). Steve had implemented an optimization last week having to do with the VB Line Reader, but we learned that it didn't really save any time (maybe one second), so we took it out since it just made the code more complicated.
It was immediately apparent that the 'MacroExpander' test class would be the focus of our effort, since it was taking up almost half the time (28 seconds)! We did some straight-forward ("easy") refactoring and cut 10 seconds off the time -- not bad for a first attempt. We also realized that the macro expander was opening and closing the target project over and over again for each test. We figure that in the future we could cut down the test time significantly by isolating the project open/close function to just one test, and have the rest of the tests use pre-created text ('fake' modules).
---
When Kristi showed up, we began our planning session. Steve started off by telling us about an aspect of XP that we had been neglecting: writing 'user tests' on the backs of story cards. These user tests are "human language" explanations of what a feature will look like and how it will behave when finished. They do not need to be as thorough as unit tests, but the process of writing them did force us to flesh out our implementation ideas more thoroughly, and should help us stay on course as we work on each story.
We then calculated our velocity, which wound up being 3 stories per iteration. We had 4 story cards remaining that we wanted to fit into the release, and started discussing and prioritizing them. "Friendly Macro Error Reporting" and "Auto-Backup" were both partially complete (we had started working on them two weeks ago), so we decided that we would tackle those first once we finished the planning session. However, we wanted to talk more about the other two story cards, "Pre-Expansion Check" and "Macro Seeding" before we moved on. This wound up taking significantly more time than we anticipated...
---
The "Pre-Expansion Check" feature is intended to be used in situations where someone may have inadvertently altered code that had previously been expanded from a macro definition. In this situation, we want to make sure that the user would be notified upon subsequent macro expansions, so that the altered code wouldn't get silently overwritten (after all, just because the third party wasn't familiar with our spiffy macro system doesn't mean they didn't write some useful code).
As we discussed this, it became apparent that implementing this feature isn't as easy as it sounds at first. After all, how do we know that someone changed the code in the macro target section, as opposed to the macro definition itself being changed? We obviously don't want to alert the user every time the macro definitions change, since that's the whole purpose of using the macro system in the first place! After wrestling with this for a while, we narrowed it down to three possibilities:
Option #1: Utilize a "checksum" as an attribute of the macro target tag, which would be calculated each time a macro expansion takes place (the checksum value would be based on character count and/or numeric representations of the VB code content). Upon subsequent expansions, any macro target code section containing this checksum tag would first have the checksum re-calculated. If the current checksum value doesn't match the old checksum value, then we'd know that the content changed, and could alert the user before overwriting this potentially important code. Unfortunately, this wouldn't tell you exactly what was changed, but at least you would have an opportunity to examine the code before it gets overwritten with the new macro expansion.
Option #2: Each time a macro is expanded, "back up" the definition by writing a copy of its text to a "
Option #3: Drop the feature altogether. We could simply notify users of the macro system that they should never change code within a macro target's opening and closing tags. We could easily have the expander add comments to the macro target areas saying something like "WARNING: Generated Code -- Do Not Modify".
Our decision was to go with option #1. Option #2 was really overkill when we thought about the intended usage of the macro system, and we didn't want to so readily admit defeat by going with option #3. The checksum feature provides a nice balance between usefulness, simplicity, and ease of implementation.
We then talked about how to handle these potential occurrences in the User Interface. We decided to show the user all macro target sections that didn't match their checksums at the same time via a simple dialog box. The user could then decide to either keep the existing code, or overwrite it with the re-expansion (or just cancel the whole expansion altogether). We thought it would be extremely annoying to be prompted over and over again for multiple checksum-mismatches. In the future, we could add checkboxes to the dialog that would allow the user to selectively overwrite on a per-target basis, but for the time being it will be an all-or-nothing choice.
---
Okay, next was the "Macro Seeding" story card. This feature is intended to be used in situations where you want to put the same macro targets in every procedure throughout a project. The primary example of this would be adding consistent error handlers to code you inherited from someone else. The two issues we had to contend with were: How should the seeding code be stored, and what should we do about overwriting existing macro code?
Keeping consistent with the way we were doing everything else, we decided to simply have the seeding code contained within "
With that out of the way, we needed to decide what to do about overwriting existing macro code. This would be a situation where a project has already been seeded, but you want to re-seed it with something different. (note that if you wanted to change existing macro code, you could simply change the macro definition and re-expand). We came up with a few possibilities for how to handle this:
Option #1: If ANY macro code exists in a procedure, don't re-seed it.
Option #2: If any tags that appear in the seed code also appear in a procedure, don't seed it.
Option 1 doesn't allow for much flexibility, and option 2 sounded like a nightmare to implement, so we came up with Option #3: Allow for a special sub-section of the seeding code where the user can list specific "Don't-Seed" text. These pieces of text would probably be macro tags, although they don't have to be. Procedures containing any of these strings would NOT be re-seeded. This would be a piece of cake to code, and provides the user with a good amount of flexibility.
Now in addition to fleshing out the macro seeding story, we have some sample seeding code to include with our "Standard Error Macros" that will be distributed upon release.
---
Phew! As I mentioned, the planning session took up a lot of time (well-spent, of course). We had one hour left, and Steve worked on "Friendly Error Messages" (for macro language syntax errors), while Kristi and myself worked on "Auto-Backup" (for copying code before being overwritten by expanded macros). Steve reported that we have 2 friendly error messages thus far, and there will probably be 5 by the time we're done with the story card.
Auto-Backup is almost finished -- we just need one more test case for backing up Forms and Class Modules (as of now, we only have tests for backing up standard code modules). We did hit one nasty snag, having to do with writing the backup files to the filesystem. Steve quickly diagnosed the problem -- we unnecessarily called the Dir() function without passing it any parameters, which apparently you can't do if you didn't call it previously WITH parameters (uhh, right...). Anyway, let this be a lesson to us all about the dangers of copying and pasting code!
I just remembered more about why we're tagging every line of VB Code. I think the idea was to be able to put context information there to help figure out where the seed points for automatic tag insertion would go.
If that's the case, we may have been thinking -way- too far ahead. Nevertheless, since the code is there now, and not causing any obvious harm, we may as well leave it for the time being. If it ever gets in the way or seems too messy, we can take it out, or we might end up actually using it when we get to macro expansion point seeding.
As it turns out, no one at all except for little ol' me was able to make the work session. I decided that the thing to do would be to simply do whatever I could by myself and use this as a chance to compare with what happens when we have a whole team, do pair programming, etc.
First off, I wanted to spend much of the time working on stuff directly related to stories in the iteration plan, but also, we've been needing a way to do source code control for a while, and I wanted to make some progress on that.
I had already written some really simple code in a previous session that writes the Access code modules out to files, so they can be added to a source code control system, but I knew that wasn't enough. Our code consists not just of code text in code modules, but of a form, and a table that contains test input/output, and might someday contain reports, and even a macro or 2. I wanted the system to be able to write out all of that, and I wanted the system to be able to compare what was written into the export directory with what exists in a directory managed by a source code control system, and update the directory accordingly. Oh, and one more little thing. I wanted the module file extensions to be .bas for standard or .cls for class modules.
Ideally, this code would have had tests, etc., but this should be very simple code, it's a tool for our process, not part of our production product, and it's harder to do TDD alone than with a pair. So, I proceeded without tests. In this case, that seems to have worked out OK.
To summarize the simple parts of what I did, I factored out the code that loops through a list of AccessObject objects, and calls SaveAsText to write them to files, then called it once for each type of object, also passing a file extension. Also, in the case of code modules, I had to add code to use the VB IDE to determine whether each module is Standard or Class to add the correct file extension. When I got to tables, though, I had some issues.
So - tables. First off, the question of whether to save tables at all, or just table structures. Is a table part of the "code", or is it application data? Some people put application data in the same MDB file as the application code, but this is not considered a good practice. In general, a table that is embedded in the application MDB is considered part of code support, and that is true in our project as well, so i decided that, if the table is not linked, it should be saved to text. Something should also be done about links, but the link should probably be just a file with a connect string and attribute settings, or something. In any case, Access has no built-in capability to do any of this.
What I ended up doing is deciding that, for now, it would be good enough to just save embedded table data as a persisted XML recordset. To recreate the table from this data later, we would be missing information about indexes, etc., but this is far better than nothing, and a good place to start, so I press on. Since I;m going to use ADO anyway to save the recordset as XML, I go ahead and use ADOX to read the list of tables, figure out whether they're linked, etc. I have to bone up a bit, but it all works out OK. It seems like I should have been able to open the output file as a stream, and write to that, but I never figured out how, so for now, I am writing the recordset's XML to a stream in memory, then saving the stream to a file. What the heck - it works.
After finishing this work, I come to a couple of realizations that some of the team reading this now might have come to as well.
Realization #1:
All of the work to save code modules is duplicating work that has already been done on a story in the application itself: "Auto Backup". If I checked that story, I might have found that it was already done last week, and I could just use what's there. If it wasn't done, my efforts might have been better applied to getting it done. With a pair working with me, we probably wouldn't have made that mistake, and would have saved a lot of time.
Realization #2:
I'm writing code to save table files embedded in an Access project for purposes of updating files under source code control, but we're going to move this code into a VB project soon, and then we'll have to store that test data some other way anyhow, probably in some sort of text file, or something. Has that been a waste of time? Again, a pair could have helped make sure that question was asked sooner, and perhaps saved the time spent working on it if it was not worth doing.
Anyway, on to the stories...
I decided to continue work on "Context Parameters" since I've done a lot of work on that recently and have a good idea what direction it's going. When we last left off, we were doing a sort of "combing" refactoring of code to gradually push replace code that deals with "code text" with code that deals with arrays of our new clsVbModuleLine objects that contain the context information we're going to need. We figured we were mostly done with the steps of that refactoring before, so I kept going. In short order, I had converted all public methods that take code text into mere wrappers for testing purposes that convert code text into arrays of code lines before calling other methods, removed all calls to those methods from the class itself, and modified code all the way down the chain to use clsVbModuleLine arrays, not text or arrays of strings.
Aha - I thought. Now, I have all the information I need to generate XML with context information derived from what was stored in the module line objects when they were read from the code module by the module line reader! Not so fast.
I know have what seems to be 2 pieces of rope to tie together, but they don't quite meet in the middle. We have module line objects with context information, but where we need the information is in the context of a Macro tag. The trouble is that we don't build the macro tag, we get it from the comment line that comes straight from the raw code text. In order to expand the macro with the right context parameter data, though, I pretty much have to get the context information into that Macro tag where it's easy to find from inside the XML DOM.
The trick I worked out is to actually check for a '<Macro... line in the ConvertCodeLinesToXml function, if so, actually insert context attributes into the tag by inserting them ahead of the > that closes the tag. It looks like this is working, but now a whole bunch of tests fail. What's up with that? Well, when we re-create the code text from the DOM document, now the context attributes are in the Macro tags. The spec doesn't call for these to appear during macro expansion, and there's no reason they should, so what do do?
I see that what I need to do is pull those attributes back aout when I'm regenerating the code text from the XML. For some reason, I decide doing it in the DOM will not be adequate because what happens with white space counts, etc.? So I spent several hours writing code that seems to be close and is making most of the tests pass by paring and ripping the extra attribute text out of the Macro tags after expansion. I was thinking, this is not working in small steps, but couldn't think of a better way to procede.
I have made a number of choices here that should preserve whatever white space was in the tag prior to inserting the extra attributes, then I think to ask this question - if it makes sense to use this logic to preserve other white space formatting, why would the DOM not just happen to use the same sensible rules? So I comment out all the code I just wrote, and add essentially 2 lines of code to delete the context attributes from each Macro tag in the DOM immediately after expansion, and it works! More tests pass now than were passing when I left off with the text-based approach, so I just delete all that code.
At this point, only a few tests were failing - the ones that test for what XML to get after parsing code into an XML document module are new failing because they weren't expecting to see the new context attributes. I could have changed the tests to match the new output, but I figured those tests should not really care what extra context parameters get added later, so I added wildcards at those points in the test data, and changed the tests to use Like instead of Equals to do the compare. VB Lite Unit doesn't have an AssertLike, so I wrote one in the project.
Finally, now the tags have the metadata we need, and we just have to pull it out when expanding the macro. By now, this was totally trivial. Just get the macro definition body from the collection into a string, and replace all instances of each context parameter with the data from the matching attribute from the Macro tag at the expansion point. It works - all the new tests pass - hurrah! There's a small amount of duplication here, and there will be duplication in the future with what custom parameters will do, but for now, it's not enough to be worth fixing (the cure would be more complex than the disease), so I call it done.
What's still left to clean up later is that, now the methods that take text as input for parsing exist only to support testing with plain text, and they should probably be moved out of the class, and into the testing code. I made to-do notes for that because it was getting late, and time to quit.
Observations from the "Context Parameters" story:
I wasted a lot of time implementing a hard solution to a problem, when the easy solution turned out to pass all tests. I could have saved several hours by trying that first.
As I recall, the reason we have been wrapping every code line in a VbLine tag when inserting code into the XML document was so there would be a place to put the context parameter information later. If that's true, that work was for nothing because that turned out not to be the easiest way to get context info into XML where we needed it. We ended up puting it right into the Macro tag's attributes instead.
The better "XP" approach to coding for this story would probably have been to start from the other end of the code, and "fake it till we make it", before we even wrote the module line reader, etc., that we were "going to need [my quote, I'm afraid] to be able to get context information from the VB IDE". We first would simply write a test that expects the correct module name and type for a single procedure, hard code that procedure name into the macro expander, and make it work for the first test. Then we would have refactored that back up through the process to remove duplication. What we ended up with works, and seems sensible enough, but it we've had to take bigger steps in between tests, and we may have implemented several things that turned out not to be needed in the eventual solution.
At this point, unless someone can remember another reason why we wrapped all the code lines in VbLine tags, we should now actually take all that back out.
Observations for the session:
If I ever had any doubt about whether pair programming was at least as efficient as working alone, I don't have it anymore. Clearly, any time that would be "wasted" by having 2 people working at one computer with only one typing is more than outweighed by the number of times having a second perspective would have prevented doing unnecessary work or taking a harder path. That's in addition to the more obvious benefit of having one person able to think tactically about the lines of code being slammed out while the other keeps an eye on the bigger picture and can monitor when the "driver" is getting tunnel vision that could eat up hours on some trivial or off-topic item.
While in a way, it's true that this team's purpose is primarily to learn XP, and secondarily to produce good software, hopefully, the purpose of learning XP is to learn to use XP to produce good software. At this point, I think it would be good to explore what has worked, what has not worked, what of the good/bad is attributable to the XP process, etc.
First, the problem: What the customer asked for has some serious problems.
Now, that doesn't mean there's anything wrong with the execution, but of course, that's missing the forest for the trees. If we have a process that helps us do a really good job of implementing something while leaving us blind to the fact that what is being implemented is not as useful as we hoped, is there something about the process, or how we're doing it that could be improved in this regard?
By the way, I think that's exactly what we've done so far. Even with a team that only meets once per week, and often has little overlap in attendance between one week and the week before, we've been able to implement some really impressive features, and implement them in a very high-quality way. I think the XP process has had a lot to do with making this level of success possible in really awkward circumstances.
Again though - the question is whether these "really impressive" features have turned out to be the most valuable things we could have been implementing.
So, what is the role of XP in all of this? Well, one of the things XP is supposed to achieve is giving the customer what they asked for in small iterations, top priorities first, and allow the customer to evaluate results along the way, changing priorities at will (of course, we have been playing both customer and programmer roles). In our case, one could say that the customer really needed to have the story "Basic Standard Error Macros" done before anything else to have a good picture of what would be the outcome of all this work. It so happens that neither the "customer" nor the programmers aver did see the need for prioritizing this story higher until well into work on the second release!
Based on my recent reading, there are 2 common practices of XP that we've neglected so far (partly because I was not fully aware of them), either or both of which might have helped:
1. At the initial release planning session, on the back of each card to be worked on in the release, the customer and programers work together to describe a test (a user test, not a unit test) that, if it passed, would indicate that the story is completed. The programmers should be able to define the story as done when they can perform the actions described in the test successfully. Essentially, this is a testable specification for the story.
2. Implement and use an automated Customer Acceptance Testing system. This is a piece of code that can read some sort of a script file, table, or something, that could be easily written by the customer, and use that input to simulate the operation of the user interface at the highest possible level in the code. Essentially, this is a way to (almost) fully automate user tests. Here's a hypothetical example of what a simple script for our system might look like...
*make-new-access-project TestProject.mdb
*make-module basTest
'/<MacroDef name="Test1">
'/ Debug.Print "Test1"
'/</MacroDef>
'/<Macro name="Test1"/>
*end
*end-new-access-project
*start-session
*ui-open-access-project TestProject.mdb
*ui-select-module basTest
*ui-expect-pre-processed
'/<MacroDef name="Test1">
'/ Debug.Print "Test1"
'/</MacroDef>
'/<Macro name="Test1"/>
*end
*ui-expect-process-preview
'/<MacroDef name="Test1">
'/ Debug.Print "Test1"
'/</MacroDef>
'/<Macro name="Test1">
Debug.Print "Test1"
'/</Macro>
*end
*end-session
The reason I think #2 might have helped is that these tests are not supposed to be as atomic as most programmer tests, and the customer gets to help write them. That might encourage the writing of tests that more closely resemble a real user session from the customer's perspective. This in turn, might give the customer a better vision of what they're really asking for, and whether that's really what they want. In other words "If these programmers work for weeks or months writing code to make this test pass, is that a -good- thing?"
Attendees were Steve, Adnan, Jordan and Sam.
[This entry was written by Sam after an extremely busy week; many details have been lost to the vagaries of CRS*.]
After a brief planning session (in which, after some discussion, we left the order unchanged), we paired up and got to coding. Jordan and Adnan took the Automatic Backup story, coming up with a solution in which all module texts across multiple backups would be appended to a single XML document, and could be backed up and restored using a not-yet-written interface. Steve and I worked with Context Parameters, paying down some of the refactoring debt on clsVBModuleLineReader and its tests.
After lunch, we came back and discussed the Auto Backup feature. The point was raised that if our software caused a situation requiring modules to be restored from a backup, anyone using it might not be too inclined to trust our interface to do the restoring. With that and other points in mind, we came up with a different scheme, where a backup folder would be created with a date/time stamp embedded in the folder name, and all modules would be saved to individual textfiles in that folder, so that they could be manually copied and pasted back into the IDE as needed. (As a bonus, this mechanism also allows us to use a diff tool for integration.) Adnan and I worked on this new scheme while Steve and Jordan kept working on Context Parameters (since Steve had a clever idea for it that I'd forgotten over lunch).
Adnan and I re-completed the Auto Backup story and moved on to the Basic Standard Error Macros story, since everyone but me had already worked on it and we thought a fresh pair of eyes might be able to pretty it up somewhat. This did not happen. In order to allow for users to include custom code everywhere it might be needed, we need no less than five separate macros per procedure. The resulting metacode tags make a procedure virtually unreadable. (As Steve said, it's a good thing that the point of this exercise is to learn XP first and produce good software second.)
Design Digression: On that topic, I had two ideas that might help. The first is this: change the expansion syntax for macros with only one body line so that the expansion fits on one line (e.g., " Debug.Print "foo" 'print foo '/<Macro lines="1" />" or similar). This would require some additional logic to recognize this as a metacode section, but could be done. I'm not sure how optimal this would be in terms of procedure readability, but it's better than what we have now. The second idea was to implement user-customizable sections that would be ignored during re-expansion. Especially if we moved the tag for such sections over from the left margin, this would actually make it extremely clear where the user should write code, and visually reinforce the concept of a try/finally/catch (in VB order) block. It might look somewhat strange for basically the entire procedure to be inside custom sections in a single macro, but even if it's not a net reduction in the number of commented lines, this might make the overall effect of those lines much less jarring. Perhaps it's worth a mockup of what a procedure might look like under such a scheme before coding that feature?
Sorry for the slow update. The team is still going, I just got too backed up with other things to post the recap before. Sam has agreed to post the recap for September 26th, so we should be seeing it pretty soon. Please excuse some inevitable inaccuracies in the following due to the memory fade over the 2-week lapse.
The attendees were Kristie, Jordan, and I. It was mid-iteration, but we spend a long time reviewing the stories, partly because Kristie came up with some good questions that were worth taking time for. It was definitely worth the time spent to get everyone on the same page as to what the software is for, and clear up confusion about what some of those cards even meant. So, we wore our customer hats for a while, rearranged some priorities, clarified some cards, etc.
Also, during the planning discussion, I brought up the fact that I had tried to user-test the front-end at the end of the previous session, and ended up in a semi-lengthy debugging session, only to find out that the code was failing with an error because I had given it bad input. Clearly, the error reporting was inadequate because bad input is expected to be par for the course in a live usage situation. We needed clear error messages.
Of course, this should have been part of the spec for each of several existing stories before we ever started, and now it was clear that we should need a new story card to add them in. Furthermore, even though it was mid-iteration, the lack of clear error messages for the user was also wasting the time of the developers, so the story needed to be moved up.
The stories we worked on were Friendly Error Messages, Re-Expand All Macros, Context Parameters, and Basic Standard Error Macros. Why so many? Well, it had to do with effective use of time. There wasn't much time spent on Basic Standard Error Macros because we decided to just give it a quick review, decide that it still looked ugly and disheartening, and put it off until next iteration in the hopes that a different set of available eyeballs could improve on the design. We did a small amount of work on Friendly Error Messages, but intentionally shunted most of that effort off to the next iteration, so we could work what was already scheduled for the iteration, Re-Expand All Macros, and Context Parameters.
As I write this, I see a semantic ambiguity in our story naming that we might want to address. Basic Standard Error handling refers to macro code we are writing for the users to employ with our metacode system to insert error handling into their code. Friendly Error Messages refers to the error messages the user sees when running our metacode system with incorrect input. We might need to come up with a more precise grammar to make these distinctions more clear.
Anyway, what we did do for the Friendly Error Messages story is get meaningful errors from the XML Parser when feeding the munged VB Code into it. The funny thing is that we were already raising a custom error code, and had a test for receiving the expected code, but we had no meaningful text to go with it, so the text we got was the text for a completely different VBA error. The XML Parser, though, is quite capable of providing concise detail about a parsing error, so we built an error message string out of that info. To be honest, it looks like we did not write a test for that, considering the test for the error code to be adequate, but (since I can see the future), I know that we have perfectly good tests for error text in other cases. It would be too much to predict the precise output from the parser, but we could at least check that the string contained certain expected constant expressions.
As for Re-Expand All Macros, I was not involved in the work on that story, but I remember that we had believed it would be trivial, and just work, but that was not the case. I believe some refactoring had to be done to make the code clear enough to work with. As I recall, this story was completed, and thus, was the sole completed story for the iteration.
The Context Parameters story had been going on for a while and was taking longer than we thought. Basically, we had decided the best way to proceed was to create a code line reader class that would create a collection of code line object that each had the raw text, and all the context information that might be needed, such as procedure name, procedure type, etc. The trouble is, getting the VB IDE to give up that information is like pulling teeth. You basically have to play 20 questions and check for errors until the VB IDE agrees that you've told it what you want to know. We did not quite finish all the needed tests for the line reader, but seemed to be quite close. Of course, finishing that still doesn't finish the story. Next, we have to modify the existing code that expects to parse text blocks to now take collections of Code Line objects instead, and figure out how to make many existing tests that use plain text work with that - I speculated that a mock line reader that uses text for input might be the way to go. See the next recap for how this turns out.
The end of the session was the end of an iteration with 1 story done. The history of the 2-week iterations up to this point reads 2, 0, 2, 1. That's gives us an average velocity of 1.25 stories per iteration.
