Late recap of Sep 18th session

Sorry for the slow update. The team is still going, I just got too backed up with other things to post the recap before. Sam has agreed to post the recap for September 26th, so we should be seeing it pretty soon. Please excuse some inevitable inaccuracies in the following due to the memory fade over the 2-week lapse.

The attendees were Kristie, Jordan, and I. It was mid-iteration, but we spend a long time reviewing the stories, partly because Kristie came up with some good questions that were worth taking time for. It was definitely worth the time spent to get everyone on the same page as to what the software is for, and clear up confusion about what some of those cards even meant. So, we wore our customer hats for a while, rearranged some priorities, clarified some cards, etc.

Also, during the planning discussion, I brought up the fact that I had tried to user-test the front-end at the end of the previous session, and ended up in a semi-lengthy debugging session, only to find out that the code was failing with an error because I had given it bad input. Clearly, the error reporting was inadequate because bad input is expected to be par for the course in a live usage situation. We needed clear error messages.

Of course, this should have been part of the spec for each of several existing stories before we ever started, and now it was clear that we should need a new story card to add them in. Furthermore, even though it was mid-iteration, the lack of clear error messages for the user was also wasting the time of the developers, so the story needed to be moved up.

The stories we worked on were Friendly Error Messages, Re-Expand All Macros, Context Parameters, and Basic Standard Error Macros. Why so many? Well, it had to do with effective use of time. There wasn't much time spent on Basic Standard Error Macros because we decided to just give it a quick review, decide that it still looked ugly and disheartening, and put it off until next iteration in the hopes that a different set of available eyeballs could improve on the design. We did a small amount of work on Friendly Error Messages, but intentionally shunted most of that effort off to the next iteration, so we could work what was already scheduled for the iteration, Re-Expand All Macros, and Context Parameters.

As I write this, I see a semantic ambiguity in our story naming that we might want to address. Basic Standard Error handling refers to macro code we are writing for the users to employ with our metacode system to insert error handling into their code. Friendly Error Messages refers to the error messages the user sees when running our metacode system with incorrect input. We might need to come up with a more precise grammar to make these distinctions more clear.

Anyway, what we did do for the Friendly Error Messages story is get meaningful errors from the XML Parser when feeding the munged VB Code into it. The funny thing is that we were already raising a custom error code, and had a test for receiving the expected code, but we had no meaningful text to go with it, so the text we got was the text for a completely different VBA error. The XML Parser, though, is quite capable of providing concise detail about a parsing error, so we built an error message string out of that info. To be honest, it looks like we did not write a test for that, considering the test for the error code to be adequate, but (since I can see the future), I know that we have perfectly good tests for error text in other cases. It would be too much to predict the precise output from the parser, but we could at least check that the string contained certain expected constant expressions.

As for Re-Expand All Macros, I was not involved in the work on that story, but I remember that we had believed it would be trivial, and just work, but that was not the case. I believe some refactoring had to be done to make the code clear enough to work with. As I recall, this story was completed, and thus, was the sole completed story for the iteration.

The Context Parameters story had been going on for a while and was taking longer than we thought. Basically, we had decided the best way to proceed was to create a code line reader class that would create a collection of code line object that each had the raw text, and all the context information that might be needed, such as procedure name, procedure type, etc. The trouble is, getting the VB IDE to give up that information is like pulling teeth. You basically have to play 20 questions and check for errors until the VB IDE agrees that you've told it what you want to know. We did not quite finish all the needed tests for the line reader, but seemed to be quite close. Of course, finishing that still doesn't finish the story. Next, we have to modify the existing code that expects to parse text blocks to now take collections of Code Line objects instead, and figure out how to make many existing tests that use plain text work with that - I speculated that a mock line reader that uses text for input might be the way to go. See the next recap for how this turns out.

The end of the session was the end of an iteration with 1 story done. The history of the 2-week iterations up to this point reads 2, 0, 2, 1. That's gives us an average velocity of 1.25 stories per iteration.

Recap of Sep 12 Work Session

Attendees were Steve, Jordan, and Adnan. This was the beginning of an iteration, Jordan had been out of the loop for a while, and Adnan is still pretty new to the team, so we spent an extra long time in the planning meeting. From our previous iterations, the estimated velocity for this iteration was 1.5, up from 1 after the previous 2-week iteration. To lay out the plan for the remainder of the release, we alternated 1 story, 2 stories for each iteration column to the end of the release.

In the course of the discussion, we relized we had incorrectly prioritized the "Basic Standard Error Handler" story too low. We all had different ideas of what needs to be in a basic standard error handler, how the standard blocks should interleave with the user's code, etc. Because of this, we decided to first have Jordan and Adnan pair on this story, picking up where I had left off with fresh eyes. Meanwhile, I worked on fixing the bug that I found after the last iteration.

As you may recall, the bug I found was that one of our earliest stories - list all macros and calls was no longer working. The symptom was that the feature was implemented when we were analyzing test modules the metacode system project itself to generate the lists, and subsequently moved the modules out into other projects to be analyzed by our own project. No code tests broke when we changed that, but since our lister form had no way for the user to select a project to read, it was not functional and returned only empty lists.

So, on the standard error handler, the work was somewhat frustrating and disheartening. The point of this project is to create tools to make it easier to do better quality software engineering in VB/VBA/Access, and the metacode system seemed like a great place to start, primarily as an aid to standardizing error handling and allowing the error handling system to be changed easily. The disheartening part is how messy the cleanest standard error handling system we've been able to come up with really is.

The problem is exascerbated by the need for opening and closing tags around the expanded sections to allow for re-expansion, and we ended up with no less than 5 separate sections per procedure to allow for all the reasonable customizations a user might need to do. Furthermore, the metacode system design, as it is now, gives no way to add comments in the sections between the tagged blocks to indicate what the programmer should put there. Attempts at clever work-arounds were thwarted by the fact that error braching and resuming has to be done all within the same procedure and can't be delegated to a called procedure.

We figured out it would help a lot (though still be pretty darned messy) if the metacode system allowed for named custom sections within macros that could have default text, so the first time the macro is expanded, the custom section can be generated with a default comment telling the user what to put in the tag. When regenerating, the text previously in the named custom section would be retained and placed back in the same named custom section, thus preserving the user's customizations. This also would allow an error handler macro to consist of only a single macro definition with one custom code section that could contain the entire original procedure body when seeding the code, along with custom sections for special case error handling code. We had briefly discussed this kind of thing in planning sessions before, but didn't think we had a compelling need, so we never fleshed it out this far. Needless to say, we have a compelling need now, and we added another story card to the stack.

We have chosen to still call the "Basic Standard Error Macros" done because we're hoping that yet another fresh look at it can come up with a cleaner way to handle all the same requriements.

So - back to the bug fix. I had thought this was a bug that crept in in spite of pretty much full test coverage, but I was wrong. When I dug into the problem, I found that the form was calling a method of the model that had no tests, and that any tests of this method would have broken when we moved the test modules out of the metacode project itself. I can't figure out how this happened because I haven't seen anyone on the team breaking the rules, but this does seem to be an isolated case, so I just added the test that should have been there, and commenced making it pass. It took a while, but when I finally figured out what needed to be done, it was just a few lines of change. Once this test passed, I fixed the user form by simply making it a tab page on the form we build for simple preview since it aldready had a file selector and a way to call to the metacode system after the file is selected. This merging when pretty smoothly and again, only requried a few lines of code to make work, once I figured out which lines.

OK, so once I finished the bug fix, I switched back to the "Standard Error Macros" to finish up as described above, and Jordan and Adnan went to work on the "Reexpand All Macros" story. We had thought that one would just work once we wrote the tests, but by the end of the session, it still wasn't quite working yet. We think we're pretty close, and it should be working early next session.

Recap of Work Session for Sept. 9th

For various reasons, only Adnan and myself were able to be at the work session this week, and if I remember correctly, it was only Adnan’s second session (possibly 3rd?) working on our project. Nevertheless, we made great progress, and finished the Simple Expand in Place story.

This was the end of an iteration, and we finished a story in addition to the story finished the previous week, giving us a velocity of 2 for the iteration, and raising our overall velocity from 1 to 1.5.

It’s during a session like this that some of the benefits of the Extreme Programming process really shine through:

1. There was no question of what stories needed to be worked on. We were in mid iteration, and there were open story cards from the previous session that were started and unfinished.

2. It was pretty clear where to pick up work on our chosen story. The last few tests and the todos at the end if the current test list showed us the direction development was going.

So, in spite of jumping in on a story we had not worked on in the previous session, and with one of us pretty new to the team and the project, we were still able to get productive right away.

Because we’ve been a bit lax on the refactoring steps (though improving), I tried to make sure we did refactoring after each change, and a little extra for catch-up. We factored the system for opening Access databases and working with their code projects and modules out into a separate VB project wrapper class, for one thing.

One issue we ran into was that the most convenient way to actually write the updated code out into the target project was to loop through the modules collection of an open instance of Access, call the code to perform metacode expansion on module one into a string, and write the new text back into the module. The code that performs metacode expansion on a module, though, was previously written to open its own instance of the same database, and that was causing a conflict. We refactored the code so that it only opens one instance of the VB project wrapper class, and passes it down the calling hierarchy. This also improves the speed a bit, which is nice since the tests were starting to get a bit slow. Of course, when you read things like “and passes it down the calling hierarchy”, you know there’s more refactoring that can be done, and I’m sure we’ll get to that as we go along.

We found one bug in a previously completed story that, because it’s a GUI issue, our unit tests did not catch. When we implemented List all Macros and Targets, we had the target modules in the same project as the metacode system, and we’ve since changed the system to work on external Access, VBA projects instead. We never went back, and added a way to select a project to read macros and targets from.

I’m wondering if there’s something currently in the GUI layer that could have been pushed down into code that would have caused a test to break when the Macro/Target List issue appeared (I can’t think of one myself, anyone?) Probably, this is something that should be documented as a user Acceptance Test, so it would be caught by manual testing. Currently, we have no system for user Acceptance Testing on this project, so this is something we probably need to discuss.

As far as fixing the bug, I imagine we can just roll it into the Simple Preview form as a separate tab or something, and use the file and module selectors that are already on the Simple Preview form.