barton's computer science blog

Silicon Valley Code Camp 2008 Wrapup

barton@netspheres.net — Tue, 11 Nov 2008 08:00:00 GMT

We had a great day at this year's Silicon Valley Code Camp presenting "Mapping Agile Practices for Scalable Teams to TFS". In case you wanted to attend but were not able to, you can download a movie of the slide presentation here and an audio recording of the session here. We had a great time and it was good to share this information with such an attentive and interested group!

Luminous Group and Microsoft's BizSpark

barton@netspheres.net — Sat, 08 Nov 2008 08:00:00 GMT

We're very excited to have been asked to be part of Microsoft's BizSpark program for startups, making it easier than ever for privately-held software development startups in business for less than 3 years and have less than US $1 million in revenues to obtain full access to Microsoft tools and technologies such as Team Foundation Server and SharePoint. We've known for some time that while these products help teams work better, they are often cost-prohibitive, especially for startups who are just starting out.

That's why we are so excited about BizSpark. As a network partner, Luminous Group are able to sponsor young startups for this program and give them full access to Microsoft's full suite of development tools - without having to pay license fees - until the startup grows up and creates a return on this investment.

So - if you are a startup and you are interested in working with what many feel are the very best tools for software development around, have a look at the startup program guide and consider joining this amazing program.

Mapping Agile Practices for Scalable Teams to TFS

barton@netspheres.net — Sat, 01 Nov 2008 08:00:00 GMT

I'm doing an overview of how agile software development practices can be more effectively supported with software tools at the upcoming Silicon Valley Code Camp on November 9th at 10:45AM with my colleague Don Robins. We'll be looking at how agile practices can be effectively supported by tools such as Microsoft's Visual Studio Team Suite 2010 to increase team coherence and performance. If you have some free time come and join us!

Agile Open California Wrapup

barton@netspheres.net — Sat, 18 Oct 2008 08:00:00 GMT

We sponsored the Agile Open California conference for the second consecutive year and the event was held last week. It is a really stimulating, interesting and non-traditional conference. For example, there is no conference agenda. Instead, it follows the open-space model for self-organization, where people put sessions on a centralized board in real time, and people show up at the events that are of interest to them. I find the format to be very dynamic and engaging, allowing for the ideas in one session to literally give birth to other sessions which follow.

If you are interested in learning more about Agile practices or have already adopted them and are looking to connect with others to share best practices and ideas, I strongly recommend this conference.

The conference is part of the AgileOpen, and in fact there are conferences organized in this style all over the world, so check out whether there is one coming up near you.

The theme at Agile Open California this year was Principles to Practice, and so in keeping with this theme, my colleague at Luminous Group, Volker Frank, hosted a session on Agile and Strategic Business Planning. My colleague Amy Lightholder hosted a survey session on Tools for Agile, while I hosted a more specific session on Team Foundation Server for Agile. We also had the opportunity to meet some really interesting people and learn a lot ourselves.

Collaborative Systems as Cultural Constructions

barton@netspheres.net — Sun, 07 Sep 2008 08:00:00 GMT

Many companies are unaware how their culture creates material effects within the organization. Social conflict and the deployment of new technologies can produce positive changes within an organization by altering social dynamics and promoting new cultural models.

Culture, those patterns of human activity and symbolic structures that give such activities significance and importance, is not generally included as part of the design and deployment process when a company goes forward with a new technology. Many significant deployment failures are attributable to cultural issues (Lorenzi  &  Riley,  2003)(Yeo,  2002).

While we do not normally think of computer systems as part of our culture, the manner, dress, language, belief systems, and norms of behavior found within an organization clearly extend beyond the boundary of a computational system, especially one that is collaborative. People define the essential meaning and structure of these systems. This is why we say that they are cultural constructions.

To say that a system is collaborative is to imply that the system is designed to help people involved in a common task achieve their goals. Examples of this include the development of an idea, the brainstorming of a design, or the tracking of a shared objective.

Collaboration technologies deliver functionality for multiple participants to create or augment a common deliverable. Document management, workflow, audit history, and other mechanisms designed to capture the efforts of many into a managed content environment are typical of these types of systems.

However, in recent years, we have also seen the rise of so-called “social networking” systems; collaborative platforms are evolving, enabling people to meet each other, share information, and provide new kinds of social activity which, prior to these systems, did not exist. Systems such as MySpace and FaceBook demonstrate that collaborative systems can be social and their mass use demonstrates cultural acceptance.

At Luminous, we take the perspective that collaborative systems are fundamentally a social construct that take the form of the culture in which they are placed. If placed properly, they can also help that culture to grow and change in new ways that are in alignment with the strategic goals of the organization. However, the ability of the organization to take advantage of this opportunity is highly dependent on the manner in which deployment of the collaborative system occurs.

That’s why we approach the deployment of collaborative systems such as Microsoft’s SharePoint with particular attention on the culture where that system will be deployed. Cultures, based on predisposition, both embrace and resist change, depending on their traits. It is therefore vital to map out where your organization sits on the fault line of change before you deploy next year’s technology.

Patterns of thought in software adoption

barton@netspheres.net — Wed, 16 Apr 2008 08:00:00 GMT

We've been doing a lot recently supporting teams in deploying and / or upgrading to Team Foundation Server 2008. It's been a very interesting and enlightening period of our work at Luminous because it is field testing and reinforcing some of the fundamental ideas we have held about the underlying patterns of thought people experience when adopting software.

Software adoption is a complex cultural process that is often relegated to a functional discipline that is ill-equipped to address the organizational and cultural requirements that support successful adoption. Adoption is not simply putting someone new in front of someone. Optimally it addresses the notion of enhancing the capability of the individual, which inherently is about changing the way people think about the world and their relationship to it. Software adoption always occurs as some substep to a larger process, namely change management within an organization. Executed properly, it is always motivated by strategic goals. There is an excellent paper on this topic written by John Kotter from the Harvard Business Review entitled "Leading Change: Why Transformation Efforts Fail".

It's funny how we can sometimes generate certain ideas and then lay them aside only to later realize how important and central those ideas are to what we do. A case in point here has to do with the ideas I presented in my 2003 CPSR paper entitled "Patterns of Thought: An alternative approach to technology design".

In this paper, I argued that regardless of the type of technology, people tend to respond in the same, predictable ways to the introduction of a new technology. These responses can be categorized as:

Masking: New technologies emerge in the guise of (or are used as if they were) old technologies;
Convergence: Technologies evolve by converging with other technologies;
Specificity: Different people use technologies differently;
Ramifications: The adoption of technologies by a society always causes fundamental changes in that society;
Evolution: Changes may take time yet they do occur;
Entropy: People resist change.

Knowing this, I argue that when deploying new technologies (or designing new ones) that it is of great value to consider these cultural responses as part of the process. In this fashion, focus is tightened around the strategic goals of the technology for the target group so that clear steps to achieve the objectives can remain at the fore and be fully realized.

At Luminous, this is a core aspect of our practice. We have deep knowledge about specific technologies, but it is our understanding of organizational dynamics, culture, and human behavior that enable us to support organizations to consider how these critical factors play into the process of deployment so that these are considered and addressed in practical terms.

What we have found is that there is a profound difference between the results acheived by those organizations who take the time to fully explore the strategic, organizational, and cultural aspects of deploying a new technology and those who simply take a functional approach to the same process.

Results and benefits of employing our suggested approach include:

Greater alignment about the strategic vision;
Tangible ways to track and measure progress;
Accelerated progress;
Greater ease in execution as technology becomes perceived as supportive rather than hindering;
Technology investments are fully aligned with business goals;
End-users have a sense of ownership about "their" process and tools;
Enthusiasm for change process and an environment where innovation is possible;
Work environment moves toward proactive initiatives;

These are results we can get really excited about.

XLINQ , Schema Inference, and Intellisense on XML Properties in VB9

barton@netspheres.net — Wed, 19 Sep 2007 08:00:00 GMT

To continue with my topic on things you can do in VB9 that you cannot (yet) do in any other .NET language, I wanted to write a bit about XML Schema Support and intellisense support for XML properties in VB9.

First, I highly recommend reading Eric Meijer and Brian Beckman’s research paper XLINQ: XML Programming Refactored (The Return Of The Monoids), which provides background as to the motivation and implementation of XLINQ technology. The Monads referred to in the paper remind us that the constructs being integrated into the .NET framework are enabling a more functional programming paradigm. This and the concurrent development of the dynamic language runtime promise to further expand the capabilities of the .NET platform to languages and programming approaches so that any programmer with any approach can make use of the platform in a fashion they are comfortable with.

The LINQ project, as many of you are aware, brings to all .NET languages capabilities languages like LISP and Scheme have long enjoyed, based on the notion of the Lambda-function. There are many examples for LINQ available so I will not bore you with that topic, however, what I do think is very interesting are the extensions to LINQ which support XML (XLINQ) as well as a number of other VB9-specific capabilities such as XML Literals that make working with and programming against XML infinitely easier.

This blog post explores the way in which Visual Studio 2008 and VB9 work together to enable programmers working with XML to:

1. Infer a schema based on a XML file itself (useful in cases where you have the XML file and not the schema);
2. Use the inferred schema to hook up Visual Studio’s intellisense capabilities for the XML properties of that schema;
3. Use XLINQ to easily load and query some XML

Ready? Here we go!

Setup

First, make sure you have downloaded and installed Visual Studio 2008 Beta 2.

Then, in order to make inferring the XML Schema easier, download and install this XML to Schema Inference Wizard, which is an add-on to Visual Studio and makes the process of inferring a schema from XML very easy.

We will be querying a XML file that contains data from a plant catalog. To obtain this file, go to http://www.w3schools.com/xml/xml_examples.asp and download the file plant_catalog.xml to your desktop.

Next, launch Visual Studio 2008 Beta 2 and create a Console Application using the Visual Basic language:

Add a XML namespace to the XML file

In the same way that the .NET framework organizes the massive API it encompasses into namespaces to establish an organization for the framework and minimize name collisions, when we use XML using these tools, it helps to add a XML namespace if the file does not already have one. This XML namespace is used by visual studio to map the schema to the actual file you are working with. The namespace must be in the form of a URI, which means it can be either a URL or a URN. In a production environment, you would agree with your development team or department what the namespace would be, but for our example, it could be anything. To add the namespace to the file you downloaded, open it in notepad and extend line 3 of the line to read as follows:

Think of this as analogous to the code you would write when importing a .NET namespace into your procedural code. Placing this namespace in the file provides a unique reference point for the schema we will infer.

Infer the Schema

This part is fun. Right-click on the project in the solution explorer and choose “Add new item”. Then from the dialog box (if you have installed the XML to Schema Inference Wizard), you will see at the bottom an item called “Xml to Schema”. Select that, and for the purposes of our demo, name it “PlantCatalog.xsd”.

The wizard will run and the following dialog box is displayed:

Note that you can infer the schema with this wizard from:

- A file on your disk
- A location on the web
- The contents of your clipboard (Add as XML)

In our case, we will “Add from File”. When you click on the “Add from File” button, a dialog will appear, where you should navigate to the plant_catalog.xml file referred to earlier in this post:

The dialog will then list the file as an input to be processed by the wizard, as follows:

Click OK and you will see that a file, “PlantCatalog.xsd” is created in the solution explorer:

Open this file by double-clicking to review the contents:

The actual work of inferring and building the schema is done by a tool that has shipped with Visual Studio since VS.NET 2003, named xsd.exe, however, this is a command-line tool while the XML to Schema Inference Wizard we are using puts a nice UI over this step and also takes care of adding the output to the current project.

Note also that the “targetnamespace” property has been inferred from the xmlns property we added earlier.

Importing the Schema into Intellisense and writing code

Now that we have a schema ready to use, all we need to do is tell the VB9 compiler that we want to use it in our code. For this we use the standard “Imports” directive that you are familiar with when you want to import .NET namespaces.

Choose “” from the intellisense choices and then type a colon. This colon defines a separator where you define an alias you will use in your code to refer to as an alias for this namespace. We use “plants” as the alias. Finally, if you type “=”” you will see the intellisense suggest the namespace for the XML schema you have brought into your project:

I should note here that the steps we took in this example of adding the namespace and importing it are not required if you are working only with one XML document in your application that does not specify a namespace. However, many real-world cases will have more than one schema, so this is a good thing to know.

Accept this option and the close the statement with a closing quotation and greater-than sign. You are now ready to begin writing code. Let’s use the following example that shows XLINQ in action:

Note that in addition to the normal intellisense you have come to expect as part of your programming experience with Visual Studio that you also get intellisense in places where you are using XML literals in the code, for example:

And also here:

The “…” operator deserves a bit of explanation here, as it is new for VB9. Referring again to Meijer and Beckman’s paper, (in section 5.2, actually) this syntax is defined as representing the child axis of the current element. This is a form of syntactic sugar that returns all PLANT nodes in the plantsXML document, regardless of how deeply in the hierarchy they occur. It is another aspect unique to VB9 at this time that will save a lot of time for those of us reading XML in code.

Finally, what you will see, if you run your lovely application is the following output:

Levelling The Playing Field

barton@netspheres.net — Wed, 29 Aug 2007 08:00:00 GMT

While I am not an expert on the subject, in my tenure as a founder of the Bay Area .NET Users Group (Bay.NET), I am very much aware of cultural differences between programmers who work with the Visual Basic and C# languages. An excellent overview written by Nigel Shaw on these cultural differences can be found here.

Having lived in many places throughout the world I would consider myself as one who is most interested in culture and cultural differences. In my experiences, I have discovered in myself an inclination to find things I appreciate about other cultures (and people).

I think we all tend to look at differences, be they cultural, as in those who prefer Apple Computers to Windows or Linux and make some value judgments based on what camp we live in and what camp they live in.

My late friend Rich Gold put it beautifully in his biographical work The Plenitude when he said:

“During my life I put on and took off four hats of creativity: artist, scientist, designer and engineer. I think of each one as quite distinct with its own methods, world views, precedents, predecessors, style of dress, interior decorations, histories, vocabularies, alliances, likes, dislikes, prejudices, tools, techniques and demeanors. I can walk into an office and know immediately if it is a designer’s office or an engineer’s office. I instantaneously know if it is an artist’s loft or a scientist’s lab even if they are filled with the same digital tools. Actually it is only with great effort that I have begun thinking about them as hats; in some real way, for me, they are states of being as different as alligators and elephants.”

The one thing Rich made extremely clear about these cultures is that all of them are essential to create a great product.

What is true regarding Gold’s observations of the artist / scientist / designer / engineer cultures is also true of the Visual Basic and C# cultures (as well as those of all other programming languages): We need all of them to make truly great products.

Prior to and since its incorporation into the .NET framework, Visual Basic has been perceived by many as somewhat of an ugly stepchild by many programmers whose language of choice is not Visual Basic.

And that is why the new version of Visual Basic.NET – VB9, to be released as part of Visual Studio 2008 is so terribly exciting to me. It marks a powerful sea change in the Visual Basic language where the culture behind that language is bringing some exciting innovation to the .NET platform based through language-specific features.

I will be spending the next few blog posts, writing about, among other things, these new language features. They are features that are important for all .NET developers to know about so informed choices can be made regarding the approach used to build software applications.

XML Literals in VB9

The first VB9 feature I think it going to make me more productive as a programmer is the ability to work with XML literals directly in code. From a description, this may not seem like much, but it represents is a new paradigm of working with XML natively in a programming language. No longer do I need to learn an API to work with XML, but I can write XML in 2 lines of code, as follows:

That may not seem like much, but VB9 also supports embedded expressions that are evaluated at runtime within the XML literals, that, when combined with language integrated query (LINQ) to enable very powerful, yet simple constructions, such as:

The <%= ... > syntax above represents an embedded expression and you can nest these recursively as many levels as you need to represent the desired structure.

These are things you just cannot do in any other .NET language and will make a huge different in productivity for any job where the construction of XML is required.

The illusion of the perfect computer

barton@netspheres.net — Sun, 15 Jul 2007 08:00:00 GMT

I recently got a new computer preloaded with Windows Vista. I must say that when I got the machine I was extremely surprised at how well it worked and how easy it was for me to transfer my data to the new machine (Lenovo T60p). With all of the comments I had been hearing which indicated that many people felt that the Vista operating system was problematic, I was rather surprised at how smooth my transition was.

Alas, this was not to last, and what is rather ironic is that I have been subject to a number of issues in the past month that have been rather disruptive and taken some time to remedy and yet are not related specifically to any flaw in the operating system.

The first sign of trouble was when I shut the computer down after doing some work and found it would not start the next morning. I believe a file was (or files were) corrupted somehow on shutdown (although I do not know for sure) and were only able to get back to an operable state by restoring the OS from a recent rescue and recovery backup, which I do regularly. One nice part about this option is that one can restore the OS to the state in the backup without touching the state of user files. In other words, fix only the OS and leave my files alone. This worked like a charm, so I was pleased to see that the recovery tools worked as they should. But I was not yet through my ordeal. This was only the beginning of a folly of further system errors.

The next thing that happened is that I started to receive visits from the Blue Screen of Death. Now, I am an intense computer user, so when this happened, I naturally assumed it was because I was doing too many things, i.e. playing music, compiling code, having too many files open, etc. Yet, when this continued to occur even at times of minimal activity and began to interfere with system backups. I knew at that point that I really needed to get to the bottom of this.

The next issue occurred, rather ironically, when I was trying to fix this problem and ran the Lenovo system update tool. When I ran the update, the tool informed me of a “critical BIOS update”. Upon reading about this update, I found it interesting that the language Lenovo used to describe what the update was addressing was “a possible processor marginality” and “a potential source of unpredictable system behavior”. Further research on the Microsoft site was more telling: “You may receive a Stop error, or you may experience unpredictable system behavior”. Oh yes, I thought I had found the issue and so proceeded to apply this update. But something odd happened when the update was complete. The computer would not start and the BIOS screen did not come up. I knew immediately that the BIOS was either erased or corrupted, and so contacted Lenovo support, who promptly sent out a technician to replace the system board.

Phew! Well, that’s over, I thought. But it wasn’t. Now, not only was I still getting blue screens, but since they had replaced the system board, all sorts of additional ramifications emerged:

I had to re-activate Windows by calling Microsoft;
I found that the technician had not entered the machine type or serial number to my machine, making it impossible to use certain ThinkVantage software such as system update;
I had previously been using the security chip on the old system board to secure the computer – now the Client Security Solution was caught in a loop and I had to uninstall and reinstall the entire subsystem;
The antivirus program I had installed no longer worked & required reinstallation;

Then when the technician came out to update the system board he proceeded to tell me that he could not put the serial number in the computer because this was not supported under Windows Vista.

Surprisingly, I found that I did not get upset as I was presented with this continual assault on my computing platform. I just decided I would fix it all, piece by piece.

I called Lenovo and asked them about what the technician had told me. They informed me that he was incorrect and arranged to have another technician who understood that one had to boot from the program using a floppy (How passé! One would think that in this day in age we could at least boot off of USB flash memory for this kind of operation…) that enters the serial number rather than double-click it from the operating system.

Armed with my new serial number - embedded system board, I proceeded to troubleshoot the ongoing bluescreens. One nice feature of Vista is that it keeps a log of all system failures (separate from the event log) and associated files. So all the memory dumps were there and available for me to analyze with the Windows debugger. My research showed that the issue was with the wireless driver for the Lenovo / Atheros chipset on my system. My hypothesis is that either the Lenovo system update or Microsoft update saw the hardware and provided the wrong update to this driver, as I was able to confirm that a system update happened on the 18 of June, which is the date of the updated driver and the date that the bluescreen problem started. I was able to resolve the issue by removing the driver and then reinstalling the driver from a fresh download package.

I then noticed that when I tried to back up the computer using the new Microsoft backup program, I received an inormative message, “An error occurred. The following information may help you to resolve the problem: Catastrophic failure (0x8000FFFF)”. I could just imagine how an average user would respond to that message. Oh, thanks for that, very helpful!

I proceeded to check the event log and found that the NTFS file system reported errors and suggested that I run chkdsk. So I did. Or, at least I tried to. But no matter what I did, on reboot, chkdsk would not run. I am apparently not the only person to experience this issue. I figured out that if I booted into the recovery partition and then to the command prompt that I could run chkdsk and was able to run it successfully. It found quite a lot of errors and corrected these. I hypothesize now that this was the actual source of many of these issues, as the initial problem where the machine would not boot properly may have been caused by a bad write to the drive.

Having repaired the disk to pristine state, I rebooted. Oh dear. Now the system informed me that a Windows dll was corrupted and continuously popped up a message box to remind me. Hmm. Well, at least I was provided with the name of the dll, so back to the recovery console I went and performed a manual copy of that dll from the recover partition to c:\windows\system32. I rebooted, saw no errors, the event log looked clean (and still does several days later).

I have now gotten the machine back to a stable state and am back up on daily backups. Altogether, it has now taken me about one month to re-stabilize this system which started out as being so wonderfully smooth I could not believe it. Maybe that was the problem all along.

In any case, what is clear to me is that no computer is free of problems, and to expect that from anything in life is something I consider to be unrealistic. If a typical user faced these issues, I loathe to think what they would do without a full battery of technical support staff on hand to assist them. The lesson? The compelling illusion of the perfect computer is so pervasive we often forget that there are many possible responsibilities that come with operating a complex piece of machinery. It is ironic that the user interfaces we strive to build often support that illusion. And yes, we do want to make it easier for people to use computers, however, we must not ourselves be seduced by the metaphor.

The imminent programming language and accompanying cultural convergence

barton@netspheres.net — Thu, 31 May 2007 08:00:00 GMT

I have often noted with intense interest that people who program in particular languages seem also to fall into certain patterns of thinking about the problems they solve. Kenneth Iverson, the inventor of the APL programming language spoke to this point in his 1980 Turing award lecture entitled “Notation as a tool of thought”. The thesis of that paper is the general idea that “the advantages of executability and universality found in programming languages can be effectively combined, in a single coherent language…”.

Fast forward to 2007 – here we are in a world where there are not five, but hundreds, if not thousands of programming languages. For many who program in a specific language, the choice to learn a specific syntax and accompanying abstractions is a decision not taken lightly. Some even describe the experience in what might be termed a religious posture.

So about a month ago, Microsoft announced, through the blog of Jim Huginin, support for an extension to the .NET Common Language Runtime (CLR) to better support dynamic languages such a Python, Ruby, JavaScript (ECMAScript 3.0), and Visual Basic. Its name? The Dynamic Language Runtime, or DLR. Also, it is interesting to note that the source code for this project is being made freely available under permissive license on Microsoft’s codeplex.

It is also very interesting to note, in case you do not know, that Jim Hugunin wrote the Java implementation of Python known as Jython, wrote the compiler for a very interesting extension to Java that is helpful in managing change among large numbers of classes called AspectJ, and most recently has been responsible for the IronPython project, whose goal is to make the Python language fully available on the .NET platform.

And to add more into the mix, when we look at the new functional extensions to C# 3.0 that are an outgrowth of Anders Hejlberg’s LINQ project, it becomes clear that .NET is moving toward an unprecedented level of support for just about any modern programming language and style.

So, rather than reality following Iverson’s idea, it seems instead that the way things are turning out is that the .NET platform will support static and dynamic languages, which opens the door for many, many more programmers, already well skilled in their programming ways, to leverage the platform.

I have to confess I find the entire prospect extremely exciting. And it is clear to me that Hugunin is dedicated to including other language designers in the process of developing the DLR so that there is a “community process”. That is certainly the right thing to be doing in today’s climate.

Every language has its lexical structure, its syntax, its keywords, identifiers, and taken together, I argue that becoming proficient in a particular programming language is equivalent to taking on a particular view of the world. The view of that world is based very much on how the language abstracts the hardware and allows the programmer to conceptualize the system they are encoding.

So, as I ponder over these differences (let’s call them cultural differences for the moment) that exist between programmers that are influenced by the language, I am even more interested in how these cultural differences will be bridged, and even more importantly, how to create situations for us to look beyond our differences towards an even higher set of goals that could serve to motivate everyone to create better software, regardless of their language world view.

I think that these issues are rather important to look at, as people, not the languages, are the ones who actually move things forward.