Agile .NET CT

Answer: Very agile, if you are Visual Studio Team System. But if you are the Pro or Standard Editions, you are not all that Agile. Or at least, not much more Agile than Vs 2005.

Which makes sense since Microsoft wants to sell lots and lots of Visual Studio Team System.

Our Connecticut Chapter of the Agile Project Leader’s Network is taking a look at VSTS and also open source Agile tools for .NET development. It turns out that a great many developers are doing fine with these freebie tools, and I’ve talked to some that are downright religious about this. It looks like the .NET community with respect to Agile is starting to divide into camps. In this corner we have corporate purchasing power and the power of Microsoft marketing and branding. In the other corner, we have a set of disparate open-source tools that are free and seem to do pretty well.

I suspect the edge goes to Microsoft over time, and we’ll see as the story unfolds here.
We are doing these meetings on Agile tools starting in February 2008. If this topic interests you, be sure to visit our APLN (CT Chapter) Website for details on these meetings.

If you have not done so, please examine Part 01 of this series of posts.

You have to give MSFT a lot of credit for knowing their business. The way they are handling ORM is a case in point. Still smarting from attempting to dictate ORM approached via ObjectSpaces (see below), the company is now taking a more empirical (read: Agile) approach to ORM.

LINQ: The ‘under’ ORM

Since the debut of LINQ and DLINQ, there is a lot of spirited debate, pro and con, all over the web, on the following:

1. On whether LINQ and/or DLINQ is actually an ORM;
2. On which features are essential parts of the “killer” ORM architecture;
3. On which ORM approach makes the most sense, now (simple CRUD adapters, or much more);
4. On which present-day .NET ORM tools are the best, now (Nhibernate, Gentle, etc) ;
5. On whether the ORM problem can EVER be completely solved (ORM as “the Vietnam of Computer Science”);
6. And so on

For sure, MSFT intentionally creates this debate.

For sure, MSFT is tracking ALL of this discussion and debate, and ultimately plans OWN this ORM parade, not simply get in front of it. And they are clearly out front with LINQ.

LINQ: The ORM API

LINQ in .NET 3.5 is a mere starting point for MSFT in the overall strategy to do the following:

1. Establish and control the API layer for all .NET ORM products;
2. Initiate the debate on what are the best and most desired features for industrial-strength ORM products;
3. Enable ORM writers to implement these consensus best-approach features, via LINQ, the de facto API of .NET ORM;
4. Slowly and surely assimilate those best ideas (built on LINQ) into the ultimate Visual Studio ORM tool. It appears first in VS Team System.

Via LINQ, in one shot, MSFT has opened the debate, enabled ORM writers, and created, and captured control of, the API layer for ORM in .NET.

Microsoft is prompting you to use your imagination with LINQ– the “under-ORM”.

LINQ, the first-class language feature, is but the first step in an adaptive (read: “Agile”) strategy with respect to ORM. The company is smart enough to know that interest in ORM and the global consensus with respect to the best approach is just developing now. By taking the API approach of creating LINQ, and baking it into the languages, Microsoft has done it again !

This analysis makes my point with respect to the creation of this API layer we call LINQ. The emphasis below is mine:

From: http://msdn2.microsoft.com/en-us/library/Aa479863.aspx

“While Project LINQ doesn’t purport to be the “final answer” to all of the world’s object-relational mismatch problems, it does represent a significant shift in direction to solving the problem; instead of taking an approach that centers around code generation, or automated mapping based around metadata and type inference, both of which are exercises in slaving the relational model to an object-oriented one, Project LINQ instead chooses to elevate relations and queries as a first-class concept within language semantics and library-based extensions.”

Do you see why?

What Microsoft is doing here is leveraging the wisdom of crowds. According to this theory, groups are very smart, provided certain conditions exist. Those conditions are:

Diversity of opinion:

Each person should have private information even if it’s just an eccentric interpretation of the known facts.

Independence:

People’s opinions aren’t determined by the opinions of those around them.

Decentralization:

People are able to specialize and draw on local knowledge.

Aggregation:

Some mechanism exists for turning private judgments into a collective decision.

Certainly the global develop community has many of these characteristics. As such, why not leverage this crowd?

Keep in mind, MSFT learned the hard way with ObjectSpaces, and got smarter about how to approach ORM as a result of this adventure. With ObjectSpaces, MSFT attempted to use a command-and-control approach to define the “right” way to do ORM.

It didn’t take.

Microsoft’s New Empirical Approach to ORM

The debate about whether LINQ represents a full-blown object relational mapper is a direct result of how MSFT is entering this market. With the debut of LINQ, MSFT is drawing attention to ORM as a solvable problem. MSFT is actively encouraging global developer-community debate here. The irony is that the developer crowd itself is building consensus now, and providing MSFT with the very best ideas on what to do next with respect to ORM via LINQ and DLINQ.

Those who think the current crop of ORM vendors have nothing to worry about are at best misguided. The genie is out of the bottle, and the debate is in full swing. Microsoft has captured the ORM API layer, kicked off the debate, and is in the absolute best position to leverage the inevitable ORM consensus.

For ORM vendors, the barn is burning !

Real ORM is needed by nearly every developer when any OOP app gets beyond trivial size. MSFT is not about to let some other organization get the high ground here. The stakes are too high; ORM is the sweet spot in the developer space, now.

With LINQ, Microsoft has executed masterfully and is in great position to make the most of the developing global consensus about the right ORM approach.

What to do NOW About ORM

All developers have to do now is get familiar with LINQ and DLINQ to stay ahead of the ORM curve. Developers must:

1. Get familiar with ORM concepts and facilities;
2. Carefully observe the ORM debate;
3. Study LINQ. Learn about anonymous types, object initializers, extension methods, and lambda expressions. This article is a great place to start. (This link also appears below.)
4. Watch Microsoft and Visual Studio Team System carefully for emerging ORM trends and tools. This is where MSFT intent is most visible.

Summary

Eventually, an ORM consensus is going to emerge here.

MSFT knows this and is hanging ‘debate’ out there– in the form of LINQ and DLINQ.

These items get the discussion going, and provide MSFT with the global consensus—the so-called ‘wisdom of crowds’.

This diverse, independent, decentralized, aggregated developer crowd is literally telling Microsoft exactly what ORM features matter.

You can expect them to deliver.

LINQ and DLINQ also provide the API-level platform for building a comprehensive ORM stack that addresses Pareto’s 80-20 rule neatly in the developer space.

You have to hand it to Microsoft—they know the developer like no one else.

Bravo to Microsoft. Good job !

LINKS:

Part 01 of this series:

http://www.newtechusa.com/agile/blog/?p=11

ORM Overview

http://www.theserverside.net/tt/articles/showarticle.tss?id=ORMGuide

LINQ Overview:

http://amroamroamro.wordpress.com/2007/06/26/understanding-linq-c-an-article-describing-the-new-language-features-of-linq/

Other notable links:

http://weblogs.asp.net/fbouma/archive/2005/09/19/425534.aspx

http://blog.hackedbrain.com/archive/2005/09/19/3182.aspx

http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx

http://www.theserverside.net/news/thread.tss?thread_id=44161

http://www.hibernate.org/343.html

…

Agile methods can encourage disciplined and successful speculative development of software systems. Ideally, these systems generate profitable revenue for the enterprise that deploys the software.

But watch out—Agile can also encourage undisciplined and unsuccessful projects too. It turns out that you have to know the total expected reward of implementing the system to understand how to manage the risk of developing it.

If you go too far with a project, you can get caught committing substantial time and money to a system that delivers a marginal payback—or no payback at all. Let’s look at some of the “knowable” variables involved here:

1. Likely total benefit and rewards of the completed system
2. Likely time to complete
3. Likely money to complete
4. Likelyhood that the system described in (1) can be completed within the time/cost constraints described in (2) and (3)
5. The ‘timelyness’ constraint: if we cannot complete by a certain date, the system offers a much lower payback. Usually, this is because of the actual or expected actions of a competitor in the market you serve.
6. The recognized presence of other high-payback, low-cost opportunities

As you can see, this is a probability problem. The problem is complex and requires collaboration with the Product Owner. For example, the Team usually has some idea of the likely time and money to complete, and therefore the likely cost. This is especially true after you are a few iterations in. But– only the Product Owner knows the total list of likely rewards. In Part 01 of this series, the issue of ‘Expected Total Reward’ is examined in isolation of the other variables.

Expected Cash Reward

Expected Cash Reward is composed of cash and non-cash components.

Expected Cash Reward components include:

1. Profit from revenues that can be mapped directly to the software implementation.
2. Cost savings
1. Quantifyable improvements in efficiency
2. Elimination of wasteful processes
3. Reduction in staff

The expected cash rewards need to be completely quantified. This is important.

If you cannot define the expected cash rewards from implementation, why are you investing time, money and effort into the construction of this system?

If you do not know the expected cash payback, you cannot proceed. Period. If you proceed without knowing the cash payback numbers and useful life time horizon, you are literally gambling on software development.

Expected Non Cash Reward

Expected Reward often has several valuable non-cash components. Often, these non-cash rewards play a critical supporting role in generating the direct cash rewards. Often the non-cash rewards also play a supporting role in improving overall efficiency and therefore profit on existing and new revenues. For example, HR costs can be reduced and valuable business knowledge can be retained if a system directly improves morale and therefore employee retention. Encourage your Product Owner to take a real shot at assigning a cash value to this non-cash outcome. Non-cash rewards indirectly save expense or indirectly generate profit. These non-cash rewards must be quantified.

Non-cash rewards include:

1. Improved customer satisfaction
2. Improvements in competitive position
3. Staff retention improvements via improved morale
4. Improved quality in overall business processes

These non-cash rewards need to be assigned reasonable best-guess dollar-values based on a rational and conservative set of evaluation criteria. Ideally, the quantifiable Cash Reward justifies the entire project, and the Non-Cash Rewards are obtained ‘for free’. Since this is not often the case, it is important to quantify—to assign a number—to the non-cash rewards. This must be done as part of overall reward assessment and analysis. This careful analysis of the non-cash rewards may make the system worth developing.

When all the cash and non-cash rewards are identified, you now have the Expected Total Reward component. With this number, you can start developing several useful ratios that help you to define the maximum economic project cost, and other important values. This number helps you to describe your risk profile, the cost threshold for deciding when to fail fast, and a host of other important decisions. Without the Expected Total Reward, you are gambling with software development. If you Product Owner is unwilling to share this number with you, and you are all employed by the same company, this is an immediate red flag that the Product Owner is unwilling to collaborate with the Team in any meaningful way.

The Product Owner is responsible for executing the rewards analysis. No one else can do this analysis. Demand that they do it. Without the Total Rewards number, no responsible Product Owner can proceed. Without this number, no responsible Team can proceed.

This needs to be brought to the Product Owner’s attention. When the team is paying attention to the Total Reward, and the Cash and Non-Cash Reward components, it helps them to make tradeoffs that maximize the reward components. This discussion is also the basis and starting point for hyper-productive collaboration with the Product Owner.

By demanding the Total Rewards number, the Team is executing on the following essential tasks:

1. Focusing on delivering business value;
2. Demonstrating that focus to the Product Owner;
3. Focusing on collaboration with the Product Owner;
4. Calling the Product Owner to accountability regarding quantifiable business value;
5. Learning about the common-sense constraints that are likely to be imposed on the development effort by the Product Owner, via the business-value considerations.

When the Total Reward is described and detailed to the Team by the Product Owner, it can be immediately used in risk assessment analysis. More often that not, the decision to develop a piece of business software is very much like placing a bet. When you wager, you must wager at favorable odds; otherwise you are gambling.

Summary: Expected Total Rewards

Expected Total Rewards is the starting point for determining all sorts of useful ratios, cost thresholds, and the like. These derivative numbers help immensely to describe, define and manage risk.

The ‘Total Risk’ component is the subject of subsequent essays in this series.

Make sure your Product Owners can quantify the Cash and Non-Cash Rewards components of Expected Total Reward. As the Team, you absolutely need this number to do the job of collaborating with your Product Owner. If the Product Owner is unwilling or unable to provide you with this analysis, consider it a test that fails, and act accordingly.

Be responsible for asking for the Expected Total Rewards analysis from your Product Owner.

…

Lowell Lindstrom, being himself, at Agile2007.

This is the first of a series of “shameless friend-promotions” (SFPs) that I intend to feature on this blog from time to time.

The intent of this week’s shameless friend-promotion post is to bring Lowell Lindstrom to your attention. If you are looking for a great Certified ScrumMaster class, or a great Agile coach for your organization’s move to Agile, I hope you might consider Lowell Lindstrom’s and the Oobeya Group.

Lowell Lindstrom is a friend of mine. He is a long-time member of the Agile community, and a Certified ScrumMaster trainer with 7+ years of experience. He is also a very accomplished Agile coach.

Lowell has quite a resume, including:

• Organizing Chair of the very first Agile conference in North America, in 2001,
• Program Chair of Agile2004;
• A founder of the Agile Project Leaders Network;
• Deep experience as a Certified Scrum Trainer and Agile coach with over 7 years of experience.

Some time ago, I decide to get certified in Scrum, and I choose training in Chicago, with Lowell and his firm, the Oobeya Group. I learn a ton about Scrum leadership by attending this class, which is first-rate.

At this class, I learn that Lowell is a guy who is a deeply involved advocate of Agile and Scrum from the very beginning. Lowell has over 7 years of Scrum experience, both coaching and teaching those Certified Scrummaster classes.

Lowell is great at helping organizations “get” Agile. One of the great things about Lowell is how he can come in, get situated and in sync, and help enable your move into Agile. Lowell is very easy-going and easy to work with.

If you are looking for real ScrumMaster training and real Agile coaching, Lowell is a real, great person to consider. Learn more about him at www.oobeyagroup.com or send him an email at Lowell[at]oobeyagroup[dotcom].

See also:

Lowell Lindstrom profile at the Scrum Alliance web site

…

The predominant model for programming processes is OOP. The predominant model for describing data is the Relational database using SQL. SQL is based on sets, while OOP is not. Here is the so-called object relational impedance mismatch.

The next version of Visual Studio has this new feature called Link Integrated Query, or LINQ for short. It does many things. By far the biggest thing you can do with it has to do with object-relational mapping, ORM.

As such, LINQ represents a great leap forward in how we program applications. This is the killer feature in the new .NET languages. This is a very big deal because setting this up is very simple. LINQ is not a bag-attached DLL, or extension to .NET. The LINQ facility is what is called a first-class language feature, meaning that it has been incorporated into the language itself.

The new LINQ feature in the next version of Visual Studio is the killer programming feature. If you are a .NET developer looking stay on top of new things that are coming, this is an excellent place to pay attention. LINQ is changing the way we program data-driven business applications. Get going now with LINQ.

LINQ does lots of other things to, like allow you to use a SQL-like syntax to get at items in arrays, lists, stacks, and queues. This is why there is a lot of confusion about LINQ.

Don’t waste your time trying to learn every feature of LINQ. Instead, focus on how LINQ enables you to access your SQL data using OOP classes, and you’ll get it—and get productive—right away.

LINQ is Object Relational Mapping for .NET. Period. That’s all you need to understand to begin your study of this tool. Using LINQ for ORM is the highest and best use of LINQ. Focus on LINQ-enabled ORM and you are focusing on THE KILLER FEATURE in the next version of .NET technology.

Any vendor shipping ORM tools for .NET is really in a bad situation now. That’s because Microsoft is solving the ORM problem and getting the high ground in terms of leading the discussion about ORM—all in one shot. LINQ is killing .NET ORM vendors, right now. Any vendor that has an innovation in this space eventually finds that innovation incorporated under LINQ in the .NET languages. This is not good, or evil, it simply is. Microsoft is 100% correct here and seems to really be hitting their stride.

We can expect:

1. ORM vendors to deny that LINQ is actually ORM;
2. ORM vendors to howl and scream as ORM becomes integral in all of .NET;
3. Two types of .NET applications; those that use LINQ and those that do not;
4. A slow and steady recognition that the next version of Visual Studio has many cool features, but if you add them all up, they do not equal the KILLER feature—Link Integrated Query. LINQ defeats the ORM problem and in so doing, it creates …..
5. ….a WHOLE NEW WAY to write data-driven business applications.

See also:

Object Relational Mapping

The Object-Relational Impedance Mismatch

The Impedance Explained- GREAT stuff from Scott Ambler

LINQ in a nutshell

Scott Guthrie of MSFT on LINQ ORM

…

What makes a software development or project management practice Agile?

That is a question a lot of people want answered. Here is the answer: if the IT process in question is inherently empirical, it’s Agile. Otherwise, it’s not.

Empiricism is the basis of science. It’s also the philosophical foundation of what we commonly call the Enlightenment.

Empiricism is also an approach used by successful entrepreneurs to launch new businesses. The other kind of entreprenuers typically never quite get the hang of empiricism. Why? Because to be empirical, you have to be willing to quickly change your beliefs when clear evidence says it is the rational thing to do. Not too many people are willing to do this, which explains why there are so few successful entrepreneurs.

Empiricism is what Agile is all about.

Fred Books knew this when he suggested “Plan to throw the first one away” in the classic foundational Agile text, The Mythical Man Month. (The first revision of this book is written in the late 1960’s)

This makes a great litmus test for you, especially in the larger organization. Is this process Agile? It is– if it embraces Empiricism.

Larger orgs naturally gravitate towards defined processes, which are not empirical in approach.

As a result, what gets called Agile is actually quite often a defined and totally non-empirical process. This is unproductive in two ways.

First, you have a defined process where an empirical process is best. Secondly and perhaps more unproductive, you are calling a defined process ‘Agile’ when in fact it is not.

So next time you ask yourself if a process if really Agile, take a look and see if it supports an empirical approach.

Empiricism is the philosophical root of all Agile practice, so let’s become experts at understanding this philosophy.

Prediction in general works against an empirical approach. Once a prediction is made, it tends to color your perceptions. You may even go so far as to distort facts and reality to fit your prediction. Prediction can make you inflexible and rigid in terms of thought and action.

Investors do this all the time, routinely filtering news that does not validate a position, and actively seeking news that supports predictions and positions taken based on those predictions.

Software project sponsors do this all the time, routinely filtering ‘bad’ news about the project that does not validate the schedule.

The book Inattentional Blindess by Arien Mack is an interesting book. It’s a book on cognitive psychology. It says that:

1. There is no conscious perception without conscious attention. You build perception on what you attend to.

2. You tend to see what you expect to see, because you are paying attention to what you expect. This means you are NOT paying attention to what you DO NOT expect.

3. What you don’t pay attention to actually matters. Alot! You have almost zero chance of building perception on subjects you are not attending to.

This has big implications for Agile project leaders, and software project sponsors.

It also explains how entrepreneurs get success.

It also implies that ‘multi-tasking’ is a huge productivity drag for software developers.

For software developers, concentration is key. When thinking about a project’s design, new perceptions come from time spent thinking deeply about the project. If you are multitasking, your opportunity to think deeply about software problem solving is in serious jeopardy.

Managers who pretend that multitasking developers are productive are seriously deluding themselves. New advances in cognitive psychology say so. For this reason, I believe that 4 hours is the minimum block of time a developer needs to really do his best when working on a project. Any less than that is simply unproductive, because that developer cannot pay (conscious) attention long enough to gain new (conscious) insights about optimal solutions.

For Agile project sponsors and Scrum masters, IB plays a huge role. If you are busy predicting, before long those predictions begin to warp your perceptions. You notice what you pay attention to, at the expense of failing to notice additional information the environment is now offering. Predictions encourage a filtering and distortion of reality.

Since your predictions about the project are in fact judgments, these judgments manifest as held beliefs.

Before long you are believing your own PR and BS about your predictions: about the predicted project cost, the predicted delivery date and predicted complexity. Any valid, incoming information to the contrary is ignored. Reality is in fact distorted by your beliefs. This means you are missing huge new important pieces of information about the project.

What you have no perception on, you cannot respond and adapt to. Predictions block empirical perceptions and first-hand observations. This is why prediction is not a best practice in software project management. We must plan without predicting. This is the essence of Agile practice.

For entrepreneurs, IB is a sure path to ruin. Few successful entrepreneurs ever suffer from it. In fact, I am sure that effective entrepreneurs (with good track records) are less prone than average to hold rigid views about anything.

Entrepreneurs are good at dropping a belief that no longer works, especially when a better one comes along. If you study entrepreneurs, you find that a huge percentage of them do not experience big success with their original idea. But they do start. And they do pay attention.

And by paying attention, they eventually perceive a bigger, better, related opportunity. This all comes from paying attention, building perception by focusing, and being willing to adopt a new and better belief.

There is no conscious perception without attention. If you are busy defending and validating your predictions, it is hard to notice what is happening now.

Inattentional Blindness by Arien Mack is a fascinating and useful book, and of real interest to serious Agile practitioners.