As I’ve mentioned a number of times, I’m a big proponent of keeping solutions simple. While doing some reading over the weekend, I ran across this presentation (pdf file) by Neil Ford of ThoughtWorks. This was his keynote at eRubyCon in August 2008.

Along with a brief history of software development, and his theory that software productivity would massively increase if the worst 30% of developers were fired, he talks about the relationship between essential and accidental complexity and summarizes it like this: 

• Essential Complexity: We have a hard problem
• Accidental Complexity: We have made a problem hard

I think this is a really clean way to describe the difference – sure some software projects actually are complex. But many are not, and the challenge for developers and designers is to keep both kinds of projects as simple as they can be. If we use that as a guide, the resulting software will be much easier to build, debug, test and maintain.

Geospatial applications are inherently more “complex” than standard “forms over data” applications. Regardless of the business logic being applied, at the very least we have that pesky geometry to deal with. Add to this, complex spatial API’s, limited spatial support in many “forms over data” power tools (NHibernate, SubSonic, Entity Framework etc), and systems that believe that they should control ALL the data (yes geodatabase I’m looking at you). Thus, keeping things simple is even more important, and I’d go as far as saying it’s crucial for our industry.

We recently moved from using Exchange to Google Apps. The transition has gone really smoothly, but there have been a few hurdles – mainly surrounding synchronization of Mail and Calendar across multiple devices.

While the Google Apps are all web-based, accessing them in a browser is not always an option – particularly when on airplanes. Additionally – if you rely heavily on your calendar for reminders like I do, remembering to keep a browser open on your calendar page is just one step too many.

So – here’s my device situation:

Under Exchange, my notebook and workstation would connect to Exchange for email and calendar, and synchronization was automatic. My phone would pick up email via IMAP to Exchange, but the calendar was synched when I connected the phone to either PC via a USB cable. Overall pretty good, but the calendar synch to the phone was a little more involved than I’d like – why not synch over the air?

Anyhow, with the move to Google, I can of course access mail and calendar from all three via a web browser. This is fine for email on the PC’s, but a little less than optimal for the calendar. Needing to have a calendar open to get reminders is not good for me. Call me old school, but I like Outlook – as far as email and calendar go – it works for me. 

Connecting Outlook with Google Apps

Actually getting a single instance of  Outlook talking to GMail via POP or IMAP is pretty easy, and well documented. If you use IMAP, then you can have multiple clients systems connected and everything is synchronized for you.

The calendar is another matter. For PC’s you need to run the Google Calendar Synch, which will copy items between your local Outlook calendar and a Google Calendar. It does the sync on a scheduled basis, so there may be some lag, but since you are planning things in the future, a 10-30 minute lag should not be a big deal.

On my phone, I’m using Mobile Outlook connecting to IMAP for mail, along with GooSync keeps my mobile Calendar in line with my Google Calendar. What’s sweet it that this works bi-directionally – over the air – something which was not an option with our Exchange setup.

If you are thinking about jumping to Google Apps, I’d say go for it. The transition has been smooth and everything we need is working with a lot less headaches for our IT staff.

In this go round, we’ll look at the questions related to Design Patterns…

Design Patterns

The question was "When you hear the term ‘Design Patterns’ what comes to mind?".

A majority of people did get that GOF = Gang of Four = Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides the authors of the book "Design Patterns: Elements of Reusable Object-Oriented Software", which describes recurring solutions to common problems in software design. Thick and meaty, it’s highly regarded and worth reading if you have the time and inclination. Martin Fowler’s site also has heaps of good info, and you’re not chopping down trees to read it. However you slice it, using patters will make you a better developer, so dig in.

Pattern Adoption

I guess this one has a call to action for someone to setup a community site to document and evolve some geospatial patterns.

If people are interested, I can stand up a wiki at ArcDeveloper.net, but someone else will need to push this forward as I’ve got a full plate.

Model View Pattern Use

"Rate your use of Model View * Patterns" was the question.

For those not familiar with MV* – this refers to a whole family of patterns, related to the Model-View-Controller pattern (Model View Presenter, Supervising Controller, Passive View), and the main intent of these is to separate business logic, user interface and data communications. Use of these patterns are helpful/ critical in enabling robust unit testing. I’ll be writing more about patterns and unit testing in the coming months, but Martin Fowler has a (burly) overview of these patterns. You can also check out the ASP.NET MVC stuff to get the Microsoft take on this, or check out MonoRail, which is part of the Castle Project. And for a change of pace, you could check out Ruby on Rails which is rooted in this pattern.

Inversion of Control & Dependency Injection

Ok, this is really maxing out the pattern / architecture geek factor, but it’s worth asking simply because this is another one of those patterns which enable unit testing.

So, with nearly 75% not knowing what this is… succinctly put, Inversion of Control and Dependency Injection patterns allow you to decouple classes from each other. This allows you to test things independently, which is critical if you want to have good test coverage. I had a series of posts partially written about this while at Sanborn, but I did not get them all polished off before leaving, and since I don’t have access to the source code anymore, I’ll need to look for opportunities to show some of this. For what it’s worth, the ArcDeveloper.net projects both use the Castle Windsor IOC container.

Ok, I’ll leave it here for now. Next time I’ll finish it up with a look at Build Processes, Unit Testing, Refactoring, API Documentation and some choice comments.

Just a quick reminder that the Geospatial Developer Survey will be closing this Friday, May 9th.

We’re at 262 respondents who are describing themselves like this:

Hey this is all .NET Stuff!

I’ve gotten a few comments back along these lines, so I thought I’d try to clarify this a little. Yes, on questions where you get to select a particular implementation, most of the options will be .NET, with one exception - “Other”. I did not do this to slight other platforms, or indicate that .NET is somehow superior.  I live in .NET land, and I simply do not know a lot about non-.NET technologies. If you are doing automated builds with ant and maven, just pick Other and write it in (15 of you already have). I’ll add these into the mix when crunching the numbers.

When will we see the results?

Survey Monkey does a great job of crunching out some simple results, and I expect to post a few interesting things during the week of the 19th.

During the ramp up for my current enterprise GIS development project I created a few simple VB.NET sample
projects which illustrate some architectural practices I want the team to use.

The first one I’m going to share covers the creation of a plugin framework,
and the use of the Supervising Controller pattern to expose user interface logic
to unit tests. In order to keep the focus on the pattern, and not on the complexities
of what the code is doing, this sample does not use any ArcObjects.
It’s just a very simple example of how you could create “property page”
type functionality that can be extended via plugins.

What is a Plugin-Framework and why would I want one?

Using a plugin framework is a way to design an application so that other
functionality can be added at a later time without needing to modify the
original source code. Putting this in ArcGIS terms, this is similar to adding commands into
ArcMap via it’s plugin framework.

If you are building large complex
applications, supporting a plugin model essentially enables the system to be
extended in a very managed way. Thinking back to ArcGIS – by simply implementing ICommand, you
can add alot of functionality into their application, and ESRI does not have to give you the source code, nor do you need to manage a very complex C++ build process.

Supervising Controller Pattern

Just a quick warning – we’re diving into formal object oriented design patterns here. Previously known as Model View
Presenter, this is a common object oriented design pattern.  Martin Fowler (who invented/named it) has
officially
retired the “Model View Presenter” pattern, and created two new ones – Supervising
Controller and Passive View.
You can read up on the details of these patterns by following those links, but
the quick and dirty explanation is that they focus on extracting logic from an
User Interface (windows or web form) into “controller” classes. (A little
pattern nugget to impress your friends with: Autonomous
View is the official name for  the “put all the code in the form”
pattern) These controllers handle most of the User Interface logic – and
since they are regular old classes – they are much easier to write unit tests
for.

One of the prime reasons to use Supervising Controller is for
testability. Assuming the view is hard to test, by moving any complex logic into
the controller, we put the logic in a place that’s easier to test. – Martin
Fowler

At first blush this seems like more work – and it actually is more coding.
BUT - if you write tests for the controllers (not covered in the sample), it will make your code more
robust and manageable over the long term.

The Plugin Framework

The plugin part is pretty straight forward. We define an interface for the
plugin, and implement it in the classes which we are going to plug-into the
application.

This is exactly the same as implemeting ICommand. The difference is that
instead of registering a COM class with the windows registry, we add the plugin
by editing the host application’s config file…

In order to load the plugins into the application we create a configuration
section handler – which I’ve posted
about
in the
past. For more details, I’d suggest checking out this MSDN
article by Roy Osherove. I’d also suggest looking at the code, since there are some other interesting bits in there that I’m skipping over to keep this somewhat short.

 
The Host Application

The plugins must plug-into something – thus the host application. This is
simply a windows form that will list the registered plugins, and then show the
selected plugin interface on the right side of the form. The image below shows it running.

The Plugins & Supervising Controller

The plugins are located in the “ExamplePlugin” assembly. In fact there is only
one plugin – “EditorPlugin ” – I just register it twice. The pluing itself is a super simple text editor. As I metioned earlier, the plugins themselves
implement the Supervising Controller pattern. Typically this is used with
“forms”, but in order to be pluggable, we are working with user controls. And to be fun, I’m using using inherited user controls.

Inheriting the user control forces a common look and feel -
granted in this case, it’s very simple (just the group box), but in our real app,
it’s got a lot more going on.

To keep things clear, I have setup a standard naming convention – the UI (user control in this
case) is <somename>View, and the Controller is <somename>Controller.
The interface that forms the contract between the two is I<somename>View. Looking at the class model above, we can also see that the View implements IPropertyPluginView – which means that it’s the thing that actually plugs into the application.

So in this case we have:

• EditorPluginView
• EditorPluginController
• IEditorPluginView

Since the Controller must manage the UI, one way to setup a contract between
the two is using an Interface. The interface is implemented by the View, and
consumed by the Controller. The interface simply specifies the methods that the
Controller can call on the View (UpdateText and GetText in this case) and the
events that the View raises and that the Controller can handle (ContentChanged
and Store).

Run Time
When the form is initialized (IPropertyPluginView.Initialize()), it creates it’s controller class and passes in a reference to itself as an IEditorPlugin. The Controller then sinks the Events on the interface, reads the file from disk (if it exists) and updates the text on the form. When the user edits the text the IEditorPluginView.ContentChanged event is raised in the form, and the Controller is notified. When the user clicks the store button, the IEditorPluginView.Store event is raised and the Controller gets the content via IEditorPluginView.GetText, and it stores it away.

Even this simple example could be extended, but this is enough to show how it all works.

Conclusion

While this was a whirlwind of terms and concepts, I encourage you to download the code, and play with it. It’s actually quite easy to implement once you get the hang of it, and it can help make your applications much more testable. I find it much easier to look at real code than to just read the generic explanations of patterns, so I hope this helps if you are thinking about implemeting a plugin framework or Supervising Controller in .NET

 
Download the code

Other Reading:

The Polymorphic Podcast has a series of great screen casts and
audio clips that go over the details of the Model View patterns. I highly
recommend these specifically, and the podcast in general.

MSDN Article: Creating a Plugin
Framework. This is from 2003, and deals with ASP.NET but the concepts have
not changed.

A while back I posted about setting up my home source control server using Subversion. The original post basically listed the steps to getting it installed.

I was just reading Omar Shahine’s blog, and he noted that there is now a 1-Click Subversion setup package for Windows users. I had not heard about this, and thought I’d point this out to others. Installing subversion was pretty simple before, and now it’s virtually painless.

So – if you still have a list of reasons why you were not using source control, you can strike out “difficult to install” and “cost”.

One challenge to software developers is to keep software simple. As
developers, we are both comfortable with complexity (just look at the Visual
Studio interface) and have a tendency to create applications that are
complex. The thing is that the end users really just want to get their
jobs done. They want software that they don’t have to think about.
Software that is obvious and easy to use. They want one click not 5, and
short-cut keys are even better. Simple is better.

When a user is given software that helps them get their job done, they will
use it. If it’s also simple and easy to use, they will be happy. And a happy
customer is the simplest way to get more customers.

So – beyond the happy user, why should we want to build simple software? The
world is complex and does not always follow simple cut and dry rules. Shouldn’t
our software handle all the possible corner cases? If the complexity is a
necessary part of the business process, it will need to be addressed in the
software. But we want to avoid unnecessary functions that are
not business critical i.e. an RSS reader a parcel management system.

In the end, the level of simplification you can achieve in your
application does come down to negotiation with your client, but there are some
compelling reasons to avoid complexity. First, as noted above, simple software
is easier to use and more likely to be used. This is a big win right out of the
gate, especially if it is a new technology deployment. If it’s really easy to
use, the organization may also see significant productivity gains – which is a
huge win.

Additionally, designing simple software forces the team to focus
their effort on only the most critical elements of the system and on executing
them in the most effective and efficient manner. Keeping the software simple
makes it easier to write and easier to test. Corner cases and “extra” rarely
used functions tend to eat up a lot of time, add a lot of complexity, and make
maintenance and testing more difficult. Since maintenance of software can be 60% of
it’s total lifetime cost, keeping it simple can save a lot in the long run.
Finally – if it turns our that some areas of the application must address more
complex aspects of the business process, simple software is easy to extend.

Additional reading on ”Keeping it simple”

Conversation with Ward
Cunningham  creator of “wiki” - quote: “Simplicity is the shortest path to a
solution”

Agile and Extreme
Programming, which is focuses on skipping “big design up front” and writing
the simplest thing that can possibly work, and adding in complexity only when it
is absolutely necessary.

37Signals.com – a web company focused
on simple solutions. They build “elegant thoughtful products that do exactly
what you want, and nothing you don’t”. They have written a free e-book about their development
philosophy called “Getting
Real”

Joel Spolsky
(JoelOnSoftware.com) also has some thoughts on “Simple”

Steve McConnell – author of Code Complete in addition to a number of other great books on software development wrote this article on simplicity as a best practice back in 1996

[UPDATE: 12/29/06 - Stephan sent me a note re: my incorrect spelling of "choropleth". At this point, I don't have time to change the code and graphics in the post, so just ignore the "l".]

As promised in my last posting, here’s an example of creating a custom configuration section handler using VB.NET in an ASP.NET application (sample code is for VS2003).

In this example I needed to pull data from the config file to setup a chloropleth choropleth mapping tool. The information is pretty straight forward – what layers can be used, for each layer, what fields can be used, and what are the minimum and maximum values for those fields. The Xml is shown below…

The corresponding class model is shown below.

The ChloroplethConfig class holds an ArrayList (if this were done in 2.0 it would be a generic List<of>) of ChloroplethLayers, which in turn holds an ArrayList of ChloroplethFields. The handler reads the Xml and converts it into these classes. Different from the C# example, these classes have overloaded constructors which allow me to pass in the Xml node from the config file, and they parse themselves. This is a nice pattern in that it makes the IConfigurationSectionHandler.Create method short and sweet…

 Public Function Create(ByVal parent As Object, ByVal configContext As Object, ByVal node As XmlNode) As Object

       Dim config As ChloroplethConfig = New ChloroplethConfig

       config.LoadValues(node)

       Return config

 End Function

When you run the app, it simply loops over the collections and reports what it has loaded from the config file. Very simple, but a nice way to validate that you’re custom config handler is working.

Anyhow – I’m working on another post which will build on using Custom Configuration Sections to build plugin frameworks – but I’ll save that for another day.

UPDATE: This post got caught up in the tidal wave of blogger spam on PlanetGeospatial,
so I am re-posting it with todays date. I’m really interested in
hearing what other people think about this. However, in re-posting, dasBlog
seems to have misplaced the post – and along with it a comment – doh! For
those who subscribe directly to by RSS feed, sorry about the
repetition. — Dave

Rather I’d like to see some real live
examples that people can play with. You would not buy a car without test driving
it, so I’m not sure why we would expect any less from our software vendor.

Here are some ideas…

ArcGIS Server: Web ADF

Let’s see a real application with large,
complex data model – maybe based on the Local
Government data model. Show us an editing environment customized for non-GIS
pros who don’t know or care about versioning. Throw in some imagery from Image
Server and make sure access is managed via ASP.NET security. Sounds like a tall
order, but consider that Local Governments are a prime target market, and it’s
not that unreasonable.

From a developer perspective, I’d love
to read about how it was built, the best practices which lead to clean,
maintainable, testable code. And a glimpse behind the scenes at the hardware
that’s powering it. This does not need to be totally public – make people login
with their ESRI Global Account, or limit it further to EDN subscribers. Whatever
- but let’s see the beast in action so we know what we can expect.

ArcGIS Server: Geoprocessing

Although the “GP Solves all
Problems” rehetoric of last years DevSummit has somewhat died down, it would be
cool to see a real GP model – doing more that a trivial buffer operation or a
hill shade on a limited area, running in ArcGIS Server, accessible for EDN
developers. Get really daring and have it do some raster processing. 

Make the model itself downloadable so we
can see how you’ve built it, talk about how you have optimized it, what hardware
it’s running on, and what it’s really doing.

ArcGIS Server: OGC Services

Fire up some WMS/WFS services that have
real data volumes behind them (say the state of California).  Then let the
community hit it with whatever client they want to use and report requests per
second, data volumes and CPU utilization. If that level of access is a little
broad, show us how to limit access via a and restrict it to EDN subscribers.

ArcGIS Server: KML

If you’ve got the OGC Services up, add
on the KML service. Let people see what can be expected with this service. Of
course it’s not going to be as fast as Google, but be transparent about the
hardware, and load, and I think that the user base will follow along.

ArcGIS Explorer “Tasks”

With the launch of ArcGIS Explorer, it
would be nice to see some real “tasks” that do something a little heavier than
reverse geocoding or returning the Zip+4 for a point. How about a site analysis
within an AOI? Or a network trace? Something that’s actually using the
analytical power of ArcGIS. Something burly because that’s what people are going
to expect/want when they drop cash for the backend ArcGIS Server software that
will support it.

 
Overall, I’d really like to see
demos that set realistic expectations. To date, the ArcGIS Server 9.2 hype has
been reaching snake-oil levels:

• Publish maps by merely thinking about it!
• Shed pounds by sleeping!
• Code? Nah - use geoprocessing for everything!
• Cures hemhroids & rhumatism!
• Run ArcGIS Server, ArcSDE and your web site all on one box!

Some reality will help everyone manage
expectations.