Just catching up on my blog reading, and read some posts about Web 2.0 and “Labs” by Adena at AllPoints
and James Fee. One thing I think they both miss is what makes a “Labs” effort successful: customer feedback.

It’s all about the money…
The bottom line is that companies are trying to sell something – in
some cases it’s software (ESRI, Microsoft), in others it’s the
advertising that wraps around the software (Google, Yahoo). In order to
have people want the software it must be usable. The best way to
detemine the REAL usability of something is to have people use it, and
provide feedback. (While this should be obvious, it took the industry a
long time to learn this).

Release Early and Often (to the Lab)
This feeds directly into a major concept in both Agile development and Web 2.0: “release early and,
release often.”
What’s implied in that statement is that between releases, there is a
learning period where the company changes the software to address
usability issues, and incrementally move the software forward. This is
what makes a “Labs” effort successful. The result is a collaboration
between the company and the clients – which is a whole lot more “2.0″
than a monolithic release followed by a few years of vaccum, possibly
punctuated by a service pack or two.

What about Beta Programs?
Labs should be different from a beta program. A beta program is
basically a way to expand your testing team. The beta participants do
not get to help set the direction of the
software, rather they try the software in existing workflows to locate
bugs which may not have been found in the standard regression testing.
Software released into the lab, while possibly incomplete, should not
be “beta” quality. The idea is to have the working part “work” and to
get input on improvments and additional functions. The goal is to have
your customers help refine your vision of what they want. This is
really helpful unless you have a team of psychics who already “know”
what the users want. (note: regardless of what they say, the marketing
team is not psychic.)

ESRI Labs (beyond ArcWebServices)
Granted they have the ArcWebServices Labs, but it’s pretty small
potatoes in the whole Arc spectrum. What would really turn things on
their head would be a Labs area relating to the core software. This
would be very interesting, and would involve a major shift in the level
of transparency. Right now, it’s pretty reasonable to describe the
company as an “Ivory Tower”. Once (or twice) a year there is the
week-long love fest, where it’s all about the customer and everything
is on the table. And then it’s back to Redlands and all we see if the
“Wall of Marketing” for the rest of the year. Sure they have
publications, and trade shows, and betas, but all this is either driven
by marketing, or weighed down by a giant NDA. There are some brave
souls who do blog, but those few voices in the wilderness are a far cry
from transparency. The recent effort towards having a blog seems stilted – it’s unclear if this is a group blog, a marketing effort, or (shudder) user-generated-content. what. And thus far, it’s pretty fluffy for  the 800lb gorilla of the spatial industry. Back to the Labs thing…

Until ESRI can have a consistent on-going real conversation with
their customers, I skeptical they can embrace a real “Lab”, where rapid
customer feedback cycles drive the direction of the software.

Lets face it ArcGIS Server is not fast. Any there are many processes (raster modeling, map production, data extraction etc) that simply can not be completed in the typical web request response time frame.

In a recent project, I was creating 36 by 36 inch PDF maps, which was taking around 3 minutes (DRG raster layers really slow this down).

My plan was to create an asynchronous web service which we could simply send requests to, and the client would just continue on, and not wait for a response. The web service itself would notify the end user, via email, when the map was ready to be picked up.

Web Method Attributes

In a generic purely .NET situation, this is relatively easy to do just add the SoapDocumentMethod attribute to the web method

When this attribute is in place, the web server returns a HTTP 202 response, which indicates that processing has begun. This occurs prior to the web method being invoked. Whats interesting is that it seems that the ASP.NET impersonation does not work in this situation. When it comes to the point of actually connecting to AGS I get an UnauthorizedAccessException. I added in a little code to show me what the current identify was (WindowsIdentity.GetCurrent().Name) and it was my local ASPNET account. If I change back to SoapDocumentMethod(OneWay=false) the impersonation is working correctly, and right before connecting, I could see the current identity was the impersonated identity that I specified in the web.config file.

Ok so simply setting SoapDocumentMethod(OneWay=true) is not going to cut it. Time to dig into threading

Asynchronous Threads
Threading is very cool, and a very hot topic now that multi-core processors are becomming the norm. But, the usual usage is to have some work occur on a background thread, which then notifies the origniating thread that the work is complete. This is not what I want – I want to create a thread, and let it do its work and have it correctly kill itself off. I do not want to have the originating thread wait for it to complete.

Early on in my Googling, I found Mike Woodrings sample that shows how to support Fire-and-Forget for Asynchronous Delegates without leaking. Nice name! Why is this needed you may ask?

Starting with the 1.1 release of the .NET Framework, the SDK docs now carry a caution that mandates calling EndInvoke on delegates you’ve called BeginInvoke on in order to avoid potential leaks. This means you cannot simply “fire-and-forget” a call to BeginInvoke without the risk of running the risk of causing problems. (from Mike’s site)

Mikes sample code shows exactly how to do this very elegant, and best of all it works.  The idea here is that I can call the web service, where Ive created a delegate to a private method, which actually does the work of creating the map (ok, it calls other classes, but I’m trying to keep this simple!).  Mikes AsyncHelper class will create a thread and call the delegate, and it will correctly call EndInvoke to clean up memory for me. As soon as FireAndForget is called, the WebMethod exits, and the client can continue. Very nice.

Heres the web method code:

As you can see, the code is pretty simple, so it was quick to implement, but I ran into the same problem the identity on the new thread was ASPNET, and not the impersonation account. But this time I was in a better situation. The impersonation was in effect prior to calling AsynchHelper.FireAndForget thus I could get the identity without having to dig into the Win32 LogOnUser mess. All I needed to do was create a wrapper function for my real CreateMap function which would take an additional argument: the Identity.

So heres the new Web Method, and the wrapper function

The CreateMapAsUser simply impersonates the passed in identity, and then calls the CreateMap function. As long as the impersonation identity has access to ArcGIS Server, all is well.

Summary
If you need to create long-running AGS based web services which can handle their own notification, this is a really good solution. I also tested its scalability by firing off requests for ~ 20 maps at a time from my unit tests. And it (slowly) ground through all of them!

Last Thursday I gave a presentation on Unit Testing with ArcObjects at the Fort Collins ArcDeverloper User Group. The presentation gives an overview of unit testing in general, how ArcObjects complicates things, and reviews one methodology that can be used for some scenarios (geometry for instance).

The session went pretty well, given the relatively dry nature of unit testing. For this presentation, I refactored the ArcUnit code, mainly creating an ArcDeveloper.Utilities namespace and assembly. The code in the unit tests code now uses the utility classes. I think this is a better organization, and provides a framework that we can grow as a community.

The next steps (when I find time) are to get a source forge project set up, and then to port these classes to C#.

This presentation and the associated code is now available on ArcDeveloper.net

Presentation Link

Source Code Link