ASP.NET Ajax: Query Strings & Manual JSON Serialization

Posted by Dave Bouwman | Posted in ASP.NET, Ajax | Posted on 30-04-2008

1

Recently I needed to parse a complex query string from a Url, and configure a map based on those settings. This parsing occurs in the server side Page_Load event, and the general process of parsing the Querystring NameValueCollection is straight forward.

The complex part comes from the fact that my application is heavily Ajax based – other than the initial markup, everything else is fetched from the server via HTTP GET’s to ASP.NET JSON web services. Still sounds good. But – here’s the trick. The query string is passed to the page, not the web service. I guess I could have tried to write all the parsing in javascript and handled it on the client side but there are some pretty complex options that involve database lookups, and I like me some MBUnit Row Tests, so server side was a better option.

Just use Session State…

The first though is – ok – I’ll parse the query string, and stuff the resultant “map settings” object into session, and have the client fetch that from the web service. Nice, but no cigar, since the web service can not access the HttpContext that the page is running in. So much for simple solutions.

ASP.NET Page Methods To the Rescue (or not!)

Next contender was ASP.NET “page methods” (aka web methods). These are static methods They should work, as they can access the HttpContext of the page. After parsing and stuffing the “map settings” into the session state, the client side code could call back to the “Page Method”, which would grab the map settings from the session and send back to the client. Sweet. After some head scratching and reviewing of sample code that does this, and a bunch of Googling, I found that Page Methods do not work on User Controls (ascx). Since I was creating a module for use in DotNetNuke, there was no way around using a user control. Time to get ugly…

 

Serialize to Json and Cram into the Page…

Yep – the old inject-it-as-it-renders trick. Ugly, but it works. So, in page load, I parse query string, create and populate the map settings class and then use the System.Web.Script.Serialization.JavascriptSerializer  to convert my object to JSON. Some people have reported issues with this serializer, but it’s been working smoothly for us so far. If you have problems, you can roll your own or use an open source option like JRock or JSON.Net. From there, I just emit it into the page as part of a script block:

//Serialize out to the browser as JSON

JavaScriptSerializer jss = new JavaScriptSerializer();

//Use this Replace operation to escape out the dates

//see: http://forums.asp.net/t/1070058.aspx post by dotnetsamurai from 5/4/2007

string json = jss.Serialize(initialMapSettings).Replace(@”\”, @”\\”);

string script = “<script>mapSettingsJson = ‘” + json + “‘;</script>”;

//Now send the json out

Page.ClientScript.RegisterClientScriptBlock(this.GetType(), “mapsettings”, script);

 

As alluded to the in the comments, I ran into all kinds of trouble with dates. Read the referenced forum article for the “why” behind the string replacement bit.

Then in the client side Page_Load (I’m using the MSAjax stuff, so we get a Page_Load that automatically linked into the client page lifecycle), we use the client side deserializer to convert our JSON string back into a handy dandy javascript object.

//Deserialize back into an object
mapSettings = Sys.Serialization.JavaScriptSerializer.deserialize(mapSettingsJson);

From here we just proceed with the rest of the page startup, and we get the correct map.

 

Note for .NET 3.5:

If you have .NET 3.5 installed, then JavaScriptSerializer is marked as deprecated, and suggests you use the DataContractJsonSerializer. And if you are using Complex types (like I am) you will need to add the [DataContract] and [DataMember] attributes to the classes being serialized. I tried both ways and for me they both worked.

DNN Module Packaging Wizard + Subversion

Posted by Dave Bouwman | Posted in Uncategorized | Posted on 30-04-2008

0

Just wrapping up a project that’s built on DotNetNuke, and was using the built in module packager to create a nice zip installation. Great. Only problem is that it packs up all my Subversion “_svn” folders as well as the actual files I want. Doh! Initially I just exported from my Subversion working copy to a new location, added that as a virtual directory in IIS, and used that instance to create the package. But this is a real pain when you need to release small patches and bug fixes, so this being open source and all, I dove into the code.

And there is a simple fix… assuming you have the entire source tree laying around ( you can download it from www.dotnetnuke.com if you don’t have it)

The module packager is the PaWriter class, which is in the DotNetNuke.Library project, under the Components/Modules path.

All you need to do is add two snippets of code to the two ParseFolder private methods:

Change:

Dim subFolders As DirectoryInfo() = folder.GetDirectories()

For Each subFolder As DirectoryInfo In subFolders

   If Not subFolder.Name.ToLower().Contains(“_sgbak”) Then

      ParseFolder(subFolder.FullName, rootPath)

   End If

Next

to:

‘Recursively parse the subFolders

Dim subFolders As DirectoryInfo() = folder.GetDirectories()

For Each subFolder As DirectoryInfo In subFolders

   If Not subFolder.Name.ToLower().Contains(“_sgbak”) And Not subFolder.Name.ToLower().Contains(“_svn”) Then

      ParseFolder(subFolder.FullName, rootPath)

   End If

Next

Once you’ve made this change, re-build the DotNetNuke.Library, and copy the DLL into the bin folder of your DNN instance (if you are working directly in the full source tree, just build the website).

Given that Subversion is so very widely used, it would be nice if the DNN team would add this (and .svn) to the “core” code for the packager.

Configuring and Debugging ASP.NET HTTPHandlers

Posted by Dave Bouwman | Posted in ASP.NET | Posted on 17-04-2008

4

I was doing some work with a database driven version of the Virtual Earth Tile Service HTTPHandler, and needed to do some debugging. I ran into some odd behavior which took some time to track down, so hopefully this saves someone else some time.

Background

To use a HttpHandler, you need to register it in your web.config. Here’s what I had :

<system.web>
      <httpHandlers>
        <add verb=”*” path=”*.ve” type=”DBTileHandler” />
      </httpHandlers>
    </system.web>

This just simply routes requests for anyoldthing.ve to the DBTileHandler class, which must implement IHTTPHandler. Very simple.

Debugging Handlers

Typically I open websites “from file” in Visual Studio as this seems to work a little more consistently with VisualSVN (I think this is some issue with my box, as others on my team have no problems opening via IIS). As a result, when I debug the site, it spins up the ASP.NET Development Webserver (aka “Cassini”), and binds to a random port (e.g 2323). Then the site launches on http://localhost:2323/site. All is well and you can step through your aspx pages.

However, you can’t debug http handlers by just starting up the site from Visual Studio. You need to change the Start Action of the project to “Don’t open a page. Wait for a request from an external application” as shown below.

asp-start-action

Then once you start the web site, and it builds, you need to use Debug –> Attache to Process, and select aspnet_wp.exe.

attach

At this point you can make requests to your handler, and breakpoints will be hit – curiously you attach to the aspnet_wp.exe process even if you are running the site through Cassini.

So, I was happily debugging my tile handler, and got everything worked out, tile were showing up in my VE Map and all was well in the world. Then I went to try it through IIS. Doh! No tiles!

Firebug showed me that the requests were correct, but they were returning with 404’s

firebug

I went back and fired it up in Cassini, and got tiles. IIS – no tiles. Cassini tiles. IIS no tiles.

This was quite odd – why would my handler work in one case and not the other? Clearly there is something different between the web servers but what?

To make a long dull story involving much Googling shorter, the answer is that Cassini automatically maps file extensions specified in the Http Handlers section of the web.config, but IIS requires that the file extension mapping be applied explicitly in IIS. Obviously.

Solution

There are two ways to solve this issue:

1) Add the .ve extension to IIS and map it to the aspnet_isapi.dll

or

2) change the extension in your configuration to a type already handled by ASP.NET – such as ashx or aspx.

I chose the latter because it will simplify the deployment of the site. All I did was tack “.aspx” on the end of the path attribute in my web.config and all was working again.

<system.web>
      <httpHandlers>
        <add verb=”*” path=”*.ve.aspx” type=”DBTileHandler” />
      </httpHandlers>
    </system.web>

Virtual Earth + National Geographic MetaLens Demo Site

Posted by Dave Bouwman | Posted in ASP.NET, Virtual Earth | Posted on 10-04-2008

0

The last two days we’ve been setting up a demo integrating the National Geographic MetaLens service with Virtual Earth. It’s now live and can be seen at: http://www.metalens.org/mlve/.

ngs

MetaLens is a geospatial content management and archival system that National Geographic uses to secure and manage it’s content. “Out of the box” it has a Flash interface, but it also has a nice REST interface. We built some server side code that requests the public assets in the current view, and then applies clustering. At this time, the site is only fetching images, but we expect to extend the demo to include audio and video as well.

Map Tools

What’s a map without some tools? The Zoom To tool allows you to hop over to a location where there is some data of interest.

ngs-zoomto

For Lake Tahoe, we have added in a Parcels layer that comes from an ArcDeveloper.net Tile Service.

ngs-parcels

For Vermont, we have a map from National Geographic being tiled using the same code base.

ngs-vermont

You can toggle these layers on and off using the Map Data command on the toolbar. We have also added a Base Map drop down that allows you to select which of the VE base layers to view.

ngs-basemap

 

What’s on the Server Side

The main app is a ASP.NET with a data provider class library that actually talks back to the Metalens service. I really like this pattern because it allows us to test the provide completely independently of the web site itself. Basically we just wire up some unit tests, and then via TestDriven.net they are automagically re-run when we build that class library. Sweetness.

MetaLens

MetaLens is built on a product called Clear Path Explorer (CPX) built by a local Fort Collins company called Clear Path Labs. Their web site talks more about CPX, and it’s API. From out brief experience working with it, it’s pretty nice.

Tile Services

Not surprisingly we are leveraging the ArcDeveloper.net Tile Service. At this time we are just pulling in 2 services, and both use the ArcGIS Server SOAP tile provider. The only wrinkle is that one service is hosted by DTS in our Fort Collins office, and the other is on a National Geographic server.

I have to give a big shout out to my colleague Mike Juniper who did an awesome job tracking down a bunch of CSS weirdness, and put the final polish on the site. We knocked this out in about 2 days start to live. Amazing what you can accomplish these days! There are still a few strange things in IE6, but that’s pretty much how it is with that browser. Works great in IE7 and FireFox.

What do you do? Geospatial Developer Survey

Posted by Dave Bouwman | Posted in .NET, ArcGIS Devt, Community | Posted on 08-04-2008

4

surveyOver the years I’ve seen a few “developer” surveys put out by various GIS organizations, and they have all been pretty bland. After talking to a few people at the Developer Summit about this, I thought I’d see if we could do a little better – thus the launch of the 2008 Geospatial Developer Survey.

Why are you doing this?

The main reason is that I’m interested in knowing more about how other people doing GIS/Geospatial development work. How many people view themselves as software engineers vs. occasional coders who hack some stuff together as needed. I want to know if people are exploring and adopting the tools, patterns and practices that are common place in main stream software development. And if not, why.

And beyond my curiosity, I’m also looking for some insight as to what topics people want to know more about. Although I have some ideas, lets hear from you. I get a fair number of requests to write more “fundamentals” type posts, and at the same time get requests for more on patterns, Inversion of Control and Unit testing. Since I don’t write this blog for a living, I need to focus my efforts, so I hope that this survey will help quantify which way I should lean in future postings.

Who should take the survey?

Anyone who writes any code that is geospatial in nature. From the Google Maps Mashup dudes, to those running some geoprocessing tasks with python. If you write code that has anything to do with a map, you’re the perfect person to take the survey. Be sure to pass this url around to everyone you work with who is a geospatial developer – the more data points we get, the better. Who knows – we may even be able to influence the content at some conferences…

How?

It’s simple – just follow this link to SurveyMonkey, and fill out the questions. Yes there are 30 questions, but they should be pretty quick – I’d guess no more than 10 minutes.

What Next?

I’ll be leaving they survey open until May 9th 2008. Then I’ll crunch the numbers and do a post or two about the results.

Link (for copy pasting to all your developer buddies):

http://www.surveymonkey.com/s.aspx?sm=cugzupf5rAG_2bIu5WJ_2f7rxQ_3d_3d

UPDATE 4/8 @ Noon: After recieveing some feedback on the questions, I have added more “Not Applicable” and “Other” options.


Developer Summit Podcast

Posted by Dave Bouwman | Posted in ArcGIS Server, Podcast | Posted on 01-04-2008

1

I did a podcast interview while out at the Developer Summit. It’s focused on the ArcGIS Server Tile Server project that I submitted to the code challenge. I give a high-level overview of the project, and talk a little about my ideas on designing for ArcGIS Server – big ups for the SOAP API, and the 9.3 REST API.

You can check it out here.

When Marketing Hype Meets Reality: ArcGIS Server

Posted by Dave Bouwman | Posted in Uncategorized | Posted on 01-04-2008

8

One issue I seem to run into a lot is the gap between client expectations of ArcGIS Server, and the reality. Although I did bring this up at the closing session of the Developer Summit, and we were told this would be addressed, I think it’s worth pointing out some specifics.

On the one hand we read “Fulfilling the Promise of Complete Enterprise GIS” in the Fall 2007 ArcNews issue.  Some quotes…

… to support fast and efficient visualization and analytics applications, regardless of the amount of data held within an organization.

Any model or tool authored in ArcGIS Desktop can be shared to a broad audience via ArcGIS Server…

It’s  typical of much of the marketing. Pushing the simple author-publish-consume model, in which Joe GIS can just take his map, and geoprocessing models and fire it up onto the web in a few short clicks. To be sure this makes a great marketing story… “Wow – I can do that! Lets buy ArcGIS Server!”

 

But what happens when reality steps in? When Joe GIS takes his map, and geoprocessing service and publishes it?

Typically the site is slow, the geoprocessing tasks randomly fail and take ages when they succeed, and the Joe GIS gets (understandably) negative feedback from his users and superiors. So he packs up his bags and heads out to the developer summit, with one agenda: make his site faster. He drops into the “Architecting ArcGIS Server Solutions for Performance and Scalability Session“. On slide 5th slide, the first golden nugget of truth is revealed:

ags-perf-slide5

The item in question is somewhat small on this slide, and it’s not spelled out directly, so let me help out – “Design your maps specifically for server deployment”.

Somehow this never makes it into the ArcNews articles or the demos. Nor do the items on the following few slides…

  • only use the ESRI Optimized symbology,
  • use annotation instead of labels,
  • skip the fancy cartography – including highway symbols
  • cache everything you can,
  • and only load a few layers.

So Joe GIS now realizes he’s going to have to re-work all his maps. But there’s got to be some good news about the geoprocessing right?

ags-perf-slide9 

Enter golden nugget of truth two – and nope. Seems Joe is going to have to do a bunch of pre-processing, and stage up his data differently. So much for just publishing the data as it is. What does this “Understand Performance Expectations” mean? Oh yeah – basically if it’s slow in desktop, it will be slow in server. And this last bit about only one instance can update data at a time? That does not sound too good. And Joe’s ESRI reps have been telling him to use geoprocessing to do everything, but if he uses Geoprocessing his site can’t handle all 5 of his users at once! Maybe it’s time to take a look at .NET and that ADF thing…

 

I could go on through the entire presentation, and point out the various inconsistencies between what we see in the ArcGIS Server marketing and the reality (aside: check out slide 28 where they tell you to disable seamless panning and drop the overview map – how many demos have you seen where seamless panning is disabled?!)

What’s really interesting is this is ESRI telling you both things!  Right hand meet left hand. Marketing meet the technical staff. Take each other out for lunch. Talk. Listen. Learn. ArcGIS Server is a great product – it can do lots of great things, but it’s not a point and shoot camera. It’s a serious SLR, that needs professional skill to operate effectively and efficiently.

Until the large scale marketing shift happens, here are a few suggestions for the ESRI technical marketing people out talking to clients:

  • Be up front about what ArcGIS server can do, and what it can’t.
  • Be up front about having to specifically author maps for server, and the limitations on cartography, number of layers and caching.
  • Be up front about having to pre-process data used in geoprocessing tasks, and concurrency limitations.
  • Be up-front about the fact that to do much beyond pan and zoom, there will be some coding needed, and the water gets deep fast.
  • Be up front about the server loads and the licensing costs to scale out.

Then show some totally kick-ass demos. Show something truly amazing, and be rock stars about it, but just don’t lip synch. Be up front about the reality of running ArcGIS Server, because every single one of your users are going to experience this reality. Set customer expectations realistically, and everyone wins.

Link to the presentation listed above