On August 14 Roundarch had the privilege of sponsoring the American Red Cross Mission: Red Experience Auction. The event was held at the Chicago Cultural Center and hosted by Kristyn Hartman who serves as a general assignment reporter for Chicago’s CBS2 weekday newscasts. We were happy to be able to sponsor an Experience Auction item and support such an important organization which serves the greater Chicagoland area. James Lazar, vice president of Roundarch, presented the “Mr. Smith Goes to Springfield” package to Mr. Eugene Varnado for his winning bid. The package was an invite from Illinois State Treasurer Alexi Giannoulias to experience Illinois’ state capital, Springfield. The package included tours of the Illinois State Capitol buildings, tickets to the 2009 Illinois State Fair and accommodations at the President Abraham Lincoln Hotel.

Kristyn Hartman and James Lazar

Thanks to the generosity of all the attendees, the event raised more than $33,000 to support the lifesaving services of the American Red Cross of Greater Chicago. In one recent weekend the Red Cross responded to 10 disasters and provided food, shelter, clothing and emotional support to 38 people in need. This is the work the Red Cross does everyday and these events help to make this type of support possible.

Additional photos can be found here and here.

Overview
Over the last twelve years of my experience in IT, I have focused ten of them in the segment of Content Management, and have seen as many changes and dynamics as the internet itself. For purposes of this writing, I’d like to focus on some key differentiators of open-source solutions and enterprise systems, as there is a fair amount of gaps and overlap between what they provide, who they serve, and their roadmap in the future of the content management paradigm…assuming we all know what a content management system should provide. Another key factor is the size of the organization. Larger organizations may have hundreds of users, making the administration tasks a difficult comparison to a small organization, where users can be setup and administered relatively quickly, regardless of the application. Granted that most of my time has been involved with implementing and improving on applications provided by a vendor (Interwoven), I was pleasantly surprised to see how quick and easy it was to rollout an open-source product (Drupal) with minimal know-how of the underlying technologies to make it work.

Installation, Configuration, and what comes OOTB (Out of the Box)
Perhaps I’m putting it too simple, but differences in installing and configuring almost came down to following a 10-page INSTALL.txt vs. a 100-page Installation_Guide.pdf. However, this may not be a fair comparison, since most large organizations will have a separate system administration/information security group within IT to perform installation tasks. Both have prerequisites before proceeding with the installation itself. If the tasks comprising the installation are distributed, they will collectively be more difficult than that of a smaller open-source installation. Installing an open-source application such as Drupal would more likely be done by the one-man system administration group, who has all the access and change controls needed to perform the install.

Once the application is installed, there’s the arduous task of setting up all the users. As stated earlier, the mere size of an organization can determine whether it’s really arduous or not, however, the task alone has had customers asking if they could use generic accounts…typically known as a bad practice from an information security perspective. Content management systems vary by supporting authentication models, but a larger organization implementing an enterprise solution will likely have the user administration delegated to other groups. Adding users in both TeamSite and Drupal is a snap, since the latest release of TeamSite supports non-OS users. Both are also very granular with what a configured user can see or do.

Drupal has an impressive set of modules that make up its core, such as configurable menus, themes, blogs, search, tracking, books, user-generated comments, and a forum. Based on these modules alone, one can gather that the nature of sites intended to be managed by the Drupal framework are geared more towards blogging sites and forums, or any other types accepting user-generated content. Interwoven’s products, on the other hand, have many features and some examples available, but with the exception of workflows, there are few templates or content types (forms) that are available with a fresh installation. Development would be required to create content types, as well as updates to those forms.  Creating new content types in Drupal is also very straightforward, assuming you’ve added and enabled the CCK  (Content Construction Kit) fields module. Updating (such as reordering or grouping fields) is also very simple in Drupal, and requires little or no development.

Drupal does offer many modules from their site to download and install to your system, which makes it very modular. There are some gotchas here, however, since you may find the module you’re looking for only to find out that it’s an alpha or beta version, or worse yet, not compatible with the Drupal version you’re running! Interwoven is compatible with most of its components, even forward compatible with its deployment mechanism. However, I have been made aware of its PERL versions not necessarily compatible with more recent versions. Because their direction over the last several years has been to move away from PERL and more towards a web services application, this becomes a moot point.

Administration
Both applications have a pretty good handle on administering users. Adding users from an LDAP source and then to a configured role is very straightforward in TeamSite. Drupal is also very straightforward, and stores user information in the Drupal database. From a system administration perspective, however, I see a short-coming in the way that Drupal has its super-user setup. There is only one true admin on the system, where in TeamSite, you can have multiple “master” roles, which I see as being more practical since you don’t want to have the one admin tied to one user. You can, however, delegate administrative capabilities to other users, essentially making *almost* admin users. Once users and roles are setup, it’s easy on both systems to delegate responsibilities, which is something that ought to be part of any content management system that may have many users.

A nice feature of Drupal is the management of themes…a theme can be configured to run for certain roles, and additional themes can be downloaded from the drupal web site. TeamSite requires that the UI be rebuilt if you want to make any menu changes or update the look and feel of the application itself. However, Drupal is a different beast in that it doesn’t have the notion of managing content and then deploying it to a test or production site. It makes little distinction between the site and the CMS itself, so it was written in such a way to make the skinning of the application more flexible. TeamSite does provide a Content Services SDK with a rich set of web services, so with development effort you could potentially build your own interface on top of the TeamSite backend.

File types and versioning
TeamSite and its distribution mechanism (OpenDeploy) do a fantastic job of allowing users to upload (manual or in batch) any file type, whether it’s for web sites or any other channel. The connector to its industrial-strength digital asset management tool (MediaBin) is also noteworthy, giving the user the ability to dynamically create web-ready assets from a different repository. With its virtual file system, it’s also very simple to work in a typical hierarchical view, either in the user interface or in a mapping to the file system itself (via Windows Explorer.) In Drupal, additional modules have to be installed and enabled to do file uploads, and size restrictions are present based on PHP settings.

Both systems support versioning, but there are a few gotchas to the way Drupal handles versions. Most information is stored in a database, and if you have access to the database, there’s nothing to prevent you from changing the content of a previous (or current) version of a content node. Some may see this as an advantage, but in general I would consider this a “hole” since you may not be guaranteed that reverting to a previous version is the one you really want. In TeamSite, edits cannot be made directly to previous versions. A user has to first revert to previous version and then that becomes the new version, which can then be edited. Also, TeamSite tracks deltas on files, so only deltas are stored, which can save a considerable amount of volume in the repository. TeamSite also has the ability to rollback entire sites or parts of a site. From what I have seen in Drupal, versions can only be reverted back on a node-by-node basis, unless a restoration is done from a snapshot backup of the entire database, which may have other implications.

Migration
So now that you have either system up and running…now what? Any organization will have the need to load the CMS with some initial content, either automatically or manually, and rarely will it be able to proceed from scratch. More likely, there will be a fair amount of content that will need to be migrated into the system, with several iterations ensuring it’s correct. With Drupal, this poses a challenge since the database schema is not widely published. A developer/DBA will need to be involved with this task, since they will have to become educated on the relationships within the database, but once that has been done the content appears nicely on the site, based on my experience.

Since TeamSite allows for any file type, migration is relatively more straightforward for non-native formats. Creating content records need to be created in a certain file structure, following a data-type definition based on the template, and an extra step of setting attributes on files that are to be typical forms. Although this may sound like a lot of work, it’s really not and can be easily scripted using a number of command-line tools that are available with TeamSite.

Upgrades
I have been through a number of upgrades of TeamSite…I recall a version 4.5 in 2000 that had gone through about 5 upgrades by the time I left the organization that had implemented it. With the exception of one backend upgrade, most went relatively well if there weren’t many customizations. Other scripts, templates, and forms were also largely backward-compatible. Interwoven has also done a great job with implementing upgrade features that customers request, by reaching out to them directly and validating their features, as well as conducting summits and focus groups to gather feedback.

I have only upgraded to a minor release of Drupal, which has also run smoothly…but a major upgrade may be a different story. Because the non-core modules are from a developer community and open-source, you are at their mercy to upgrade the modules along with the version of Drupal to which they work. Therefore, careful considerations need to be taken when planning and executing a major upgrade version of Drupal.

Summary
Having spent some time in both systems, I’ve come to the conclusion that they both shine in most areas, and there’s no reason an organization shouldn’t be compelled to use either system, considering the cost of infrastructure, support, and upgrade paths. Drupal is well-suited for a small to mid-size organization that is looking to get a site up and running with a great set of features quickly and low-cost, but heed the upgrade. Drupal is also currently being used to manage high-profile sites for larger organizations, such as Lifetime, The Onion, and Nasa. Interwoven’s products are truly industrial-strength, and scale well with volume of content, users, and requirements. Interwoven has also recently integrated products with parent-company Autonomy, putting them in a position to deliver high-quality, marketing-rich sites to organizations that have a high stake in their eCommerce offerings.

A look at the user experience visualizing historical data with varying levels of granularity.

Introduction
In the world of rich internet applications a very common goal is taking not easily accessible or multi-point data and outputting reports and visualizations. One thing that rich internet applications do very well is to display data across a period of time, and allowing the user to select ranges of time. Usually the output consists of a numerical report grid and some accompanying visualization. Most if not all, reporting applications focus around this capability.

Recently I had the opportunity to work on a Search Engine Optimization application that is designed to accept analytical and keyword data, then output reports and visualizations. In addition, the application will guide the user in creating keyword segments, and in turn produce personas. These activities will ease the management of campaign by managing segments and personas rather than the possible tens of thousands of keywords. At this time, let’s focus on the first phase of the reporting business process.

The user will upload data to the application and then the application will output reports and visualizations. As we dove deeper we discovered that the analytical SEO Data could be made available to the user on daily, weekly, or monthly basis depending on the client’s business objectives or processes. Thus, the user could upload daily data or summary data for weeks or months at a time. The Keyword data would remain largely unchanged, unless completely new campaigns were created, or drastic changes were made.
The client’s business requirement for this phase is to have a flexible application to allow the user to upload any range of data, at any time.

Design Approach
Keeping in mind the client’s business requirements for flexible data importing and reporting, our design focused around a date range slider to control this reporting interaction. Sliders are intuitive to use and we have successfully implemented them in many of our solutions. We coupled the slider with a report granularity selector for daily, weekly, and monthly reporting. The image below is an example of date range slider:

At first glance, it seemed like a straightforward objective and we did not see any issues with the process nor our approach, but the following is a narrative of the events that took place during 2 days of working through this business requirement, and the use scenarios that resulted from it.

Discovery
As I began to lay all this out, I started to see that the interaction between the data available and the date range slider would be critical. At first I saw two options:
The first option, the date range slider would drive how the data was to be reported, but the data available would drive what reporting options are available to the user. The user could only select by the period the data is available.
The second option was to have all reporting options available regardless of what data is available and normalize the data when applicable. For example, if the user selected to report by day, but the only data available is by week, the application would average the values by day.
To help us through this exercise, we developed a series of user scenarios to test both approaches and weigh the pros and cons of each.

We used this data import condition for all user scenarios:
The user will import 7 daily files, then 3 weekly files for the next 21 days for the first month, and a monthly file for the next.

Visually represented the data would look like this:

User Scenario 1:
The user wishes to view daily, weekly, and monthly data.
We set up this scenario as starting point based on a fairly complex data import. The objective here was to see how the slider would behave for simple reporting requests.

The Issue(s):
How should weekly and monthly reporting options be displayed, if the user selects report by day? The user knows daily, weekly, and monthly data are available.

Our Response – Limit the Reporting Granularity:
We responded to this issue by limiting the reporting granularity selector with daily, weekly, and monthly options based on the data available. Therefore if the user selected daily data only the purple 7 data points would be available as shown in the image below. The same would apply for weekly and monthly data.

We could play around with some visual elements to either ghost the remaining selections, to show the data’s availability but will not be displayed based on the reporting granularity selected. This corresponds to possibly large amounts of data not being available based on the user’s selection. We felt this was a very limiting experience, and not a viable option.

User Scenario 2:
The user selects a period from the middle of the first of the first week to the middle of the next week. There is a mix of daily and week data.

The Issue:
How does the application visualize data with different levels of data granularity?

Our Response - Normalization:
If the user selected a reporting granularity lower than what is included in the data set, the application will average, where applicable, to normalize the data for the block of less granular data.

The image below shows the normalization for the weekly data based on the selected range for by day reporting. A simplified view is available on the right. If the user selected to report by week, the values would be normalized across the entire reporting period and look like the red block of week data.

This would allow the user to see all data in all granularities over the entire reporting period. If we display the previous slider rule as mentioned in user scenario 1 with the normalization options outlined in user scenario 2, the image below displays all the reporting options with and without normalization, which we may allow the user to toggle on and off. The image below displays the possible options.

As a result some charting would require a good amount of normalization and mislead the user in their evaluation. If we refer back to the supporting image for user scenario 1, days 4 through 7 may not be an accurate representation of that period. We felt normalization is a transformation of data, and does not lend itself to good user experience.  We did not feel comfortable proceeding with this option, and dismissed it.

Final Solution and Lesson Learnt – Trade Flexibility for Data Integrity
We found that to deliver a consistent experience, we need to limit some of the options presented to the user. So much of the UI will be driven by the data, and that any amount of data could be available, at any granularity. The primary issue is not adjusting or implementing new user interactions, we needed to address the primary issue of “garbage in/garbage out”. We saw finally that by allowing the user to import inconsistent data, we were trying to control the user experience, rather than fix the root cause.

The Final Solution
We felt that an adjustment to the data import business rule would get the user in the right direction. We concluded by only allowing the user to import data in one granularity and for one calendar period at a time. This means the user has to find one granularity and stick to that. In addition, the user’s import would have to complete a calendar period. The sweet spot is probably a weekly import, but our user interview also points to monthly uploads being common place. By implementing this rule, we would certainly address all the issues listed above but at the price of flexibility and user experience. Sure our application would not be able to bend over backwards to allow all permutations of data and thus report that back out, but it would be consistent and deliver value.

Author Acknowledgments
I would like to acknowledge Gaurishankar Krishnanan and the SEO team for the time and effort during this exercise.

HTML (”hypertext markup language”) is the core language that powers the World Wide Web. Any server side technology must in the end, display HTML in order for web browsers to be able to render web pages. HTML 5 is the latest revision of the HTML spec that is slowly being adopted by the different major browsers. There are some pretty exciting additions to the HTML spec and I wanted to go over a few and what they potentially mean for the future of the web.

New Tags
Structure Tags
New additions to the HTML markup family include tags like header, nav, article, section, and footer. From the visual perspective these probably won’t have a huge impact as users are unlikely to see the difference between pages built on the old DIV based layout and the new structure tags. However from the search engine perspective, these tags will make it easier for crawlers to distinguish between what’s the meat of the content and what is just fluff. Current DIV based layouts structure the content semantically through unique IDs and classes. We give these DIVs IDs and classnames like header, footer, post, etc in an attempt to classify and organize the markup. However these are abstract concepts from the browser’s perspective. If they wanted, there’s nothing stopping the web developer from naming the header something arcane like foobar or even call the footer something like nav_sub2. As you can imagine then, search engines and web crawlers must develop sophisticated algorithms to detect patterns in order to infer what’s a header or footer. However with these additions, the developer must clearly demarcate what section each HTML piece is, thus taking the guesswork out of the search engine. This has the benefit of potentially improving search engine results.

Canvas Tag
The new Canvas tag is exactly what is says it is - a blank canvas with infinite possibilities. Flash developers used to drawing pixels on the MovieClip object will immediately be able to relate. Essentially one would use the canvas tag to render any number of things from manipulated images, animation, or even 3D imagery. With the canvas tag, you could build applications such as a paint program, or a 3D slideshow without having to rely on Flash. As usage of the Canvas tag increases, you’ll see more animation and renderings that were typically done in Flash re-envisioned completely in HTML5 and Javascript. The drawback however is that whereas Flash is build once run everywhere that supports Flash, an HTML 5 solution would leave you vulnerable to browser compatibility issues.

Here’s a video of a Coverflow implementaion done purely in JS + the new Canvas tag

AV Tags
The Audio and Video tags promise to simplify the mess that is currently the state of the art when embedding video. Whereas before you had either the <embed> or the <object> tag depending on what browser you are using or maybe you just turned to a javascript based wrapper to handle your media needs, you can have a very simple video or audio tag. With the ubiquity of the Flash platform as a video player, I’m not sure this is going to make much difference. Sure this is a lot easier, but we really could have used this 10 years ago.

<video width="640" height="360" src="/demo/google_main.mp4?2"
autobuffer></video>

Web Worker
Think of Web Workers as threads - any jobs that can be computationally expensive and intensive. In the current model, a complex task on a webpage might bring the interactivity of the page to a crawl while it’s busy number crunching. A worker thread could be spawned off to do some intense client side crunching without bogging down the page. This is even more relevant in today’s time when so much is being offloaded to the front end UI with javascript libraries. A good candidate for web workers would be a browser based excel spreadsheet like Google Docs where number crunching on the client site is potentially very slow.

<script>
   var worker = new Worker('worker.js');
   worker.onmessage = function (event) {
     document.getElementById('result').textContent = event.data;
   };
</script>

Application cache
The Application cache allows web applications to function offline when it’s not connected to the Internet. Google Gears is an implementation of this. All the developer has to do is provide a manifest of files that the web application needs in order to function offline. I see this as a really great feature to make web apps more robust. With most webapps, if you lose connection, you most certainly lose whatever you were working on. I can see more and more applications handling loss of network connectivity more gracefully by taking advantage of the application cache.

All you need is this code snippet and a manifest file which lists all the files the application needs.

<html manifest="foobar.manifest">

Geolocation
The Geolocation API provides a scripting interface that lets the developer determine the user’s location (based on GPS or inferred from IP, Wifi, etc). The user must however allow the application to access that information. Although geolocation has been inferred by IP for a while now on the backend, we’re seeing an increase of functionality performed on the front end with AJAX and this is no different.

navigator.geolocation.getCurrentPosition(function(position));

Should I be using HTML5 tags?
Here’s a table outlining features I played around with in the latest browsers from Google, Apple and Mozilla.

* only certain formats
** sort of worked

As you can see, coverage on some of the new HTML 5 features is pretty good on Firefox and Safari. However with the audio and video tags, I did find that Firefox supports the open source codex Ogg Vorbis while Safari’s supports all the formats that Quicktime supports, naturally. So if you are looking to use some of the new HTML 5 features now, coverage on all the browsers is sketchy at best except for the Canvas tag. If you are trying to do video or audio, you’d best stick to Flash. I think where HTML5 is useful for the here and now is in the mobile sector. Many new mobile OSes including iPhone OS 3.0 and webOS have started supporting some of the HTML 5 features and since you would be developing platform specific apps, compatiblity issues are non issues.