Trick or treat? Group items by month

OrderedMonthsIt’s that time of the year again when the unnatural becomes the norm, so let’s continue the tradition started last year with the SPELL program. Our goal today will be to display list or library items grouped by month, as shown in the screenshot.

The deal is that we don’t want any custom code or workflow here, just the regular out of the box UI features. We’ll be creating two calculated columns, called Year and Month, where we’ll insert the appropriate formulas. I chose the “Modified” column for my sample formulas, but of course any other date column would work too.

The YEAR and MONTH functions

YearNumber

A quick review of the SharePoint date functions gives you YEAR and MONTH that should fit the bill:

Year:  =YEAR(Modified)
Month: =MONTH(Modified)

The result is less than satisfactory though (see screenshot), as the year is displayed with a thousands separator. and months are displayed as numbers.

The magical TEXT function

MixedMonthsFurther exploration will take you to the TEXT function. It is not very well documented in SharePoint, fortunately you can rely on the Excel documentation and come out with the following formulas:
Year:  =TEXT(Modified,"yyyy")
Month: =TEXT(Modified,"mmmm")

Still not happy with the result? Right, the months are displayed in alphabetical order, not sequential order, not yet an ideal experience for our end user.

So let’s pull our last trick, and use the following formula for the calculated month:
Month: =REPT(" ",13-MONTH(Modified))&TEXT(Modified,"mmmm")

You can see the final result live on this page.

What’s the trick? We are still relying on the out of the box alphabetical sorting, but to force the order, we are adding a bunch of white spaces before the month name. The calculated Month actually contains the following values (each _ represents a white space):
____________January
___________February
__________March

__November
_December

Now, why don’t we see these spaces on the Web page? What makes the magic work is that when you insert multiple spaces in a Web page, the html specification says that

user agents should collapse input white space sequences

That’s it!

If you want to get really fancy, you could even use the zero-width space character. The best part is that people who edit your formula won’t even understand the trick, as the zero-width space won’t be visible (there’s however a good chance that they break your cool formula).

To take this further

You can apply this trick to other situations. A typical example is a color code. The alphabetical order will give you Green-Red-Yellow or Amber-Green-Red, you can address that by adding the appropriate leading spaces.

Be careful with this technique though: even if the rendering looks fine, the spaces are indeed stored in the field, and this might break other customizations. So this trick is better kept in a calculated column that will be exclusively used for rendering purposes.

Quick Demo: Process Navigation for SharePoint Forms

ProcessSteps

This month I am releasing version 1.1 of the SPELL library, with two major module rewrites. One of them is the SPELL Form module, which allows to enhance the out of the box SharePoint forms, and now includes features such as cascading selections and inline navigation. I’ll have more code to share next month, but for now let me just share a quick demo.

When we refer to inline navigation, we usually think tabs, like what we’ve been accustomed to with the Easy Tabs. The SPELL Nav and SPELL Form modules will render tabs by default, but beyond that they also have the ability to integrate other designs. For today’s demo, I have chosen a type of navigation often found in forms, and usually referred to as “process navigation” or “process steps”, where each “tab” is displayed as an arrow.

Click here for the live demo. Note that such designs rely on css properties that might not work in older browsers.

For the record, I had to fake the form, as you cannot display the out of the box forms on an Office 365 public site (!).

Quick update: more pocketSOAP testing

RetrieveWebPartsThe other day, someone on twitter made fun of the name I chose for my demo library, pocketSOAP. That’s ok, and I am actually glad that the name matches the intent. As I have already said, pocketSOAP is just a toy to experiment with SharePoint Web Services.

Something else that sounds funny: people sometimes ask me which services are supported by pocketSOAP (see for example the twitter link above), and my answer is… I don’t know. At the end of the first pocketSOAP article, I listed a number of existing libraries that abstract SOAP services. All those libraries follow the same technique and hardcode the service call. pocketSOAP and its parent library SPELL are radically different: instead of having the information hardcoded, they go and fetch it on SharePoint. That explains why the libraries are so lightweight. By the way SPELL is the smartest of the two, and when it learns a new service it memorizes it for next time.

In the past few weeks, I had the opportunity to do more testing based on users’ requests. I confirmed that my libraries worked fine for the following calls:

  • Lists.asmx:
    • GetListItems: retrieves items from SharePoint lists
  • WebPartPages.asmx:
    • GetWebPartProperties2: collects the list of Web Parts present on the page and their properties
    • SaveWebPart2: saves changes made to the properties of a Web Part (e.g. its title)
  • Workflow.asmx:
    • GetTemplatesForItem: collects the list of workflows for a given list
    • StartWorkflow: allows to trigger a workflow

The code for WebPartPages and Workflow is more intricate than what I posted in my previous pocketSOAP articles, so I am not sharing it here. If you have registered to the SPELL interest list, you’ll receive two entry level demos with the August newsletter:

  • a Web Part Page Quick Editor (features the WebPartPages service)
  • a button that triggers workflows (features the Workflow service)

So far so good, all the services I have tested have responded just fine. Now waiting for the next challenge!

HTML Calculated Column + Client-Side Rendering

Five years after the first release, the HTML Calculated Column remains the most popular topic on this blog.

The original page has been visited more than 200,000 times. It is definitely outdated, and in recent years I have pushed several new variations of this technique. The most popular is the color coding solution posted in the SharePoint User Toolkit, backed by this tutorial.

The most frequent issue reported by users has been the upgrade from SharePoint 2007 to SharePoint 2010. This is actually all taken care of in the above links… but you need to read the fine print.

I plan to rewrite the instructions, especially as in the meantime Microsoft has pushed another version of SharePoint, and the SP 2010 update… doesn’t work with SP 2013! Well, there’s actually a simple fix for SP 2013, and “Panoone” posted it as a comment a couple days ago (@Panoone: thanks again! And let’s get in touch to discuss this further).

But that’s not all. SP 2013 brings a bunch of new client side technologies, and one of them works very well in our case: Client-Side Rendering.

What is Client-Side Rendering?

JSLinkEditYou can use Client-Side Rendering (CSR) in SharePoint to manipulate the rendering process of list views. Does this sound familiar? That’s exactly what my HTML Calculated Column has been doing for years! Except that now it is an official component integrated with list views. When you edit a Web Part, the very last option is the JS Link placeholder (see screenshot).

It will certainly take several weeks before I find time to update the SharePoint User Toolkit, so for those of you who are already familiar with both CSR and the HTML Calculated Column, let me share the code for SP 2013.

In the code, Calculated is the name of the calculated column.

var ccContext = {
  Templates: {
    Fields: {
      "Calculated": {"View": "<#=STSHtmlDecode(ctx.CurrentItem.Calculated)#>"}
    }
  }
};
SPClientTemplates.TemplateManager.RegisterTemplateOverrides(ccContext);

So easy, just 200 characters! STSHtmlDecode() is a JavaScript function provided by SharePoint that allows the conversion of the html string.

Of course, you could avoid using a calculated column, and build the html directly in the JS Link file. But then each rendering would require a different file. The beauty of the HTML Calculated Column is that one single JS Link file can support all the views in your site.

A warning!

Some time ago, I wrote about security risks when you use the HTML Calculated Column. As far as I can tell, the same warning applies to Client-Side Rendering. Handle with care!

CSR references

When Microsoft released SharePoint 2013, the documentation on Client-Side Rendering was quite poor. Fortunately SharePoint bloggers stepped in and you can find some nice posts to get familiar with CSR. Wes Preston’s blog is an excellent start.

So is the old way dead in SharePoint 2013?

Not quite yet! Not every view accepts Client-Side Rendering, and for example you’ll still find my tutorial helpful for calendar views.

More pocketSOAP examples: Pie and Bubble Charts

A couple weeks ago, I introduced pocketSOAP, an experimental JavaScript library that facilitates interactions with SharePoint SOAP services.

Today let me share two more examples built on SOAP and the templating engine, combined with Google Visualization Charts, to render charts from SharePoint lists.

If you have subscribed to the SPELL newsletter, you’ll find pocketSOAP in your mailbox this week. Keep in mind that it is an experimental library, only the full SPELL library is meant for production environments (cf. disclaimer in the previous post).

I am not expecting you to be impressed by the screenshots below. After all, I’ve been publishing client side charting demos in this blog since 2008, and since then many others have followed. The real kicker here is how compact and versatile the code is. Even if you are not interested in client side scripting for a production environment, a tool like SPELL/pocketSOAP could prove really useful in a prototyping phase, and save you many hours of hardcore (re)programming!

Example 1: Pie Chart
PieChart
For this first example, I used the same SharePoint list as in the previous post, with two columns:

  • the default “Title” for the item name
  • “Priority” as a choice field with 3 options (high/medium/low)

The code:

<div id="results"></div>
<script type="text/javascript" src="pocketSOAP.min.js"></script>
<script type="text/javascript" src="//www.google.com/jsapi"></script>

<script type="text/javascript">
// Load the Google charting library
google.load("visualization", "1", {packages:["corechart"]});
// Get items from the Projects list
var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestZone",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"Projects",
 viewFields:"<ViewFields><FieldRef Name='Title' /><FieldRef Name='Priority' /></ViewFields>"
});
pS.when(promisedItems)
.then(function(request){
 // When ready, run the charting function
 google.setOnLoadCallback(function(){drawChart(request);});
});
 function drawChart(request) {
 // The templating engine formats the data string according to Google format
 var dataString=pS.applyTemplate({
 template:'[["Project","Priority"][<|,["|Title|","|Priority|"]|>]]',
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z")
 });
 var dataTable = google.visualization.arrayToDataTable(JSON.parse(dataString));

// Group by priority. Priority is the second column (index = 1)
 var data = google.visualization.data.group(dataTable, [1], [{'column': 1, 'type': 'number', 'aggregation': google.visualization.data.count}]);
 var chartOptions = {'title':'Priority','width':400,'height':300,colors:['red','#FFC200','green']}; // #FFC200 is the color code for amber
 var chart = new google.visualization.PieChart(document.getElementById('results'));
 chart.draw(data, chartOptions);
}
</script>

The SOAP call is almost the same as in the previous post. The only difference is that I have explicitly told SharePoint which fields I want to retrieve, via the viewFields option. This is not mandatory but it will reduce the size of the dataset and improve performance.

The use of the templating pattern (pS.applyTemplate) to convert data formats is unusual – as a matter of fact I have never seen this used elsewhere. I am not advertising it as a best practice, and more traditional object manipulation could be used here to the same effect. But I really like how templating makes the code easier to maintain.

Templating is a very popular pattern in modern JavaScript, and can be found in many libraries, like jQuery, mustache, handlebars and others. The SPELL/pocketSOAP flavor might not be as powerful as those big names, but in just 1 kb of code it still has a lot to offer, and – as you would expect – it is well adapted to work with SharePoint Web Services, for both XML and JSON output (to use it with JSON instead of XML, simply replace the [< >] notation with [{ }]).

Example 2: Bubble Chart

BubbleChart

For this second example, I created a SharePoint list with 5 columns:

  • the default “Title” for the product name
  • “Risk” as a choice field with 3 options (low/medium/high)
  • “Cost” and “Revenue” are numbers
  • “Profit” is a calculated column:
    [Profit]=[Revenue]-{Cost]

The code:

<div id="results"></div>
<script type="text/javascript" src="pocketSOAP.min.js"></script>
<script type="text/javascript" src="//www.google.com/jsapi"></script>

<script type="text/javascript">
// Load the Google charting library
google.load("visualization", "1", {packages:["corechart"]});

// Get items from the PMO list
var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestSite",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"PMO"
});

pS.when(promisedItems)
.then(function(request){
 // When ready, run the charting function
 google.setOnLoadCallback(function(){drawChart(request);});
});

function drawChart(request) {
 // The templating engine formats the data string according to Google format
 var dataString=pS.applyTemplate({
 template:'[["Product","Cost","Revenue","Risk","Profit"][<|,["|Title|",|Cost|,|Revenue|,"|Risk|",|Profit|]|>]]',
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z"),
 sanitize:function(string){return string.replace(/float;#/,"");}
 });

var dataTable = google.visualization.arrayToDataTable(JSON.parse(dataString));

var chartOptions = {
 title: 'PMO',
 width:500,
 height:400,
 colors:['green','#FFC200','red'], // #FFC200 is the color code for amber
 legend:{position:'top'},
 hAxis:{title: 'Cost',maxValue:100},
 vAxis:{title: 'Revenue',maxValue:100}
 };

var chart = new google.visualization.BubbleChart(document.getElementById('results'));
 chart.draw(dataTable, chartOptions);
}
</script>

Note in the template the difference between text fields (for example “|Title|”) and number fields (for example |Cost|). Also, this time I had to include a sanitize function to remove the weird “float;#” string that SharePoint prepends to calculated columns.

The above code samples could easily be tweaked to work with other client side charting libraries, like Dojo, HighCharts, or [name-your-own]. If you are interested in a demo, leave a comment with the name of your favorite charting tool!

SPELL Tabs: commercial and free versions now available

This week, the SPELL framework is reaching version 1.0.0, and I am releasing the first component of this new version: the SPELL Tabs, successor of the Easy Tabs.

SPELL is not just about new or upgraded scripts, it also introduces a new way for end users to manage their solutions. In the past, the advertised, end-user friendly way to include solutions in a page was to link to code through a Content Editor Web Part. This is still an option in SPELL, but the default method is now to use… a Page Viewer Web Part! This offers end users a no-code experience, where each solution behaves like an “app” and comes with its own, client side edit menu.

Here is what the SPELL Tabs edit menu looks like:

TabsEditMenu

The commercial version is distributed by User Managed Solutions LLC. SPELL is a tentacular program, so feel free to contact me with your questions or to schedule a call. For now, SPELL is still in “early adopter” mode, and the final blocks – shopping cart, forum, videos – will be added in the next 2 months (early adopters naturally benefit from an extended subscription period).

For those who are migrating to SharePoint 2013 and are just looking for a way to fix the broken Easy Tabs, I am also making an entry level version of the SPELL Tabs available at no charge. As you might expect, features, terms of use and support are limited compared to the full version. If you are a subscriber of the SPELL newsletter, you’re already familiar with it, as it is the evaluation version you’ve been playing with in the past few weeks!

Teaser: pocketSOAP, an ultra lightweight library to interact with SharePoint SOAP services

[6/13/2013] Update: I have changed the terminology, from Future back to Promise, to conform to the living standard.

Disclaimer: the solution presented in this article is part of SPELL, a coaching program offered by User Managed Solutions LLC. Access to the full set of SPELL solutions requires a paid registration. However, the program also includes free evaluation samples like pocketSOAP, all you need to do to get them is show your interest by registering to the SPELL newsletter.

The SPELL newsletter currently has ~180 subscriber companies. The issue featuring pocketSOAP is scheduled for the first half of June.

For years, the SharePoint SOAP services have been a favorite among power users and front end developers. Their popularity is due to an accumulation of factors:

  • They are supported across all SharePoint versions
  • They offer a wide range of features, from simply accessing list items to building sites or managing permissions
  • They work in client side environment, and don’t require server side access.
  • They are well documented, thanks in particular to the work of SharePoint bloggers (cf. references at the end of this post)

Since the SP 2010 release, Microsoft has been embracing more and more the client side, and has pushed additional tools like CSOM, REST (listdata.svc, _api), and client side rendering for list views (JSLink). At the same time SOAP services have been deprecated (but are still supported in SP 2013). Developers are faced with a difficult choice: they’d like to adopt new technologies, but at the same time these technologies don’t offer (yet) the same coverage as the legacy SOAP services. And of course they don’t work in older versions.

The purpose of pocketSOAP is to showcase how I dealt with this dilemma in SPELL. When you want to interact with half a dozen interfaces, as SPELL does, you can’t afford a 100 kb footprint just for one! So here we go, 5 kb of fully functional code, no dependency! The list of pocketSOAP ingredients:

  • SOAP: ~ 1 kb
  • Ajax: ~ 0.4 kb
  • Promises: ~ 1 kb
  • Templating: ~ 1 kb
  • Various helper functions: ~ 1.6 kb

A simple example

Note: pocketSOAP targets developers and assumes that you are comfortable with JavaScript patterns such as ajax, promises and templating. If you are an end user, you might be more interested in user friendly samples like the SPELL Tabs or the SPELL Charts (and of course the SharePoint User Toolkit).

ProjectColorFor my simple example,  I have just created a custom list called “Projects”, with 2 columns:

  •  Title is a text field for the item name (Project 1, Project 2,…)
  •  Status is a choice field for the item status: green, yellow or red.

Expected result (cf. screenshot): color coded indicators that displays the health of each project.

The code (pS is the shorthand for pocketSOAP):

var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestSite",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"Projects"
});

pS.when(promisedItems).then(function(request){
 // Apply template
 document.getElementById("results").innerHTML=pS.applyHTMLTemplate({
 template:"<table>[<|<tr><td><div style='width:12px;height:12px;background-color:|Status|;'>&nbsp;</div></td><td>|Title|</td></tr>|>]</table>",
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z")
 });
});

How the code works:

  • pS.soap() handles the SOAP request (ajax). It needs two sets of options: information about the service itself (in our case Lists/GetListItems) and parameters specific to the call (in the case of GetListItems: the list name, and maybe specific filtering options and a shortlist of the fields to retrieve).
  • pS.when().then() is a promises pattern. It ensures that the SOAP request is completed before we move to the next step.
  • pS.applyHTMLTemplate() runs the templating engine, in this specific example to output a html string. In the template expression, [< >] means that we are dealing with a collection of nodes. If we were using the new REST interface and retrieving JSON, we would simply use  [{ }] instead of [< >].
  • the prefix ows_ is just here to address an oddity with SharePoint Web services, where “ows_” is prepended to each field name. So for example to get the Title field, you need to retrieve the attribute called ows_Title.
  • the data is the result of ajax xmlHttpRequest. If you are familiar with the SOAP service GetListItems, you know that each item is returned as a namespaced “z:row” node.

How many SOAP services does pocketSOAP support?

I don’t know. So far I have only tested it for half a dozen services representative of my scenarios (90% of real life use cases are about getting or updating list items). I plan to document this as I and the SPELL followers continue testing more services.

Can I use SPELL/pocketSOAP to aggregate values from multiple lists?

Yes. Thanks to the promises pattern, the script can collect data from multiple lists in parallel, then trigger the rendering when all the items have been collected. The pocketSOAP documentation will include such an example.

Note however that for a large number of lists, client side code might not be the best approach and you’ll want to consider other techniques.

How do I make a synchronous call?

You can’t, this option has been removed from SPELL last year. The choice is to embrace a modern JavaScript approach where remote calls are asynchronous (the first “a” in ajax), and dependencies are managed via promises: when(this is completed).then(do that).

In SPELL, not only ajax calls, but also script loading, iframe loading, document ready, and some other operations are treated as promises.

Did I go too far with pocketSOAP?

The answer is… yes! If you play with pocketSOAP, keep in mind that it is a laboratory rat. The SPELL library – the production version – offers much better balance between the load effort  and efficiency at runtime.

How is SPELL different from pocketSOAP?

1/ SPELL is for production, while pocketSOAP is an experiment. For example error handling has been reduced to a minimum in pocketSOAP.

2/ SPELL supports half a dozen interfaces, SOAP just being one of them. Apart from SOAP ($P.soap), it also relies on the RPC method ($P.rpc), the REST services ($P.listdata and $P._api), etc.

3/ SPELL offers not only core features aimed at developers, but also end user solutions like the SPELL Tabs and mini-BI.

How come SOAP has never been mentioned before on Path to SharePoint?

Until recently, I have always considered that SOAP was a heavy solution and should only be used in large scale implementations, or when everything else fails. For  the samples showcased in this blog and in the SharePoint User Toolkit, my preference went to the RPC method, a more straightforward and end user friendly approach.

However when I started building SPELL two years ago, SOAP was an essential building block in a projet of that size. The decision to include SOAP was even easier after I reduced the code size to just a couple kb!

References

The official Microsoft reference for SharePoint 2013 Web services can be found here.

The first to publish a JavaScript library to wrap SOAP services was Darren Johnstone in 2008. His original site doesn’t seem to exist anymore, to get the files try an internet search for “SPAPI”.

SharePoint superstar Jan Tielens published in 2009 a series of posts to showcase the use of SOAP services in conjunction with jQuery.

That same year, Marc Anderson released SPServices. This remains to date the most useful and comprehensive resource on the subject. When people look for documentation on the SOAP services, I usually point them to Marc’s Codeplex site rather than the official Microsoft documentation.

SOAP services are also a favorite of Alexander Bautz, and he uses them in some of his published work.

Alexander’s blog drove me to another library called SharePointPlus, developed by DELL employee Aymeric Kodono and released under a GPL v2 license.

There are certainly others I am not aware of. For example I haven’t tested this framework that seems to deal with the SOAP services (and btw is not maintained anymore). If you have more information feel free to post it in the comments!