Teaser: pocketSOAP, an ultra lightweight library to interact with SharePoint SOAP services

[6/13/2013] Update: I have changed the terminology, from Future back to Promise, to conform to the living standard.

Disclaimer: the solution presented in this article is part of SPELL, a coaching program offered by User Managed Solutions LLC. Access to the full set of SPELL solutions requires a paid registration. However, the program also includes free evaluation samples like pocketSOAP, all you need to do to get them is show your interest by registering to the SPELL newsletter.

The SPELL newsletter currently has ~180 subscriber companies. The issue featuring pocketSOAP is scheduled for the first half of June.

For years, the SharePoint SOAP services have been a favorite among power users and front end developers. Their popularity is due to an accumulation of factors:

  • They are supported across all SharePoint versions
  • They offer a wide range of features, from simply accessing list items to building sites or managing permissions
  • They work in client side environment, and don’t require server side access.
  • They are well documented, thanks in particular to the work of SharePoint bloggers (cf. references at the end of this post)

Since the SP 2010 release, Microsoft has been embracing more and more the client side, and has pushed additional tools like CSOM, REST (listdata.svc, _api), and client side rendering for list views (JSLink). At the same time SOAP services have been deprecated (but are still supported in SP 2013). Developers are faced with a difficult choice: they’d like to adopt new technologies, but at the same time these technologies don’t offer (yet) the same coverage as the legacy SOAP services. And of course they don’t work in older versions.

The purpose of pocketSOAP is to showcase how I dealt with this dilemma in SPELL. When you want to interact with half a dozen interfaces, as SPELL does, you can’t afford a 100 kb footprint just for one! So here we go, 5 kb of fully functional code, no dependency! The list of pocketSOAP ingredients:

  • SOAP: ~ 1 kb
  • Ajax: ~ 0.4 kb
  • Promises: ~ 1 kb
  • Templating: ~ 1 kb
  • Various helper functions: ~ 1.6 kb

A simple example

Note: pocketSOAP targets developers and assumes that you are comfortable with JavaScript patterns such as ajax, promises and templating. If you are an end user, you might be more interested in user friendly samples like the SPELL Tabs or the SPELL Charts (and of course the SharePoint User Toolkit).

ProjectColorFor my simple example,  I have just created a custom list called “Projects”, with 2 columns:

  •  Title is a text field for the item name (Project 1, Project 2,…)
  •  Status is a choice field for the item status: green, yellow or red.

Expected result (cf. screenshot): color coded indicators that displays the health of each project.

The code (pS is the shorthand for pocketSOAP):

var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestSite",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"Projects"
});

pS.when(promisedItems).then(function(request){
 // Apply template
 document.getElementById("results").innerHTML=pS.applyHTMLTemplate({
 template:"<table>[<|<tr><td><div style='width:12px;height:12px;background-color:|Status|;'>&nbsp;</div></td><td>|Title|</td></tr>|>]</table>",
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z")
 });
});

How the code works:

  • pS.soap() handles the SOAP request (ajax). It needs two sets of options: information about the service itself (in our case Lists/GetListItems) and parameters specific to the call (in the case of GetListItems: the list name, and maybe specific filtering options and a shortlist of the fields to retrieve).
  • pS.when().then() is a promises pattern. It ensures that the SOAP request is completed before we move to the next step.
  • pS.applyHTMLTemplate() runs the templating engine, in this specific example to output a html string. In the template expression, [< >] means that we are dealing with a collection of nodes. If we were using the new REST interface and retrieving JSON, we would simply use  [{ }] instead of [< >].
  • the prefix ows_ is just here to address an oddity with SharePoint Web services, where “ows_” is prepended to each field name. So for example to get the Title field, you need to retrieve the attribute called ows_Title.
  • the data is the result of ajax xmlHttpRequest. If you are familiar with the SOAP service GetListItems, you know that each item is returned as a namespaced “z:row” node.

How many SOAP services does pocketSOAP support?

I don’t know. So far I have only tested it for half a dozen services representative of my scenarios (90% of real life use cases are about getting or updating list items). I plan to document this as I and the SPELL followers continue testing more services.

Can I use SPELL/pocketSOAP to aggregate values from multiple lists?

Yes. Thanks to the promises pattern, the script can collect data from multiple lists in parallel, then trigger the rendering when all the items have been collected. The pocketSOAP documentation will include such an example.

Note however that for a large number of lists, client side code might not be the best approach and you’ll want to consider other techniques.

How do I make a synchronous call?

You can’t, this option has been removed from SPELL last year. The choice is to embrace a modern JavaScript approach where remote calls are asynchronous (the first “a” in ajax), and dependencies are managed via promises: when(this is completed).then(do that).

In SPELL, not only ajax calls, but also script loading, iframe loading, document ready, and some other operations are treated as promises.

Did I go too far with pocketSOAP?

The answer is… yes! If you play with pocketSOAP, keep in mind that it is a laboratory rat. The SPELL library – the production version – offers much better balance between the load effort  and efficiency at runtime.

How is SPELL different from pocketSOAP?

1/ SPELL is for production, while pocketSOAP is an experiment. For example error handling has been reduced to a minimum in pocketSOAP.

2/ SPELL supports half a dozen interfaces, SOAP just being one of them. Apart from SOAP ($P.soap), it also relies on the RPC method ($P.rpc), the REST services ($P.listdata and $P._api), etc.

3/ SPELL offers not only core features aimed at developers, but also end user solutions like the SPELL Tabs and mini-BI.

How come SOAP has never been mentioned before on Path to SharePoint?

Until recently, I have always considered that SOAP was a heavy solution and should only be used in large scale implementations, or when everything else fails. For  the samples showcased in this blog and in the SharePoint User Toolkit, my preference went to the RPC method, a more straightforward and end user friendly approach.

However when I started building SPELL two years ago, SOAP was an essential building block in a projet of that size. The decision to include SOAP was even easier after I reduced the code size to just a couple kb!

References

The official Microsoft reference for SharePoint 2013 Web services can be found here.

The first to publish a JavaScript library to wrap SOAP services was Darren Johnstone in 2008. His original site doesn’t seem to exist anymore, to get the files try an internet search for “SPAPI”.

SharePoint superstar Jan Tielens published in 2009 a series of posts to showcase the use of SOAP services in conjunction with jQuery.

That same year, Marc Anderson released SPServices. This remains to date the most useful and comprehensive resource on the subject. When people look for documentation on the SOAP services, I usually point them to Marc’s Codeplex site rather than the official Microsoft documentation.

SOAP services are also a favorite of Alexander Bautz, and he uses them in some of his published work.

Alexander’s blog drove me to another library called SharePointPlus, developed by DELL employee Aymeric Kodono and released under a GPL v2 license.

There are certainly others I am not aware of. For example I haven’t tested this framework that seems to deal with the SOAP services (and btw is not maintained anymore). If you have more information feel free to post it in the comments!

Coming soon on Path to SharePoint

No post in more than a month, this had never happened since I started this blog in 2008! Yes, starting my new company, traveling across the US, and the tax return have taken their toll…

Mind you, the blog still remained very active, with more than a hundred comments posted in the past month. But now it’s about time I add new content. So what’s coming next?

My next tutorial will be about countdowns/countups. I already published a live demo in February. If you can’t wait, head out for the SharePoint User Toolkit, a first countdown-countup page is already posted there! (and remember to send me some feedback)

I am also working on the Easy Tabs v 5.1. One of the most common request is to have more branding options (colors, hover effects, etc.). I already made a leap forward between v4 and v5.0, and I hope to bring some more improvements soon. I already published an example of custom style (including a live demo), but the release of Internet Explorer 9 changes the game (for example IE 9 natively supports rounded corners, no need for workarounds).

In March, I spoke at the San Diego SharePoint User Group. After I clean up my slides, I’ll post them, along with some interesting Q&As from the session.

I also have a couple projects with my new company, User Managed Solutions LLC. In particular I am working on a new online training offer for this Fall – more details coming in May!

Last, but not least, I am honored to welcome guest blogger Trudy Hutzler for a series on creating a “school site”. Trudy is a Senior Technical Architect for AT&T Hosting and Application Management, and a contributing author for the Mastering SharePoint Foundation 2010 book. As this blog is user focused, she is not going to talk about her experience on SharePoint 2010 migrations, but about a custom site she built for her daughter who attends school online. Stay tuned!

Planning my trip to California

I am currently planning my trip to California, for the second half of April. I’ll be in the San Diego area the week of April 19th, and possibly in the San Francisco area the week after.

As usual, I am trying to make the most of such trips, meet with user groups, and discuss business opportunities with partners and customers. If you’d like me to come and visit you, please drop me a note at Christophe@PathToSharePoint.com and I’ll be happy to respond!

A workshop to get started with jQuery

Over the past year, the use of jQuery to enhance the SharePoint interface has exploded. Maybe you were tempted to take the plunge, but as an end user you were intimidated by this technology you know little (or nothing) about?

Well, this coming Monday is your chance to get on the bandwagon. Mark Miller and I will be hosting a new workshop: Get Started with jQuery in SharePoint.

Your goal is of course not to become a jQuery expert. But still, by the end of the workshop:
– You’ll have written – and tested – your first jQuery script
– You’ll know the key domains in which jQuery shines
– You’ll be aware that jQuery is not always the best option, and that you should not grab every script you find on the Internet

As usual we’ll provide concrete examples that you’ll test live in your own sandbox.

For more information and to register:
http://series-jquery.eventbrite.com

A great workshop!

Last Thursday, Mark Miller and I presented our last workshop of the year, dedicated to the SharePoint Gantt view.

As for previous workshops, I had prepared some customizations (7 to be precise), each packaged as a Web Part. As I was detailing them, I received some great feedback from Mark and the participants, on the usability of each solution. One advantage of these client side scripts is that the code is accessible and can be easily tweaked, so we were able to make changes in real time, that the participants tested immediately in their own sandbox.

After the workshop, I finalized the scripts and added a couple notes. I also added two more Web Parts, also based on the comments I had received. And one participant, Jeremy, shared another variation he had worked on after the workshop. What a teamwork!

Mark and I share the same view of what training should be. When we say live workshops, we mean it:
– live: the participants have their word to say, and their reactions drive the pace of the session.
– workshop: what is important is not what we show, but what the participant is able to accomplish.

Mark and I will team up again in January for a new series, and I am really looking forward to it!

If possible, I’d like to take the experience even further, and tune up the workshops content based on your feedback. We have already heard from several people that it would be good to have an introduction to jQuery for non-programmers, and we are working on this.