Pie and Bar Charts (Google connector)

Pie

I already mentioned it briefly in an earlier post: the SharePoint User Toolkit now includes a tool that allows you to add simple pie or bar charts to your SharePoint pages.

The code is actually a connector that points to the powerful Google Visualization interface. For this reason, Internet access is required.

A common concern with online tools is data security. In this case, we are safe, Google’s privacy policy explicitly states that no data is sent to the server. The charts are directly rendered in your browser. This is a major difference with the Google Image Chart API, where data is sent in clear to the server, which uses it to build and return an image.

Under the hood, the code works the same way as the Image Rotator: it retrieves data from a specific view of your SharePoint list.

Coming soon on Path to SharePoint

No post in more than a month, this had never happened since I started this blog in 2008! Yes, starting my new company, traveling across the US, and the tax return have taken their toll…

Mind you, the blog still remained very active, with more than a hundred comments posted in the past month. But now it’s about time I add new content. So what’s coming next?

My next tutorial will be about countdowns/countups. I already published a live demo in February. If you can’t wait, head out for the SharePoint User Toolkit, a first countdown-countup page is already posted there! (and remember to send me some feedback)

I am also working on the Easy Tabs v 5.1. One of the most common request is to have more branding options (colors, hover effects, etc.). I already made a leap forward between v4 and v5.0, and I hope to bring some more improvements soon. I already published an example of custom style (including a live demo), but the release of Internet Explorer 9 changes the game (for example IE 9 natively supports rounded corners, no need for workarounds).

In March, I spoke at the San Diego SharePoint User Group. After I clean up my slides, I’ll post them, along with some interesting Q&As from the session.

I also have a couple projects with my new company, User Managed Solutions LLC. In particular I am working on a new online training offer for this Fall – more details coming in May!

Last, but not least, I am honored to welcome guest blogger Trudy Hutzler for a series on creating a “school site”. Trudy is a Senior Technical Architect for AT&T Hosting and Application Management, and a contributing author for the Mastering SharePoint Foundation 2010 book. As this blog is user focused, she is not going to talk about her experience on SharePoint 2010 migrations, but about a custom site she built for her daughter who attends school online. Stay tuned!

Case study: course compliance report (Part III)

Author: Larry Pfaff, Sr Systems Analyst, Convergys Corp.

Adding The Final Pieces

Can you see the finish line yet? As you remember in Part 1 we reviewed the requirements and expectations of the data. This output needed to be easy to read and use by all end users throughout the entire organization and most of the views returned the desired results.

One advantage of using these views was no additional scripts were required to display data in table format with one column. Once we had a direction, in Part 2, we discussed some of the obstacles and limitations we ran into and how we overcame them. Although we are very close to completion there are still some details to review and modifications to make. For this all to work we will be adding the final parts (scripts) into a Content Editor Web Parts (CEWP).

Create a view – one column
Begin creating a Standard view with the listed options. This should be fairly simple. If you need details for this you may go here.

  • Added one column, which contained all the data for the table layout.
    • Table – (table of data)

  • Select your Sort and Filter options. I set sort to Ascending by Process Total. This allowed the teams who were less compliant to show closer to the top of the view
  • No Group By
  • No Totals none available
  • Style – tested in all views
  • Item limit – your choice, this one was set at 150

CSS

Add a CEWP above your list view. This will contain the classes and formatting of the table. Be specific in the class names otherwise the formatting would be transferred to other parts of the page.

There are several references available on the net to provide you with more details: HeatherSolomon.com is one of the most commonly used.

Once you have added your CEWP to your list view you can add the CSS below. Initially my styles were off, which cause some formatting conflicts. Thanks Christophe for the style update for the table borders mentioned in Part 1.

<style type="text/css">
table.myboxed, table.myboxed TD, table.myboxed TH {
				border: 1px solid black!important;
				border-collapse: collapse;
				font-size: 10pt;
}
.twidth {
				width: 100px;
}
.comp {
				color: green;
				font-weight: bold;
}
.nc {
				color: maroon;
				font-weight: bold;
}
.e {
				color: orange;
				font-weight: bold;
}
.label {
				font-weight: bold;
}
</style>

HTML Calculated Column

The key script to display the images, color and table layout can be downloaded here with instructions for installation.

Final product looks like this:

Final thoughts

As I sat down to complete this final part in this 3 part series “Case study: course compliance report” I found IE and FF definitely don’t play well together. I also found there are several things I would do differently. With all the pieces placed together the results are astounding, but this is only one of many ways that scripts, intuitiveness and desire can produce outstanding results.

One of the changes I thought about making was to the progress bar. Currently I am in the process of showing both Complete and Exempt on the same bar as one continuous bar like this.

54% 8%

In theory this works. I can replicate it in a regular html table with div tags, but when implemented into a CC I lose either the color or the percent for the % Exempt. Will keep you posted on the progress, but for now I will continue to hack away until I figure it out.

Since the beginning of this article I was inspired to start my own blog at http://sharepointhacker.wordpress.com. Although this blog may never have any new intermediate scripts it will have scripts implemented in real world scenarios, which I feel will provide beneficial learning. For a long time I always considered myself just a hacker, never wrote any original scripts, so I thought. I would always find someone else’s code and tweak it to fit my needs until one day a blogger said to me:

“More than the script we come up with, what is important is the innovative way in which we approach a real world issue”.

Christophe

Thanks again Christophe for your support.

Case study: course compliance report (Part II)

Table

Guest author: Larry Pfaff, Convergys.

I left off explaining how I went from raw data to manipulating views to achieve a table layout with usable results.
Continuing forward I will describe how to achieve this output through the use of Calculated Columns (CC) in a custom list. It’s fairly easy to build a table in a CC with some simple HTML. I built my first test table with some test data and found it worked well. I began interchanging my test data with my list data until I hit a wall. The simple HTML table worked perfectly with test data, but when all my formulas replaced my test data it failed. Initially, I thought my formulas contained errors, but upon further research I found it to be the result of a character length limitation in the CC.

Size Limitation

There isn’t much discussion on the web about the character length limitation of a CC. I actually stumbled upon it without realizing. To troubleshoot this limitation I began to break down my formula into useable chunks. By adding the useable chunks to the CC I could confirm parts of the formula were correct. Once I confirmed all the parts were correct I added them back together, but they failed again! I was receiving the error “The formula contains a syntax error or is not supported.”
I confirmed the formulas were correct piece by piece, but once I put them all together it was no good.

This was very frustrating so I reached out to our SP Admin to inquire why this is happening. He stated something about the formula was too long and I couldn’t use it. Well, I am not one that takes no for an answer. So I began trying to figure out ways to reduce the size of my formula string. This is when I had two ideas for reduction.

  1. Remove the inline styles and replace with CSS classes
  2. Go back to breaking the formula into usable chunks, leave them as separate CC’s and use a final CC to pull all the parts together.

This result was very successful. In fact later research I found a single post “Calculated Column Limits (1024 Characters)”.

Now that we have a work around for the size limitation we can discuss the fields and calculations.

My list data was updating twice daily through the use of a macro in an Access database paired with the use of scheduled tasks. Below is the list of fields used in my reporting requirements. The gray highlighted rows in the table below is my raw data automatically updated. Each CC will be described and what calculations were used, when necessary.

Field Name Field Type Function
Sr. Executive Single line of text Raw data
Name – Executive Single line of text Raw data
Completed Number Raw data
Not Completed Number Raw data
Exempt Number Raw data
Grand Total Calculated Total of the 3 above
Report Calculated not required
% Not Comp Calculated  
% Complete Calculated  
% Exempt Calculated  
Progress Totla Calculated Display progress bar
Table Calculated Display table with data
Chart Calculated Chart Image

Calculated Columns

Grand Total
Data type returned as Number with Auto decimals places. Simple field returning the summed value of three columns for a specific row.

 =SUM([Completed],[Not Completed],[Exempt]) 

Report – Progress by Executive Data type returned as Single line of text

This is not a required field, but it was useful in providing a navigation method to the source data (numbers/details). Here I used an IF statement to build a hyperlink back to the original excel file stored in a local document library. The excel files were names according to the Sr Executive, making the hyperlink easier to build dynamically.

“Ayers” the first name in the formula had the most line items. The report was so large my Access macro would fail trying to upload data into SharePoint. So I split the report into 3 separate files, this formula takes that into account and links to one of three excel files for Ayers.

 ="<DIV><a href="http://cvgsharepoint/sites/learning/DataSecurityPrivacy/Data%20Privacy%20and%20Security%20Status%20-%2520"&IF([EXEC%20Select]="Ayers,Andrea%20J",IF(OR(Name="Frank%20J%20Corso",Name="Victoria%20Perez%20Mueco"),[EXEC%20Select]&"-New",IF(OR(Name="Steve%20M%20Heffron",Name="Sukant%20Srivastava"),[EXEC%20Select]&"-New",[EXEC%20Select]&"-New")),[EXEC%20Select])&".xls' title='Report Details for "&[EXEC Select]&"' target='_blank'>Report Status Details -
  "&[EXEC Select]&".xls</a></DIV>"


figure 2.1

The next three formulas are just calculating a percentage for that value

% Complete
Data type returned as Number with 2 decimals places.

 =ROUND(([Completed]/[Grand Total]),2) 

% Not Comp Data type returned as Number with 2 decimals places.

 =ROUND(([Not Completed]/[Grand Total]),2) 

% Exempt Data type returned as Number with 2 decimals places.

 =ROUND(([Exempt]/[Grand Total]),2) 

Progress Total
Another Idea came from a progress bars and adding color.
Currently this CC totals [% Complete] and [% Exempt] as total progress. Because some new employees and upper management were exempt from taking the course Exempt could not count against the total. .

This formula worked perfectly in IE, but I found it to fail in Firefox (FF). I tried multiple changes, but still no success. Christophe brought to my attention I was missing the non breaking space discussed in this post Progress bar + color coding. Now it made sense why it wasn’t working correctly in FF. I made the simple change and confirmed no visual difference between both IE and FF. This is the resulting formula:

 ="<DIV style='position:relative;'> <div style='background-color:#ccccff;'>
<div style='background-color:#0000ff; width:"&(([% Complete]+[% Exempt])*100)&"%;'>&nbsp;</div> <div style='position:absolute;top:0px;color:white;font-weight:bold;padding:0 0 0 2px;'>"&TEXT(([% Complete]+[% Exempt]),"0%")&"</div> </div></DIV>"


figure 2.2

Table This formula is a simple table with some added styles and nested CCs. Styles and nested CCs were used to help with the character length limitation by minimizing total string length. A few trial and errors later and a successful table output. This formula was tested in both IE and FF with success. This is the resulting formula:

 ="<div><b>Sr. Exec.: </b> "&[Report]&"<table class='myboxed'><tr><th colspan='2'><b>Exec.:</b> "&[Name]&"</th><th class='twidth'>Completed</th><th class='twidth'>Not Completed</th><th class='twidth'>Exempt</th><th class='twidth'>Total</th></tr> <tr style='text-align:right'><td rowspan='2'>"&[Chart]&"</td><td  class='label'>Number</td><td class='comp'>"&[Completed]&"</td><td class='nc'>"&[Not Completed]&"</td><td class='e'>"&[Exempt]&"</td><td>"&[Grand Total]&"</td></tr> <tr style='text-align:right'><td class='label'>Percentage</td><td class='comp'>"&[% Complete]*100&"%</td><td class='nc'>"&[% Not Comp]*100&"%</td><td class='e'>"&[% Exempt]*100&"%</td><td></td></tr> <tr><td colspan='6' style='text-align:left;'>"&[Progress Total]&"</td></tr></table></div>"

Table
figure 2.3

Note: the styles used for the above classes will be provided later.

Chart
Data type returned as Single line of text. There are several types of chart providers. At the time I was developing this I only knew of google charts. This CC was created for 2 purposes. 1. to return the data as a chart and 2. to save on character length and to be shared in the Table field.

 ="<DIV><img src='http://chart.apis.google.com/chart?cht=p3&chs=150x50&chd=t:"&[% Complete]&","&[% Exempt]&","&[% Not Comp]&"&chl=Comp|Ex|NC&chco=0000FF'/></DIV>"


figure 2.4

Conclusion
Although we are not quite finished we are well on our way. In Part I we discussed the process that lead me to the table layout and styles described. In Part 2 we discussed some obstacles and work rounds to get the required results. In our final installment, Part 3, I will describe the CSS and scripts that provided the final presentation.

Update [06/07/2010]: screenshots updated.

Case study: course compliance report (Part I)

A couple weeks ago, Larry Pfaff shared a screenshot that impressed me. First, it made full use of my “HTML Calculated Column” method, including in a same report color coding, a pie chart and a progress bar. But also it offered an elegant, condensed layout, different from the traditional “one row per item” tables.
I asked Larry if he could tell us more about his design. After introducing the business context and providing an overview in part I,  in part II and III we’ll dive into the formulas behind this customization.

 This past year I was given a set of requirements to develop a reporting mechanism that needed to track course compliance for all company employees. Although our Learning Management System (LMS) has some reporting functionality it requires a higher permission level to run some of the basic vanilla reports and the vanilla report data is for specific groups or teams, not the entire organization. With over 70k employees I needed to find a simple useable way to report out on the course compliance (Complete, Exempt and Not Complete).

Some of the key features I kept in mind were usability, simplicity and functionality. From inception I knew I wanted a condensed view that would display the data across multiple rows that included color coding and inline charts. From a usability standpoint a table output would be easier to interpret then the standard SharePoint views and would be very helpful in meeting the simplicity and usability requirements.

For the last six plus years I have been actively using SharePoint (SP). I utilize SP whenever I can with the understanding that SP is not the solution for every request. Considering the end user population and usage of this data, SP was the solution for this project. I know SP offers multiple views and the views could include numerous sorts and filters that can be applied, but even the Table view SP offered could not organize the data on multiple rows. Once I was able to get my LMS data into SP I could manipulate the output into a table format grouping the data across multiple rows.

First I needed to automate data migration into SP. Then I needed to organize my output to obtain a table result. For the past six years I struggled on my own learning SP trying to stay ahead of the curve. One of the toughest obstacles I’ve come across is the lack of shared knowledge from within my company. When I first came across the Path to SharePoint blog this opened doors to a whole new world, starting with how to convert text to HTML by adding script to a Content Editor Web Parts (CEWP). This provided a way to change the standard SP pages without damaging or unghosting them.

Once my data automation was implemented I began trying different outputs (views). In one of my initial views (Default) I was able to display a simple colored progress bar and inline charts through the use of the HTMLcc script. This put me on the way to achieving the visual effects I was looking for. 
 

Figure 1.1

 This view was a simplistic, but not the table output I needed. It returned the data with colors and charts, but the results are still on one row and not easy to read. I began investigating other view types.

Implementing the Boxed view condensed the data some, but it was confusing because the data had no delimitation.
 

Figure 1.2

The “Boxed view, no labels” returned a cleaner results, but still missing organization.
 

Figure 1.3

 So I began thinking how this data could be displayed in a way that would provide simple results across multiple rows of a table. I began with a simple table design for the data. With the HTMLcc script already in use I knew I could convert any HTML I put into a calculated field. My first attempt to create a table with some test data was successful.

As I began testing my table across the different view types (Basic Table view) my results were less satisfying even with the addition of CSS and “HTMLcc” scripts it resulted in missing formatting. 
 

Figure 1.4

In figure 1.4, every other table was missing borders. Looking through the page source I realized the TR class “.ms-alternating” was causing a conflict. Initially I did not pursue a fix. When I presented this to Christophe he agreed the issue was the TR class and modified the table styles for a successful solution.

 Even without all the borders the results were much more manageable and easier to read. I continued testing the table layout across multiple views. Initially, during development, I was only using Internet Explorer 7, but once loaded into Firefox the results were identical.
 

Figure 1.5

Wrap up
Today we were able to achieve extraordinary results that displayed data across multiple rows in a table format. The tables contained color and inline charts something the OOB SharePoint could not achieve. The output was tested across multiple view types and across multiple browsers with identical results.  In Part 2, I will discuss the steps to creating a table output incorporating color and Google charts through the use of scripts and calculated columns, with an identical look and feel across multiple browsers.

Credits
I am a long time reader of Christophe’s blog and have always been amazed at the work he has put out. This article could not be possible if not for the work and scripts of Christophe Humbert at Path to SharePoint.

About Larry
Larry Pfaff, Sr, Systems Analyst working for Convergys Corp, a global company, based out of Jacksonville Florida delivering training and outsourcing for major corporations. I have been working with SharePoint for almost 7 years and manage many internal projects using SharePoint as the collaboration tool. I enjoy new challenges and automating existing process. I seek out cookie cutter solution and mold them to fit for “Non-Developers” working with SharePoint by developing and providing training for end user and sharing my knowledge with as many as I can.”

Update [06/07/2010]: Screenshots updated.

This week’s workshops: visualization, Gantt and Sparklines

I am presenting three live online workshops this week, with Mark Miller from EndUserSharePoint.com. As usual, each participant receives a sandbox to try out the solutions we provide.

March 23, 2010 – Inline Visualizations in SharePoint
An entry level workshop where you’ll learn how to add color and other effects to your SharePoint lists. Here is a live demo:
http://www.pathtosharepoint.com/Lists/TasksVisualization/AllItems.aspx

March 24, 2010 – SharePoint Gantt Chart Enhancements
The participants will receive 6 snippets of code. The centerpiece is the dynamic timescale (month/week/day), as demonstrated on this mockup:
http://www.pathtosharepoint.com/Pages/GanttTimeScale.aspx

March 25, 2010 – Dynamic Feedback through SharePoint and Sparkline Charts
Sparklines are a fascinating, relatively new way to communicate. More compact than traditional charts, richer than traditional indicators, they are very popular in dashboards that require to synthesize large amounts of data.
jQuery and Google will come to the rescue to help us integrate these mini-charts in SharePoint. But of course we’ll also share some homemade recipes!

For more information and to register, follow this link:
http://eusp-chartsandgraphs.eventbrite.com

My slides from SharePoint Saturday EMEA

My slide deck from the SharePoint Saturday EMEA conference  is now online. You can find it on Slideshare, or get it from the download section of my Website (under KPI roll-up). The recording of the session should be available soon, watch out for updates on EndUserSharePoint.com.

SharePoint Saturday EMEA was a fantastic event, kudos to Mark Miller and the organization team! As Mark explained in his post mortem, “there were a couple of roadblocks to making this thing actually work. First was that Europeans treasure their weekends and we had to assume that many people would not attend because it was scheduled to be a Saturday event.” Well, the Europeans proved that they had the motivation to dedicate their Saturday to SharePoint. And the Americans proved that they are always ready for action, even at 3 am!

Smart TextToHTML

First things first: if you don’t know what I mean by “TextToHTML “, you won’t get much from this post. In this case, I recommend that you start with this introduction.
In short, the TextToHTML script has two roles:
1/ find HTML strings in a SharePoint page
2/ Convert these strings into actual HTML

Yesterday, I came across a discussion between @EUSP, @webdes03 and @ebrackley on Twitter:
“TextToHTML can be a bad performer when tasked with lots of HTML; also research DVWPs”

This is a timely comment, as I am about to present at SharePoint Saturday EMEA. Let me expand on this, and provide a few hints on how to better use the HTML Calculated Column.

Remember: TextToHTML is not the only way

Right, most of the examples you’ll find on my blog rely on the TextToHTML script, embedded in a Content Editor Web Part. But there are other ways to render the HTML. In particular, the Data View Web Part, used in crosslist mode, or the Content Query Web Part can directly do the rendering. For more information, see these series:
– for MOSS
KPI roll-up in MOSS
– for SharePoint 2007 (applies to both wss and MOSS):
KPI roll-up in SharePoint (Part I)
KPI roll-up in SharePoint (Part II)

The latter will be the theme of my presentation at SharePoint Saturday.

Get a faster browser

Not always a choice, but if you can, upgrade your browser to the latest version. These days, browsers are improving by leaps and bounds, with Google leading the charge.
My tests show that the TextToHTML script is four times faster in IE 8 than in IE 7. That’s huge! Some other browsers offer even better response time.

Faster TextToHTML

If you are using the TextToHTML script, you’ve certainly got the current version from the download section. What I am making available over there is a generic script, for both SharePoint 2003 and 2007. If you are on SharePoint 2007, you can make it more specific and thus faster.

Focus on the main content

As is, the script will scan the whole page to find table cells (TD elements):

TextToHTML(document.getElementsByTagName("TD"),regexpTD);

This is a waste of time, as you just need the main content, excluding header and Quicklaunch. You can easily do this by replacing the above line with the following code:

var theMainContent = document.getElementById("MSO_MainContent");
TextToHTML(theMainContent.getElementsByTagName("TD"),regexpTD);


Use selectors

Instead of grabbing all the cells in the document, try to restrict your scope to the cells that may contain HTML Calculated Columns.

For example, Paul Grenier proposed a jQuery version of my script, which allows you to grab only certain cells. If HTML Calculated Columns are only in List View Web Parts on your page, the simple $(“td.ms-vb2”) selector should be enough. For better performance, you may want to combine it with my first advice, and focus on the main content: $(“#MSO_MainContent td.ms-vb2”).

Here again, a modern browser will give you better performance (for example if it has native tools to find elements by class name).

Place your script close to the point of consumption

Imagine that you are displaying 10 lists on your page, but only the third one is using HTML calculated columns. If you place the CEWP that contains TextToHTML at the bottom of the page, the script will go through the 10 lists. If you place the script right below the third list: when the script runs, it will be faster as it only sees 3 lists, the 7 others being displayed later.
btw placing the scripts close to the point of consumption is a good practice that also applies to other scripts, not just here. In traditional Web design, you would find all the scripts under the head section of the page, but this practice has evolved now that pages are more dynamic.

If you have other ideas, feel free to share them with me and the other readers! I’ll push some of these performance improvements in the next release of the TextToHTML script. I am also looking for volunteers to test the beta versions…