Using Infrastructure to Hide All Sins

There’s a secret that the infrastructure guys don’t want to share with the developers. The secret is that for almost any sin you can commit as a developer there’s a way to resolve it as an infrastructure guy — given money. They don’t want you to know that though — because they actually like getting sleep and don’t like alert emails from the system at two in the morning. I’m hoping that I don’t lose my infrastructure person card (IPC) for telling everyone this but I had to in order to make a point.

Recently I had a customer that had a three server SharePoint farm have two servers stop functioning correctly. (I think it’s really a switch issue but I’m not close enough to know for sure.) As a result of this they took two of the three servers out of the load balancing cluster and … the remaining server was able to keep up quite happily with the load being applied. The infrastructure manager I was working with was surprised. My response was to explain that the architecture in addition to being poised for growth was also designed around the idea that the developer code would be very bad — and in this case it wasn’t.

You see, I architected the system to account for a fairly high probability that the developer code would randomly and inexplicitly cause a server to crash, run out of memory, blue screen, or just generally go dark from time-to-time. With that in mind, we put two servers in that should be able to cope with the load from everyone. The third server in the farm was just there to be the token server that was in the process of crashing and coming back.

Load balancing can hide almost any server stability sin that you can come up with. Simple Network Load Balancing (NLB) included in Microsoft Windows Server operating systems can hide problems. Tools like F5‘s BigIP can hide them better.

In fact, infrastructure is built on the idea that some packets are going to get dropped, lost, or delayed, and thus the TCP (Transmission Control Protocol) of TCP/IP manages the resending, reorganizing, and reassembly of packets across the network into a stream. It’s a core part of what infrastructure does — it recovers from failures — it hides all sins.

I’m not suggesting that this is always a good thing — adding more memory to a server — or switching it to 64 bit just hides memory leaks. Eventually the cost of hiding the problem will be worse than owning up to it, finding it, and resolving it.

While load balancing is the first in my bag of tricks, it’s by far not the only trick. Things like Link Aggregation Control Protocol (LACP) can be used to address a choke point to a server — the network interface. On the server side it’s called network adapter teaming while it’s LACP on the switch side. In either case the effect is aggregating two network cards into a faster virtual connection. Suddenly you double the amount of traffic you can get to and from your server.

Simple solutions to problems with bad query designs generally involve more RAM in a SQL server and smaller, faster, better organized disks. SQL server is REALLY good at caching data. If you put RAM in a box it will reduce the load on the IO subsystem. If you put smaller, faster disks in you’ll be able to get more IO operations to and from the disk — no matter how bad the query, if you throw enough hardware at it you can make it work.

For instance, I’ve made SQL server be faster at serving queries that originally were serviced by a Pervasive SQL system that was used to making ISAM file calls. I’m not talking faster by a few percent — I’m talking more than 30% faster. Sure I bought faster disks, formatted them RAID10, threw a ton of memory in the box, and ran Enterprise Edition — but we did get the goal accomplished.

That’s just in the standard bag of tricks. There are third parties in the act like the tools from Strangeloop networks which hide problems with view state in ASP.NET applications. WAN acceleration devices from Riverbed, Cisco, and Certeon can all reduce the amount of bandwidth your application uses over the WAN.

Why am I telling you this? Well, if you’re an infrastructure person just pretend it’s so you can buy cool new toys. If you’re a developer, I want you to know that if you make a mistake or overlook something that your infrastructure colleagues have your back. Of course, some of the solutions don’t come cheap so you may have to forgo your next bonus or your next raise.

So I’m pleading with you, do your development right so that your infrastructure brethren don’t have to hide your sins. Appropriately optimize your code. Make sure everyone on the development team understands the fundamentals of good design. Do peer software reviews. Do whatever it takes to make great code.

Book Review-Building Content Type Solutions in SharePoint 2007

When I picked up Building Content Type Solutions in SharePoint 2007 I was hoping that it would be the powerhouse book that helped the SharePoint community realize the power of content types. You see, I’ve decided that they’re one of the most powerful — if not the most powerful — feature in SharePoint. I’m fascinated by the idea that you would create a solution that includes the document template as well as the process (i.e. workflow) around that document when it has been created.

Unfortunately, I can’t say that the book lived up to my expectations. (One could easily say that they weren’t fair.) The book has a ton of great content in it. If you’re an end user, power user, or site administrator — you’ll learn something from the book. My issue is that the book never really develops an audience. It feels like a semi-random collection of stuff about content types. The stuff in the back of the book about development just doesn’t feel like it was intentional. (Not that it’s not valuable.)

So while the book is useful if you’re trying to see some of the power of content types, I’m concerned that it’s not the content type book that the market needs to really realize the power of this particular feature of SharePoint.

Book Review-SharePoint for Project Management

I’ve fallen more than a bit behind on my reading. I’ve got a stack of books here that I keep meaning to read, skim, review, etc. I did get a chance to take a look at one in the last few weeks and I’m glad I did. The book, SharePoint for Project Management, is a book I can respect. Why? Well, it’s a very practical how-to guide for using SharePoint for Project Management (one of its strengths). In some ways I think of it as The SharePoint Shepherd’s Guide for End Users with a slant towards project management.

Of course, there’s more introductory text/conceptual stuff that’s needed in the project management space and that’s well covered, but the book also has the step-by-step processes that the uninitiated find helpful for finding the right switches, dials, and do-dads to make the product do what you want.

If you’ve got a project management office team or a group of project managers, this book can help.

The SharePoint Shepherd’s Guide goes Live!

With slightly more than a year behind The SharePoint Shepherd’s Guide for End Users, I’m happy to finally announce that there’s a live version available. I’ve partnered with Captaré to deliver an on-line, live, instructor lead version of the Guide. I’m excited about this option for organizations who can’t bring the content in house and teach it themselves. Even for those who can deliver the material in side, this offers a unique way to do a train-the-trainer session to see how we think that you should get users up to speed with SharePoint.

Although I won’t be personally teaching the classes, I’m planning on “dropping by” to hear about the experience and see how that we can provide the best possible training experience.

The Best of the Microsoft SharePoint Conference 2008 – Chicago

On Wednesday I had the pleasure of presenting two sessions at the “Best of” Microsoft SharePoint Conference 2008 in Chicago. The invitation only event was for larger Microsoft clients. We had about 300 people in attendance. I delivered two sessions:

The above links go to the presentation decks that I used for the presentations.

There are a few interesting things about the conference. First, in my governance session I got asked about how to create a taxonomy. My response was first the creator/consumer conversation (I’ll blog in detail on that in a bit.) The additionally response was that there were two other folks that were talking about Taxonomy. The first is Mark Schneider and the second is Bob Mixon. I’ve talked about taxonomy creation as well, but not much publically. I did mention the book Organising Knowledge — which is the best taxonomy book that I’ve ever run across.

It was also interesting because Joel Oleson, Asif Rehmani, and I sat down to chat. You could have easily thought we would have talked about SharePoint Governance, SharePoint Designer, or SharePoint in general. Although the topic came up, we mostly talked about religion. Joel is a Mormon, I’m a protestant Christian, and Asif is an Ismaili Muslim. The really cool thing about this is that all of us have such great respect for one another that we could have a conversation where we were discovering what we each believed — without any of us getting the slightest bit upset.

Social Media for an Organization that has Email as Last Name, First Name

I recently got an email message from someone mentioning that their organization is heavily focused on enabling social media for their organization. The irony was that the name in the email came up as Last Name, First Name (Smith, Robert). Social media is about making connections with other people. With a handful of exceptions I don’t call people by their last names. (I have only one or two people that call me by my last name.) I find it interesting that organizations do so much to make it difficult for people to build relationships and yet are considering non-trivial investments in social computing. My recommendation if you want people to start to communicate more fluidly with one another. Make their email name friendly name [space] last name. By friendly name I mean Bob — not Robert or whatever name people know the person as.

Governance Plan or Process?

I’ve been having a few conversations lately about Governance and how organizations can get it. At the SharePoint Best Practices Conference in San Diego, Joel Oleson asked me whether I used the sample governance plans available from Microsoft’s web site. I said no, a bit too abruptly. I said that I use the “Paint Cards” that he put together leveraging some content I had created and content from others as well. Fundamentally, I explained, I’m more interested in the governance process — the process of making the decisions — than I am in the results.

I was speaking with Rob Wilson via email and some of his comments brought to mind an older article I wrote back in 2005 for TechRepublic titled “Creating artifacts — what you don’t know” The gist of the article is that artifacts (documents) are good for helping folks to focus but you shouldn’t spend too much time focused on their creation and that the process of creating them is sometimes more important than the document itself.

I really firmly believe that your governance documents should be as short as possible (but no shorter) — and that the decisions are the important part of the process, not necessarily a fancy document.

SharePoint Shepherd Updates

I’m getting a ton of email about where things are with The SharePoint Shepherd’s Video Academy for Planning and Governance. I’m also getting folks asking me about The SharePoint Shepherd’s Guide for End Users, so I wanted to take a quick minute to provide some updates.

On the Video Academy, I’m having some major issues with HP desktop computers which have delayed the completion until at least April 1st, if not longer. The short of the issues are that I bought two refurbished HP Pavilion m9350f machines to repack and prepare the video. In the last month I’ve essentially not had either of these two machines working. I’ve had just about every issue known to man with HP’s processes including the fact that it took them 3 days to correct a problem with my warranty (they said it wasn’t valid). I’ve also had a hard drive replaced with bad partitioning. (The first machine failure was a hard drive failure.) The second system has random lockups that haven’t been diagnosed — despite the fact they sent the unit back to me. Now, I’m waiting on the escalation support representative to fulfill the promises he made two days ago to ship me a new hard drive and a box to ship the second computer back again. I’ll provide a more detailed post later — but to all of you who I know are anxiously awaiting the video please accept my apologies. If you know anyone at HP — you might point them to this blog post.

On The SharePoint Shepherd’s Guide for End Users, there’s great news. I’ve posted pages on the web site where you can buy the book directly from me. Why is this a good thing? Well, first, I’ll sell you the book for $30 including shipping and handling to the United States. That’s a few dollars savings over folks like especially if you are paying for shipping. That’s not all, I’ll give you the opportunity to buy a personally signed copy of the book. The payments are handled by PayPal and the book will ship out within 2 business days of the order.

We’re also working on a deal that may allow you to take a class based on the book, but I can’t quite announce that yet. I continue to be pleased with the book’s progress both on the retail front as well as on the number of organizations and consulting companies that are realizing that the book’s corporate editions can improve end user satisfaction and reduce helpdesk calls. I’m still working on some hard statistics from a few client on how much their end user satisfaction went up and how much their helpdesk calls went down but we know it can make a significant impact.

Where’s Rob — Oops Missed One

In my last post I missed one of the users group talks I’m going to be doing. I’m going to be speaking at the Evansville SharePoint Users Group on May 14th. It will be another one of those governance talks. I’m not sure exactly which spin we’ll put on the topic, but I know it will be fun. Evansville, though in the same state (Indiana) is about as far away as it gets to stay in the state. It’s over a 4 hour drive. I’ve not been to Evansville for years — so I’m looking forward to going back.

Where’s Rob – Spring 2009 Edition

It’s been a while since I’ve sat down with a calendar to see where I’m going to be. So I wanted to let folks know where I’m going to be and we can play our own form of Where’s Waldo. In fact, the first two people at each event to walk up to me and call me Waldo will get a free copy of the SharePoint Shepherd’s Guide for End Users — mailed to their house so they don’t have to carry it. (Except for the users groups, I’ll hand them to you there.)

So where am I going to be? My next public event is SharePoint Connections in Orlando March 22nd-March 25th I love this event because it’s big enough to give the attendees the content they need — while keeping that laid back and casual feel. This is my 3rd or 4th year presenting at the conference (I forget.)

I get to do my "All About SharePoint Workflows" all day pre-conference again. Basically, it’s the core stuff you need to know about writing workflows in SharePoint — in a day. Generally you’re taking a several day class to get this much content. If you don’t believe me that there’s a ton of content here, you should ask someone from last year. There’s a workshop fee for this session.

In the main content I’m talking about "Getting Your SharePoint Project Started Right", "SharePoint Workflow Forms", and "Custom Authentication for SharePoint".

Getting Your SharePoint Project Started Right is a getting started spin on the governance and planning aspects of a SharePoint project. You’ll learn the things that tend to make SharePoint projects spin out of control and what you can do about them.

SharePoint Workflow Forms is a conversation and demos around building forms for Workflow. We’ll talk about Association, Instantiation, task forms, etc. — if we have time we’ll even talk about routing forms as the payload of workflows.

The Custom Authentication for SharePoint talk has me rebooting my virtual machine in the middle of it (because you pretty much have to do that to get FBA to work) — so it’s always great fun to see my say a little prayer on stage. I’ll be showing you how to write your own membership provider for ASP.NET + what you need to make it work with SharePoint.

Next up for me will be TechEd NA 2009 May 11th-15th in Los Angeles. I get to do a slightly different edition of workflow "SharePoint Workflow in a Day". I also get to do "Patterns with Microsoft SharePoint" — that’s a session based on the work I’ve been honored to do with The Microsoft Patterns and Practices team on the SharePoint Guidance. If you made it to the talk I did at the SharePoint Best Practices Conference you’ll know I get just a tad bit excited while delivering this content. It’s good stuff.

The next public sessions after that are at SPTechCon in Boston June 22nd-24th. I get to deliver a "Tour of Development Governance" and "Getting Just Enough Governance". Both talks are about how you can get governance in your organization without destroying adoption or engagement. One is more developer/development lead/architect focused (guess which one) and the other is more about the business of governance for business analysts, project managers, managers, and business leaders. (Remember, I define governance as "risk management" — so it’s all about managing risks to your SharePoint implementation and your business.)

On a more regional level, I’m also doing a SharePoint Governance event in central Illinois April 8th. This is a half day session on what you need to know to keep your SharePoint deployment out of trouble. It’s an invitation only event, but send me an email if you need an invitation.

I’m also going to be speaking at the Kentucky SharePoint Group on April 16th. I’ll be talking about Governance there as well, although I don’t remember nailing down exactly what specific aspect we’ll be discussing. At any rate, we’ll make it interesting.