*Update 1/26/2015 – The actual cause for the performance difference between the Organization Service and 2007 SOAP Endpoint can be found here. Since we were using Windows Authentication to access the 2007 endpoint we were able to bypass the overhead added to Organization Service calls by Claims Authentication.
Early in 2013 we had a client preparing to go live on CRM 2011 using our standard CRM adapters that leverage the CRM Organization Service to perform CRUD and Execute operations from our various custom applications that are standard in Cobalt’s suite of products. Almost immediately under load tests the applications came to a grinding halt and became pretty much unresponsive. Our previous tests and client implementations under moderate load hadn’t experienced the same behavior and it became obvious that we had a problem somewhere in the software.
The first inclination for our team was to look into the custom aspects of the client’s system for culprits for the performance issues. There were some areas of the code that were tweaked to perform better under load and there were also hardware changes made to increase throughput of the applications, but none of this seemed to be the silver bullet and at best only gave us modest performance gains.
With all hands on deck to fix the issue we began running performance tests against our adapters that connect CRM to Cobalt’s Products. Initially we ran a comparison test between querying for data through Cobalt’s Adapter and performing the same query directly using the CRM SDK through the Organization Service. The results were pretty much the same. There didn’t appear to be any overhead added to the calls by our Adapter, but the times we were seeing for the calls through the Organization Service were troubling.
Under serious load a call to CRM to retrieve 0 records was taking ~500 ms. The minimum amount of time it was taking was ~200 ms. With hundreds of calls being made to CRM from our custom applications this was undoubtedly the cause of the performance issues, but what could we do if this was the best we could get from CRM Organization Service? The decision was made to test the 2007 SOAP end point against the Organization Service to see if the issue was with the end point itself or just database throughput. What we found was the Organization Service was in fact the issue. The SOAP end point performed the same query for 0 records in a tenth of the time it took the Organization Service under the same load.
Our only solution at this point was to swap out the Organization Service for the 2007 end point for certain functions in our applications. In the end this was the silver bullet and has since been used in all of our implementations. Unfortunately, when we move to CRM 2013 early this year the 2007 end point will no longer be available and we will have to come up with a new solution. My hope is that the issues with the Organization Service have been resolved in 2013 but that’s a blog for another day.