Qualitative insight into the industry’s top vendors through 18 key performance indicators

Black Book utilizes a three-step process to collect candid performance data.


Black Book collects a series of direct evaluations covering 31 performance areas on leadership and senior management which comprise the scored ranking of the “Black Book 50 Best Managed Vendors”. These scores are isolated and used in a like comparison of all outsourcing vendors of all domains.


Black Book collects ballot results on 18 performance areas of operational excellence to rank vendors by outsourced service lines. Client users submit ballots per vendor per function which are juxtaposed with client size, functions outsourced, industry/vertical(s) affected, country of service origin and overall outsourcing domain.


The gathered data is subjected immediately to an internal and external audit to verify completeness and accuracy and to make sure the respondent is valid while ensuring the anonymity of the client company is maintained. During the audit, each data set is reviewed by a Brown-Wilson executive and at least two other external reviewers. In this way, Black Book report clients are able to clearly see how a vendor is truly performing. The 18 criteria on operational excellence are subdivided by the client’s industry, market size, geography and function outsourced and reported accordingly.

Understanding the Statistical Confidence of Black Book Data

Statistical confidence for each performance rating is based upon the number of organizations scoring the outsourcing service. Black Book identifies data confidence by one of several means:
    • Top-10-ranked vendors and advisors must have a minimum of five unique clients represented. Broad categories require a minimum of 10 unique client ballots. Data that are asterisked (*) represent a sample size below required limits and are intended to be used for tracking purposes only, not ranking purposes. Performance data for an asterisked vendor’s services can vary widely until a larger sample size is achieved. The margin of error can be very large and the reader is responsible for considering the possible current and future variation (margin of error) in the Black Book performance score reported.
    • Vendors and advisors with over 10 unique client votes are eligible for top 10 rankings and are assured to have highest confidence and lowest variation. Confidence increases as more organizations report on their outsourcing vendor. Data reported in this form are shown with a 95% confidence level (within a margin of 0.25, 0.20 or 0.15, respectively).
    • Raw numbers include the quantity of completed surveys and the number of unique organizations contributing the data for the survey pool of interest.

Vendor achieves the overall preference for expertise, appreciation and insight within the client’s respective vertical. Organizational structure meets or exceeds the needs of stakeholders and customers in this industry. The client is overwhelmingly likely to recommend the vendor for an outsourcing engagement for the same vertical outsourcing initiatives.

Customers are also continuing to push the envelope for further enhancements to which the outsourcing vendor is responsive. Clients also believe that their vendors’ technology is helping them compete more effectively, generate larger revenues and profits and cut their overhead in ways that were difficult or impossible to accomplish before innovative outsourced services were introduced.
Outsourcing leadership provides significant and meaningful training opportunities for employees and client staff. Leadership strives to develop technology staff, customer service and consultant employees in particular.
Outsourcing leadership honors customer relationships highly. The relationship with the vendor elevates the institution’s reputation. Improving customer efficiency and effectiveness is a priority of the supplier. Governance of engagement is neither complex for buyer nor does it require vendor management attention regularly. There is no regular transparency or quality issue. Account management liaisons affect relations positively.

Trust in enterprise reputation is crucial to the outsourced services client as well as prospects. Client possesses an understanding that their outsourcing organization has the people, processes and resources to effectively deliver the desired business results, based on its industry reputation and past performance.

Industry recognized horizontal functionality and vertical industry applications to manage bundled managed services. Vendor routinely drives operational performance improvements and results in the areas they affect. Multiple offerings allow vendor to meet the progressive needs of the client. The delivery of services meets or exceeds standards of outcomes.
Client maintains expedient deployments on all initiatives. Outsourcing solutions eliminate excessive buyer supervision over vendor implementations. There is a minimized need for consultant management of deployments due to highly astute vendor staff. Services vendor overcomes implementation obstacles and challenges effectively. Technical, organizational and cultural implementation obstacles are handled professionally and timely. Implementation time exceeds expectations.
Outsourcing products and process services are customized to meet the unique needs of specific client purpose, processes and models. Little resistance to changing performance measurements clients’ needs vary.
Managed services vendor supports interfaces so information can be shared between necessary applications. Solutions are easily integrated to existing backend systems. Seamless interfaces to legacy applications are performed as required for optimal functioning. Human integration and interface activities are administered precisely. Cross-industry and corporate services expertise is evidenced by execution and orchestration of multiple systems and processes.
Vendor provides flexible outsourcing pricing, allowing client to choose and pay for the precise functionality and services needed. Invested in significant infrastructure and have the ability to provide services to enterprise organizations. Technology and business process products and services meet the changing and varied needs of the customer. Vendor adapts to the client needs with appreciation and elasticity.
Vendor is focused on building and developing a strong employee team of producers. Employees act like owners/leaders. Company is moving towards leveraged pay at all levels. Implications of the “contractor mentality” are minimized. Vendor is using effective tools to tie performance metrics to compensation policy and compensating top leaders. Human resources- related criteria are scored from the client perspective on this indicator.

Vendor meets agreed terms as evidenced by routine, acceptable service level reporting and industry expectations. Depth and breadth of applications/solutions are acceptable in meeting client needs. Online reliability is maximized and outages/downtimes are minimized. Solid product and outsourcing service capacities are demonstrated consistently. Service levels are consistently met as agreed. Services and support response is maximized by vendor team

The vendor’s sales and marketing processes accurately and appropriately represent service deliverables. Image is consistent with top service rankings. Sales presentations and proposals delivered upon and corporate integrity/honesty in marketing and business development are highly valued. Company image and integrity are values upheld top-down consistently. Vendor maintains high image and reputation so that customers of outsourced services openly reference their vendor’s representatives as client team members with high level of relevant client communications.
Outsourcing vendor’s cost savings are realized as generally estimated and not over positioned or over/underestimated in ways that effect major client satisfaction or costs. Savings expectations such as decreased service handling and support staff are realized. Buyer’s managed service support costs are reduced significantly. Vendor offers value-adds as a cross- industry outsourcing partner in cost savings management initiatives and creative programs through bundled delivery design. Vendor provides true business transformation opportunities to buyer and opportunities for top line contributions.

Vendor’s viability, employee turnover, financial stability and/or cultural mismatches do not threaten relationship. This outsourcing vendor takes extraordinary efforts to avoid relationship problems among service providers and the client as well. Senior management and board exemplify strong leadership principals to steward resources appropriate that impact buyers.

In order to provide a secure and constantly dependable service offering for product development entities, an outsourcing vendor has to provide the highest level of security and data back-up services. In some cases, you may find their service in these two areas superior to the security and back-up system in your own firm/corporation.

Outsourcing leadership provides an adequate amount of onsite administration and support to clients. There exists a formal outsourcing relationship and account management program that meets client needs among suppliers for the customer. Vendor evidences successful management strategies or services. Media and clients reference this vendor as a service leader and top producer correctly based on evidenced support programming. Customer services and relationship satisfaction is manifested through significant flagship clients as well as smaller and newest customers similarly. Customer satisfaction is manifested as a vendor priority without compromise.

Related technology services and product management are considered best in breed. Vendor technology elevates customers via capabilities, equipment,  processes, deliverables, professional  staff, leadership, quality assurance  and innovative initiatives. Technology services are delivered at or above current/former in-house service levels. Quality assurance and process improvement is integrated into all initiatives. Vendor leadership drives continuous services improvements to point of client service