What are the comparative benchmarks?
We measure every firm against the average score given by clients to all firms.
For a firm-wide score, that will be the average score for all clients of all firms.
For a practice area (eg Real Estate), that will be the average score for all clients in that practice area (for instance, the scores given by all Real Estate clients for Real Estate work). We do this because there are different average scores in different practice areas.
So, the score for any firm shows its variance from that average (baseline) score.
How do you measure Client Satisfaction?
We measure Client Satisfaction across 15 criteria:
- LAWYERS/TEAM QUALITY
- Partner availability & engagement
- Quality of partners
- Quality of associates
- SECTOR/INDUSTRY KNOWLEDGE OF THE SECTOR
- Industry/commercial knowledge of the sector
- Profile in the sector
- VALUE (BILLING & EFFICIENCY)
- Appropriate resourcing
- Efficiency in delivering the legal product
- Billing: transparency
- Billing: value for work done
- Communication & case matter management
- CONSISTENCY SCORE
- OVERALL CLIENT SERVICE SCORE
Market research: fact, or perception?
Our surveys are snapshots of client perceptions. They reveal what clients think of a firm.
The survey will be factually correct – even if the client perception is incorrect. For instance, the data may show a firm scoring relatively poorly for ‘Industry/Commercial Knowledge’ within a particular practice area; if so, the survey is not saying that firm does actually lack Industry/Commercial Knowledge – what the survey is saying is that clients perceive it as lacking Industry/Commercial Knowledge.
Our data therefore reveals a firm’s perceived strengths and perceived weaknesses.
Why the market perceives a firm in a particular way, and what remedial measures may be taken are beyond the scope of our standard surveys. But we do offer bespoke research services to help answer those additional questions.
Are scores relative or absolute?
We focus on the relative performance of a firm compared to other firms.
This is not something a traditional client satisfaction, or client feedback, survey can do – they will not have sufficient comparative data to be able to compare one firm with another. Instead they merely rely on absolute scores.
A traditional survey will be restricted to looking at a firm’s absolute, not its relative performance.
Illustration: a firm may score 65/100 for ‘Billing: transparency’ within ‘Corporate and Commercial’, but that absolute score is meaningless unless measured against the ‘Billing: transparency’ scores of other ‘Corporate and Commercial’ firms. Thus, if the average for all other firms is 70, the seemingly good absolute score of 65 is revealed to be a poor relative score of -5%. And the firm is not doing as well as it thinks it is.
Without knowing the scores of other competitor firms, it is not possible to do a relative comparison. Only Legalease Research Services has sufficient data (both UK- and worldwide) to measure relative performance.
How do you measure the average (baseline) scores of different practice areas?
The volume of our data (7 million + data points), reveals differences of up to 10% in average scores between different practice areas. This is because different types of clients, in different practice areas, have different expectations of their lawyers. Accordingly, different client-types, in different practice areas, will score differently, and average scores will vary from practice area to practice area.
This becomes particularly important when making internal firm comparisons – comparing the performance of one department/practice area in the firm against another in the same firm. A traditional client satisfaction, or client feedback, survey might give a simple comparison of the absolute scores of two practice areas – but that will almost certainly be misleading, because you need to know the relative average scores for both those practice areas:
Illustration: a firm organises a feedback survey of hundreds of its clients. Dept A scores 72, and Dept B scores 69 (ie Dept A scores 3% better than Dept B). Management rewards Dept A and penalises Dept B. But if the average score of clients doing Dept A work across all firms is 77, and for firms doing Dept B work it is 67, then Dept A scores -5% whilst Dept B scores +2% (ie Dept B’s relative score is 7% better than that of Dept A – even though Dept A’s absolute score is higher).
In short, you can only compare one department (or practice area) against another if you adjust for the different average (benchmark) scores given by clients in those work areas. To make an absolute -v- absolute comparison is meaningless (and flawed). Only a relative comparison will give a fair and accurate comparison of practice areas. And only Legalease Research Services has sufficient data to be able to do that.
What are Bands of Uncertainty?
Each score is marked with its band of uncertainty. This shows the statistical confidence of the score.
This illustration shows a positive relative score of 7.17%, with a certainty/uncertainty band of +/-0.26%. So the score could be as high as 7.41%, or as low as 6.91%.
If a score has a larger band of uncertainty, that will, typically, be because the range of scores for the firm was unusually wide and inconsistent. That uncertainty does not mean the result is not statistically valid, though it may well increase the extent of the statistical error bands.
Response rates vary from firm to firm and from practice area to practice area. It is important to note that getting more responses (replies) does not, typically, alter the score, other than within the existing band of certainty/uncertainty (although an increased number of scores will, typically, reduce the size of the band of certainty/uncertainty).
What is consistency scoring?
Our Consistency score assesses the range of scores submitted by clients across the 10 base-level client service criteria We take 95% consistency as our benchmark score for consistency. The score given for your firm indicates the variance of client scores from that 95% benchmark. We expect there is an inherent positive bias in the feedback we receive, and the Consistency is a measure of this. If you see negative variances to the Consistency Benchmark, it may be a measure of the degree of positive bias we observe, conversely positive variations may be indicative of a negative bias we observe.
Can we publicise our good results?
Yes, but you can’t publicise the scores of other firms.
Our data reports are designed to inform business decisions. Typically, they are bought by Managing Partners or Heads of Department to attain an objective insight into a firm’s strengths and weaknesses compared to its market competitors.
By providing the scores of competitor firms, we enable those business leaders to better understand their own firm’s performance. But those competitor scores are confidential. They are disclosed for business planning purposes only – not for comparative marketing purposes. So, no firm can reveal the score, or relative performance, of any other firm.
Our data is copyright. It is provided on the strict understanding that (i) it will not be shared with any third party (ie outside the firm that is buying it), and (ii) the specific listed scores and performance of all other firms will remain confidential.
Which practice areas can I select in my UK Drill-Down subscription?
You can only select practice areas for which we have sufficient statistically valid data for your firm. If there is not enough data, then talk to us and we can work with you to get more data.
Our standard UK practice areas are: Banking and finance; Commercial litigation; Competition; Construction; Corporate and commercial; Crime, fraud and licensing; Dispute resolution; Employment; Family; Insurance; Intellectual property; IT and Telecoms; M&A; Media and entertainment; Planning and environment; Projects, energy and natural resources; Public sector; Real estate; Risk advisory; Transport; Wills, tax and trusts. Additional, more specialist, practice areas are available on request.
What are practice area competitor firms?
The UK Drill-Down Subscription includes an analysis of how your firm does in selected practice areas.
We only compare your firm with leading or recognised firms in each practice area for which we have statistically valid data: we do not compare to all firms that might do some work in those practice area. So, there may be a considerably smaller number of comparator firms in a practice area analysis than there will be in a firm-wide comparison (when we do include all firms – typically, 800+ firms).
Why is my firm not included?
We only include firms for which we have statistically valid Client Satisfaction data.
If a firm’s scores do not meet our statistical standards, then that firm is not included. Typically, this will be because of a lack of response data or an unacceptable variance in the spread of those responses – both of which can usually be resolved by our working with the firm to increase the response rate (and flatten out any inconsistencies).
If your firm is not included, then contact us. We will work with you (free of charge) to ensure that we do have statistically valid data for the firm. Contact enquiries@legaleasedata.com
Why is my firm ranked highly in The Legal 500 but not in your Client Satisfaction scores?
The Legal 500 looks at a range of criteria when deciding how to rank firms. These can include: the firm’s resources; quality of client base; quality and scope of deals, transactions, litigation etc; peer recommendations; the reputation of the firm among its competitors; reputation within the market generally (including the Bar, where relevant); client satisfaction; performance record in recent years; quality of management; firm culture; recent arrivals or departures; and more. Our editorial staff then make a professional judgement which takes into account the factors that seem most relevant to them. Importantly, Client Satisfaction is only one of the criteria taken into account. So, while Client Satisfaction will be a factor, it will not necessarily be the overriding factor. This means it is possible for a firm to score well (or less well) in a Client Satisfaction survey, but be ranked low (or high) in The Legal 500.