Skip to main content
Blog

Data2 min read

What software vendors' studies show, and what they do not

How to read research that a software company paid for.

Software companies publish studies about their own software. They can be useful, and they have limits that a reader should know.

This article reports three Clio findings. It says who ran each one and does not generalize them to other tools or to our work.

A study of mental effort

Clio ran a study with the firm Neuro-Insight. The vendor funded it, and it has not been peer reviewed.

Limits of that study

The tasks were short and done in a controlled setting. The group was small. The comparison was between spreadsheets and PDF files and one software tool.

It says nothing about other tools, about longer real-world work or about what Kalla Connect builds.

Correlations reported by the vendor

Clio also reports differences between firms. These are correlations: firms that grow tend to adopt more tools, and that does not show that the tools caused the growth.

What we did not find

We did not find an independent, peer-reviewed study that measures the effect of automation on U.S. law firms. That does not mean there is no effect. It means we have nothing measured to report.

How to read a vendor study

  • Who paid for it, and who ran it?
  • How many people took part, and what did they do?
  • Is it a comparison or a correlation?
  • Does it say anything about the firm's own kind of work?

Sources

Talk to us about your firm

Book a consultation to tell us about your firm. We will reply to arrange a first conversation.

Book a consultation