Users who are new to Validation Manager often ask us for a recommendation on how to organize their oncoming big verification in Validation Manager.
This is a very good question to consider.
Most laboratories are used to organizing their verification so that each analyte will have their own report files. The reason for this is usually that it’s practically the only possible way to manage the data.
In Validation Manager on the other hand, you can make deliberate choices based on your needs.
But if you’ve never got to do that, it may be difficult to know how to do it. After all, you would need to recognize the patterns that at the moment add unnecessary complexity to your work.
Being used to the restrictions related to the old data management system probably affects the way you think about verifications. That’s why when talking about verifications, people often think of a verification of one analyte on one instrument.
The best success stories come from organizations utilizing Validation Manager to take their verifications to the next level, rather than to maintain their old workflows. Our customers are able to cut more than 95% of data management time and reap more benefits from their verifications than ever.
How to define a verification project?
Validation Manager gives you detailed reports for single analytes run on single instruments as you probably expected it to do. But that’s not the level of reporting that you should be thinking about when creating your project into Validation Manager.
A better way to approach a verification project would be to think about the folder that contains all your data related to e.g. verifications of a new instrument system. Or you could think about a binder into which you have gathered all the printed verification reports related to the instrument system.
Generally we advice to think of one project as something that gives answers to high level questions. A good question could be “can we take the new line of chemistry instruments into use“. Your project aims to answer this question, and therefore it should contain pretty much everything that’s relevant for that decision.
How many studies to have within a project?
A study is for one type of statistical analysis to describe a performance characteristic (e.g. precision). One project can have many of them.
Within a verification project, each study can handle multiple instruments and analytes. So when considering the scope of a study, you shouldn’t divide your work into single analytes. Instead you should include into one study everything that is supposed to answer a question like
How does the behavior of the new method differ from the old method when measuring patient samples of different concentrations?
How much variance is there when doing multiple replicate measurements over multiple days?
So basically, most verification projects can be taken through with one project containing only two studies:
One ANOVA study to measure precision of all new methods
One comparison study to compare all the new methods to old ones
In some cases, adding a third study for parallel instrument comparisons is a good idea. And sometimes you may want to have a precision study for pooled patient samples and quantitative accuracy study for controls. But in the end, all you need is a handful of studies, even if you are verifying a large set of tests on multiple instruments.
So if you want some kind of a rule of thumb on how to organize your verification project, it’s here: Have only one study for estimating each type of performance characteristics. Include all the instruments and analytes within each study.
Experience has shown though that this is not the whole truth. How to best organize the verification is pretty much up to what you really need to get out of the report. Sometimes it may be meaningful to divide the analytes into groups, for example having one comparison study and precision study for electrolytes, and a separate comparison study and precision study for metabolites. That way each group will have their own chapters on your report. And the study names are used as chapter headings on the report.
The most important things to remember
When you are new to Validation Manager, it may feel difficult to make these kind of choices. The most important things to remember are
One report can only include content related to one project. So to make sure that it’s easy for you to fill your reporting needs, think big when considering the scope of a project in Validation Manager.
Avoid studies that only contain one analyte, especially in big verifications, where there are lots of analytes that need to be verified.
It’s not always easy to find the best solution before you have tried it. But there are many ways to do things well enough.
If you really need to change something in the structure of your ongoing verifications, it’s rather easy to create new studies and copy content from one study to another within the same project.
They say that practice makes perfect, and that applies to using Validation Manager too.