EBME Forums Forums Free discussion Free discussion [poll]What do YOU use to check the accuracy of infusion devices?

For the burette you might have to choose different delivered volumes to maintain the level of accuracy you require using a particular accuracy burette, e.g. if you deliver 50mls into your 100ml burette then accuracy is essentially 0.2ml/50ml or +/-0.4% (of reading) because the basic accuracy of your burette is at full-scale. Same thing as with accuracy on analogue gauges and suchlike - accuracy can be stated as a percentage of full-scale or of reading.

Don't you see that the basic full-scale accuracy of the 10ml burette is just the same as the full-scale accuracy of the 100ml burette? Actually a 0.2% basic accuracy is less than 10% of the stated system volumetric accuracy of a volumetric-pump (usually 5%). This means that the figure you read off the burette is basically pretty close to that of the actual delivered volume. In a practical workshop setting you can pretty much say that if you can ensure the accuracy of the measuring device is less than 10% of the tolerances of the device you're measuring then you can ignore the measurement inaccuracy and take the reading as the delivered volume (including errors in the pump only).

What makes you think your test has failed? What are you trying to achieve from the measurement? Your burette has a basic volumetric accuracy of 0.2ml /100ml*100% or 0.2%. Thus it has a volumetric accuracy of 0.2ml /75mlor 0.26% if 75ml is delivered. Overall the %age error indicated by the burette in your test was: (75-79/75)*100 = +5.3%

The indicated error is a combination of pump error in delivery due to tolerances in performance and burette error due to measurement uncertainty (inaccuracy). Hence error in pump delivered volume is: 105.3% +/-0.2% = 105.1-105.5% delivered by the pump. If I measured this initially I would look to the pump or the test setup since 5.1-5.5% over-delivery indicates accuracy is marginally unacceptable (assuming a spec of 5%). I would probably repeat the test, possibly average 5 results (Baxter suggest this in some manuals) and if the values for pump tolerance do not improve then I'd look for wear and tear or damage to the pump.

Your initial results show that the IDA gave an indication for pump accuracy that is potentially worse than that indicated by the burette. This may be because the potential for measurement uncertainties could be greater with the IDA, i.e. the specified volumetric measurement accuracy is >0.2%. What is the stated volumetric accuracy of the IDA? Is this stated accuracy consistent at all rates/volumes delivered?

At different rates the manufacturer should tell you the specified accuracy of the IDA and you can determine the accuracy expected of the burette yourself. Hence given the overall error from a volumetric measurement (at the same rate, over the same time, with the same setup and method) you should be able to estimate the tolerances in pump delivery. Assuming the same pump is used at the same rate, over the same observation time, for each method, of course.

Then if there is a discrepancy between the estimated pump tolerance between the burette and IDA measurements, even when you take into account the specified errors in the burette and IDA measurement, then you need to investigate whether the IDA, the burette or the method you're using on either has a problem, i.e. does not meet it's specified/estimated tolerance for accuracy, or whether there's variation in the same pump between each test.

If you're trying to compare accuracy of one method to another, i.e. burette to IDA, then I would have thought that you need a benchmark or gold-standard to compare them both to, i.e. gravimetric method as used in type-testing and evaluation of infusion devices. But it's pretty obvious really that if you use one method of measuring volume that has a basic accuracy that is better than another then it's likely that this will produce an indication of delivered volumes that are more representative of the actual delivered volumes of the pump hence there's going to be some discrepancy between indicated readings obtained by measuring instruments that have different system accuracies.

In this case the error in actual delivered volume and the nominal volume expected is predominantly due to tolerances in the pump: Error = (actual reading - nominal reading/nominal reading)*100% plus/minus the worst-case accuracy of the measuring instrument as a percentage. Therefore with any measurement you end up with %age pump accuracy +/- %age worst-case measurement uncertainty. For your pump 105.1-105.3% delivered volume assuming 0.2% accuracy of the burette or 104%-108% delivered volume assuming 2% volumetric accuracy of the IDA. What is the specified accuracy of the IDA you’re using?

This aside; once you've established that the volumetric errors in the burette and IDA are within their specified tolerances then I suppose you can then determine which method gives the most appropriate readings over the range of rates/volumes you require to test pumps at.

If you're trying to determine whether burette or IDA is most suitable for pumps and/or syringe drivers given typical rates they operate at then perhaps you need to select one pump and one syringe driver and then select a high, medium and low rate tests for each using trumpet and startup curves to determine the minimum observation windows you need to do the measurement over. You then obtain results using each method that you can compare to each other.

Perhaps you could obtain a pulsatile delivery pump and then do a comparison between the estimated pump volume delivered using the burette (minus burette measurement tolerances) and the volume indicated by the IDA (minus IDA measurement tolerances).

Richard, a very good treatise of the subject of measurement. One comment; I have always considered that with a burette you are making two measurements for each test, a zero setting 'measurement' and a volume delivered measurement, and so the inherent accuracy, resolution/test volume, should be muiltiplied by two. Is this correct?

In practice, the zero or datum reading should be taken from the bottom of the fluid meniscus each time. The zero error is usually slightly positive with respect to the zero-line, if anything, since you can actually see if the reading is above or below it. You add the potential zero "error" to the volume delivered since this is, in effect, a zero-offset not a span-error. When considering the zero-error we would need to “subtract” it from our overall measurement if we were being pedantic.

100ml burette, 0.1ml resoultion, basic accuracy 0.1%. E.g. Zero error somewhere between zero and 0.1ml. 75ml delivered (estimated accuracy of burette 0.13%) 79ml indicated (including zero and span error) hence error overall is ((79-75)/75)*100 = +5.3%.

The fixed worst-case zero error is 0.1ml (and usually positive) hence delivered volume is actually between 79-78.9ml by subtracting this +ve offset error from the volume delivered. The overall error in delivered volume taking into account fixed zero offset is ((78.9-75)/75)*100 =+5.2% Assuming accuracy of 0.13% then tolerance in the measured parameter, i.e. pump, is +5.3%+/-0.13%(basic accuracy of burette) -0.13% (fixed zero error) = +5.07 to -5.43% of pump nominal (75ml).

Just multiplying the error by 2 does not take into account the zero being positive (i.e. it would include -ve errors) which it usually is in practice otherwise the measurment would not be adequately zeroed.

Gentle Men,

There is but one thing that strikes me as being more important than the accuracy or otherwise of the test methods given much coverage by many learned people on this quest by K.I.T.T.

Our exalted Sage, Monsignor RoJo hit it on the head on day one of this request for help, in fact he was indeed the first to reply.

As Monsignor RoJo said:

“If you are looking at accuracies of drug delivery systems look at the whole system not just the electrical pump. IV pump accuracy often depends on the diameter and elasticity of a piece of tube, I would guess this would be more prone to inaccuracy than the pump, especially as time passes and the properties of the tube material changes.”

Hit on the head why don't you, spot on!

This plastic tube, especially the RMC9608 Solution Administration Set supplied by Baxter has been proven over many years to be very inconsistent in accuracy. Syringes by various manufacturers are also prone to large variations in accuracy and stiction.

Having just repaired a Flogarde 6201 Volumetric Infusion Pump I placed it on test using our IDA4+ with Hydrograph, using a new standard admin set as above type. One hour later the displayed graph and printout showed a flow rate, which equated to an error of minus 4.53%. This was not believed because as we all know the 6201's are inherently accurate. Not like the Colleague!

The test can be done, my learned student friend, by using a standard Baxter Flogard Administration set or Baxter calibration tubing; my preference is to use the standard admin set because that is used on the patient.

Having from experience decided this test was inaccurate and was in fact probably the solution admin set, I then used an old set – used twice before – same parameters/set-up. Result – error now minus 1. 23%. Pump's a good un! Test equipment is also accurate, but the solution set wasn't, and consistently are not, even in the same batch.

This however K.I.T.T., is not always the case with other similar products of different manufacture. Having just also calibrated an Alaris Signature Gold I.V. Pump, using the manufacturer's specially supplied Volume Calibration Set; I then tested it using a standard – ward use – solution set.

The calibration set gave a pump volumetric accuracy of plus 0.15%. With the standard solution set – using the same test parameters – it gave a volumetric accuracy of minus 0.75%.

This test K.I.T.T., was done gravimetrically, using a glass flask on a 0.1g resolution, high precision top pan balance, which has a tolerance of +/- 0.3g, [actual deviation upto 500g however is nil.] This is calibrated in-situ annually by the manufacturer - Avery Berkel. [Now Avery Weigh-Tronix] To ensure good delivery – no bubbles or other fluid errors, the pump is run for a short time to prime the system and the balance is tared out before starting the test proper.

What you could do K.I.T.T. To make an accurate gravimetric testing device is to use the output from the RS232 interface of the balance to a computer.

Then write a programme that gives a real-time graphic display, with auto tare on start, programmable start/finish times, date/time annotation, user entry fields for: set rate, set volume, pump tolerance for rate and time, and who is doing the test & on what balance, etc. With the final test results given as an average rate [ml/h], % rate accuracy, delivered volume [ml], % volume, and delivery time accuracy.

You might then be close to our in-house developed system which, may I add, the IDA4+ compares very closely to. [With-in +/- 1%.]

Best of luck.
Ian

Most if not all manufacturers specify system accuracy under stated test conditions (these include giving set, temperature of test, rates, etc, etc) so any element that can affect system accuracy, taken in isolation, is irrelevant. The pump is actually required to create a positive pressure and regulate the fluid through the giving set (to within system tolerances)otherwise we would just rely on the giving set on it's own and have one less inaccuracy (pump) to think about. Any giving set out of the box with any pump designed to use it should work within the system tolerances - that's what it's all about - otherwise why bother.

The whole point of making accurate measurements is to find these problems. I think you're missing this point Ian, which is why measurements are carried out - to able to detect the sort of problems you're discussing with sets. Of course there will be sifferences between sets but how much and is it acceptable (tolerances)? Use a burette, IDA or gravimetric method, with adequate accuracy for the job, and you should be able to determine system tolerances, i.e. if pump delivery is outside tolerances then you have a problem with the pump or the giving set.

Whether it's a burette (direct volumetric measurement), IDA or weighing scales (derived mass measurement) - the volume of the fluid delivered is what's used to measure system volumetric accuracy - fullstop. The point I've been trying to put over for KITT's benefit is that measurement of volumetric accuracy and above all the methods you use to achieve this is important whether you're comparing one method to another or just trying to make an accurate measurement for the sake of repeatability.

As per my previous posting and highlighted by Ian, the disposables will have a bearing on the "accuracy" of any measured flow rate etc.
All medical plastic components, in particular tubing, have an engineering tolerance with regard to I.D. and shore hardness (elasticity). In particular, shore hardness can have a tolerance of +/-5 which equates to over 7.5% for typical PVC tubing.
Therefore, you can take two new IV sets from the same batch and there could be a difference in ID and elasticity which could have a marked bearing on the flow characteristics.
I think the whole scenario shows that if we want "accurate" readings there are going to be inaccuracies which can be factored in or not depending on how pedantic you want to be.

Ian,

I can't understand the point of your previous posting - lots of red-herrings and all it pointed out to me was that pumps can operate within a band of tolerances specified by the manufacturer when we change system components. That is patently obvious if the manufacturer actually quotes a volumetric tolerance, e.g. +/-5% and specifies the conditions the tests should be done under. Irrespective of measurement method aren't all of the errors you discussed earlier actually within the manufacturers specification for a floguard pump? Or does it deliver to a system volumetric accuracy of 2% or 3% or 4%? When you stated it was inaccurate did you mean to say the results were not typical of the majority of floguards?

When we do use appropriate test-kit and make measurements we're not really interested in fluctuations within these tolerances are we? If so that is being pedantic since system volumetric accuracy (of the pump system not measurement system necessarily) likely to change for a whole host of reasons - the most significant ones being giving set, temperature, rate that the volume is delivered at, etc, etc. All that matters is that we perform an accruate assessment (one that we can justify) of the pump system accuracy with a standard giving set to give confidence (and provide a record) that the pump is operating within tolerances after servicing, PPM repairs, whatever.

The original gist of this thread was about whether IDA and other devices are up to the measurements we are using them for or not - I think KITT may have been "on a mission" to compare burette accuracy to IDA accuracy. Personally I don't think worst-case accuracy of +/-2% such as the specification on our infutest IDAs is good enough for measuring syringe drivers with a system accuracy of 2% and I've explained why previously but its precisely because of worst-case tolerances and suchlike. If your IDA is within +/-1% of your gravimetric tester then its just over +/-1% accuracy (for that single measurement not taking into account worst-case tolerances) that may just be acceptable for testing sryinge drivers but is the +/-1% you estimated the manufacturers specified tolerances?

The only thing that would concern me about choosing any measurement method is if it produces significant errors that mean we can't determine whether the pump is actually operating within manufacturers tolerances or not, i.e. measuring instrument tolerances are greater or approaching the pump system tolerances as I've mentioned before.

Mr Ling,

To refresh your memory, from the very first posting on 09 August 2005 by K.I.T.T. who started this discussion thread.

I quote:

“I'd be grateful if you fellow forumers would answer the above question on what you use to verify whether your infusion pumps, syringe drivers, etc are accurate enough. Do you use devices like the IDA2/4, a burette, weighing the solution or something different? And why did you choose that method?

The reason I ask you is as follows:

I've now ended up on infusion device analysis as I feel this is one field in which it would be possible to carry out research into and be able to come out with an outcome. My main objective is to find out which is the most accurate by performing various analyses using the different techniques. It would also be quite easy to build a new device for this task in the EBME dept. etc.”

Richard, the red herrings my friend, percentage wise, [100%] are all of your own making, with much verbal fluidity, and bovine waste!

Good luck K.I.T.T. [or is it Ash] with your task.

Ian

"To whom much is given, much will be expected.'

Ian,

I discussed the burette, some problems with it, how I use it and inaccuracies with it - you discussed the gravimetic method. Any of these methods is suitable if applied correctly although have reservations about 100% automated methods, e.g. IDA-like devices, used with some devices, e.g. syringe drivers (on the limits acceptable accuracy) and pulsatile delivery devices (problems with accuracy fullstop).

I think what I've posted is actually quite accurate even if it's wasted on you Ian and I certainly wouldn't say what you posted is bullsh*t, which is what you've said about what I've posted. It was longwinded because that's how I am and anyway it was intended for someone with no experience to read and digest for future reference if they think its valid.

Having developed your own gravimetric system I guess you're biased towards it to the extent that you feel the use of burettes and understanding of instrument errors is beneath you.

However I think it's important to have an appreciation of the basics beyond the "if I have a 5% tolerance pump system then I use a 5% tolerance measuring instrument" attitude which is a bit slack in my opinion.

It was intended for KITT hopefully as a means for him to get the most accurate measurements from his £65 pounds worth of burettes. Even though you and I know that the gravimetric method is the way to go for absolute accuracy and repeatability.

All we need to do on the bench is choose a method that allows us to say with confidence what the pump system tolerances are within spec. You have not respnded to my previous post regarding your findings with floguard and ALARIS pumps thus I can only assume that your pumps are all within specification despite the diatribe about variation in giving sets.

If anyone thinks what I've posted is bull**** and more importantly inaccurate or mis-informed then please comment - it will be taken in the spirit intended.

Richard,
I think kitt appreciates your comments and everyone elses. It is up to him how he uses the advice he gets from this forum. Ian, I like the fact that we allow 'free speech' on this forum but please (and anyone else who feels the inclination) refrain from cryptic and possibly offensive remarks. As an administrator for this forum, I do not think it is productive and could put off some members from joining in.

Thank you, John.