Measurement on the main shaft could give more reliable information. But we do not have that.
V-belts should have an effieciency of about 90-96 % according to this reference. But with belts the effieciency do depend on maintenance factors, so of cause it can be bad.
Book on transmission
Worm gears with a gear ratio of 1:5 should have an effiecency of about 90 % according to this reference:
Article on worm gear
So from a theoretical point of view, I do not find a big difference in the two ways of the drive. I guess, that the V-belt may have some more static friction. But in both cases the static friction of the commutator in the motor will be significant.