Two things that come to mind is maybe injecting some oil into the little motors. In and around, just to lube it all up. Might reduce some noise.
The clear type oil.I'm thinking the kind for knife blades, but I imagine hobby shops have something, some kind of oil for moving parts in remote control cars and planes.
Oil doesn't affect electronics, so that's not a problem. But Hobby shops might have something for just this kind of application.
The other would be like was said already, Just wrap the servos in foam, without obstructing the moving parts.
I guess the denser the foam the more it will muffle the sound.