Good question! Another issue that others have brought up is the nose. An experienced gagee knows that they can make all sorts of noises through their nostrils even if their mouth is sealed and stuffed.
I've been really pushing for this multi-layer method by Jam-orbital on deviantart;
http://jam-orbital.deviantart.com/art/The-Jam-Gag-An-Illustrated-Guide-545544786Jam tackles both issues, the throat noises and the nose-based sounds. For the throat, he ties a bandanna loosely over the neck, which would muffle the sound a bit while also keeping the other layers from slipping off the face.
Personally I think the nose noises are a much bigger 'issue' if you want silence. I don't like covering the nose with anything other than 1 very breathable bandanna 'over-the-nose' gag, and I like to do this for cosmetic reasons. Personally I like the noises, it's like nonverbal communication, takes a little figuring out, kind of a challenge, but you are still able to stay in touch with your partner.
IMO no completely safe gag is completely silencing.