Hmmm. Have you tried like a rubber cement or is that bad too?
you could check the hardware department for some glue that will hold onto glass. most craft glues are not good with non porous materials. That is the problem with the hot glue and glass there is really nothing there for the glue to hold on to!