I'm going to guess it's the shelf support. Those brackets can bend which means they have some give. If possible I would replace them with some solid wood corner brackets. Basically a wood triangle. If possible to put some at the back, I would do that too.