Mice enjoy wheels a lot. If you put a wheel in your cage, you'll have a mouse running in it all the time! You should be careful with the wheel you pick for your mice, though. Metal or plastic rung wheels are not safe for mice because they can catch a paw or their tail in it and break it. Plastic or metal solid wheels are good for them. I like the metal mesh wheels, as they are safe for their feet/tails and almost impossible to destroy. Also, pet stores sell these tiny "mouse wheels" that are about 3-4 inches in diameter. I suggest that you get them a wheel that's a bit bigger, the size you would get for a syrian hamster or gerbil. This is because mice are small, but have long tails, and their tails tend to curve upward when they use the tiny wheel much. Here is a picture so you see what it looks like.
 Wood toys are also good chew toys as mice need to chew to control the growth of their incisive teeth, so at least a small wooden toy is a must in a mouse cage. You can also provide cardboard for your mice to chew and hide in them. Paper towel and toilet paper rolls are good, also cereal boxes. Avoid printed cardboard that didn't had human food in it (like a printer ink box, for example) as it can be toxic. Also avoid boxes of frozen food, because they are covered in a layer of wax which can also be toxic.