Cats begin urinating outside the box as a response to something that is wrong, either with them or their environment. It is not just them “behaving badly” and cats don't urinate inappropriately out of “spite”. ... Some cats can also have microscopic crystals in their bladders that can cause irritation.