There are a few things that really distinguish Z-Wave and ZigBee from each other.
The first (as Eirik M noted) is the frequency on which they operate. Z-Wave operates within the 915 MHz ISM band. This gives it reasonable penetration of building materials (better than Wi-Fi) and good overall distance. The fact that few other household devices use that band (now that 900 MHz cordless phones are less widespread) means there's also less interference.
ZigBee can operate at either 2.4 GHz or 915 MHz.1 2.4 GHz is a busy band; it's where Wi-Fi and microwave ovens (among other things) operate. That means that 2.4 GHz ZigBee devices are subject to more interference than 915 MHz Z-Wave and ZigBee devices. They also don't go through walls as readily. (The 2.4 GHz band does give higher bit rates, which is why WiFi lives there (and also uses the 5 GHz band), but most IoT devices don't need to transfer lots of data quickly, so the lower bandwidth of the 915 MHz band isn't a drawback.)
1 915 MHz is only used in North America. Although 2.4 GHz is available worldwide, ZigBee's lower frequency band varies from one regulatory region to another. The various bands are mostly in the 700 MHz to 900 MHz range, so the statements about the 915 MHz North American band are generally applicable to other regions, too.
ZigBee is an open standard, though you need to join the ZigBee alliance (for a fee), if you want to sell ZigBee devices. Z-Wave is a licensed proprietary standard, although the high-level protocol is documented publicly. If you want to make Z-Wave hardware, you have to license the specification from the Z-Wave Alliance and then have your device tested for compliance to the standard. If you buy a Z-Wave device with an appropriately-programmable interface, you can use the already-licensed hardware with the public protocol specification to write your own software.
Because of the lower barrier to entry, ZigBee devices can often be less expensive than Z-Wave devices with the same functionality. Consumer IoT hardware can vary widely in price for many other reasons, of course.
Z-Wave devices tend to have better interoperability overall. When new versions of the Z-Wave standard have been released, they've maintained backward compatibility; any Z-Wave device should be able to communicate sensibly with any other Z-Wave device, regardless of the age or manufacturer of each. (Obviously, newer protocol features won't be present, but the older functionality will be preserved.) Interoperability testing is part of the Z-Wave compliance process. ZigBee doesn't have as rigorous a testing regimen, so it sometimes happens that two ZigBee devices that should be able to talk to each other cannot, due to implementation flaws in one or both devices.
On top of that, ZigBee supports a number of different profiles which all share the same underlying protocol but use different communication details. (This is somewhat analogous to two different HTTP APIs; both use HTTP as a transport, but the Google Maps API isn't going to be very useful if you're talking to GitHub's servers.) Most IoT ZigBee devices use the Home Automation profile, but that's not typically documented on the device, so you can run into unexpected problems. As an example, Philips Hue lights use ZigBee, but do so in a deliberately inoperable way so you have to use the Philips Hue Bridge to control them. (Contrast that with Z-Wave: the Z-Wave certification process requires that any Z-Wave light bulbs use the standard control classes and, thus, can be managed by any compliant Z-Wave controller.)
The ZigBee Alliance is currently in the process of developing a new iteration of the ZigBee protocol named ZigBee 3.0. It looks like part of the new specification's goal will be to increase interoperability among ZigBee devices. We'll have to see how that goes, though. There doesn't seem to e a timetable for finalization of the new standard yet, though.
As long as I've written the above, I figured I'd mention a few things that ZigBee and Z-Wave have in common that differentiate them from other protocols used for IoT devices.
ZigBee and Z-Wave are both mesh networks. Unlike WiFi and Bluetooth, where every device needs to see the controller, Z* devices are okay as long as there's some communication path between them, other Z* devices in the same network, and the controller. (Z-Wave devices will only mesh with Z-Wave devices, and ZigBee devices with a particular profile will only mesh with other ZigBee devices with that profile, of course.)
ZigBee and Z-Wave are both multi-vendor protocols. Notwithstanding the stuff in the "Openness" section above, both ZigBee and Z-Wave have devices available from a variety of companies who often compete with each other. (e.g. companies making Z-Wave light switches include GE, Aeotec, Linear, DragonTech, and others.) Many other IoT-related protocols are single-company silos (e.g. Lutron Caséta); while they might have gateways that let other systems control them, only that company's devices can join the network.