My experience is that it rarely matters — you can try to define precisely what is or isn't IoT, but you're probably wasting your time splitting hairs rather than solving problems.
For a good overview of what different groups consider "IoT", you might want to read What classifies a device as IoT? — you quickly see that one question gives you at least ten different and conflicting views, and I'd suggest you ask yourself "Why do I need to know if it's IoT anyway?".
In direct response to your question, though, I think most authors would consider wearables part of the Internet of Things. Mouser Electronics discuss this in an article on their website, which I found interesting:
For example, the wearable devices’ question on many minds these days is “Are wearable devices going to just be peripherals for a smart phone, or is there a more important role for them as part of the Internet of Things?” If we are really moving toward a more pervasive deployment of intelligence into just about everything in our environment, shouldn’t this apply to wearable devices too?
[...] The promise of the IoT is based on pervasive connectivity and when associated with large collections of connected devices, significant benefits can accrue. How can wearable devices benefit from this concept too? For example, could your wearable devices interact with the devices of others in a crowd? Would you want to know if someone sitting near you on the train had a high fever?
The gist of their article is that the interconnectivity is the key part of whether they'll be part of the IoT—if your device doesn't allow you to aggregate, process and use data through a network, it's probably not an IoT device. For example, a simple pedometer is probably not IoT, but your Fitbit might be!