Yes, when using LTE-M's eDRX mode, the TCP connection is still open, but there are some complications depending on which carrier you are using.
First of all, sending data from the server to the mobile device will never wake up the mobile device instantly. The mobile device will wake up at the next eDRX interval (although some other non-cellular event could also cause it to wake sooner).
Now, the confusing part is whether or not your mobile device will receive the data from the server when it wakes at that next eDRX interval. The GSMA LTE-M Deployment Guide actually discusses this in sections 6.2 and 6.4. The troubling line is in section 6.4:
Thus, it means that in case when a LTE-M device is in either PSM or eDRX, mobile terminating messages, depending on MNO choice the messages will either be buffered or discarded
To put it in different terms, the cell service provider (e.g. Verizon, AT&T, etc) gets to decide whether they want to buffer the TCP packets to forward your device when it wakes up later, or just discard them.
Even on carriers which do buffer data during eDRX sleep periods, they tend to have an upper time limit on the order of minutes.
However, your server is likely unaware of the intermittent nature of the eDRX connection, and will be doing it's normal TCP retries, and there is always the chance that one of those retries will be sent during the eDRX window and get through to the mobile device. When this happens, it looks like it "just works" even on networks which do discard data, but it's more luck than anything else. (Shortening the eDRX cycle time and using faster TCP retry rates on the server will both improve your luck in this setup.)