My IOT device is as big as a USB stick. I am building a datacenter for 1 million IOT devices. Each datacenter is a large room with 100 server racks. Each server rack will contact 10,000 devices. I have full design control of the IOT device - MCU/MPU, firmware, and so on.

I need to design a network technology that will allow me to connect all 1M IOT devices to a local network, so that I can address (select) a device and talk it from a central application running on a server, and for each device to be able to talk back the application. Here are some considerations:

• Each device will connect to a PCB board inside the servrer rack using a connector type that I choose, e.g. USB-connector, microSD connector, etc. I can choose any type of connector I need.
• The entire network will reside inside this datacenter. There is no need to connect to the outside world.
• The devices will not need to talk to each other. My requirement is only to be able to address (select) a device and talk it from a central application running on a server, and for each device to be able to talk back to the application.
• I do not have any specific bandwidth requirements. Bandwidth is not critical.
• I need to keep the per-device cost low, so I must avoid choices like putting Ethernet on each device, with cabling, switches, routers, and so on.

I can skip Layer 1 and Layer 2 (PHY and MAC), and use Layer-3 IP-protocol to create a network. So all I need to do is to put a TCP/IP stack into the MCU/MPU on each device. But how do I create an IP network of them?

My understanding is that an IP network is created by a switch, but I don't think I can find a switch with 10,000 ports for each server rack. So maybe I could have 1 switch for each 100 devices (I am OK to use an integrated-circuit switch, or build it with an FPGA core), and then connect 100 switches together inside the rack, so each server rack of 10,000 devices ends up being a small LAN?

Then I could connected all rack servers to another switch (or router?), and end up with 1 big LAN. Would this work?

I am OK with designing and building my own PCB circuits/FPGA for the switches, routers, etc , whatever is necessary.

Other questions:

• How would IP addresses be acquired? i.e. what would provide DHCP on the network?
• There must exist already some networking technology for IOT networks that contain millions of devices. Is there?

This is the best I can think of using existing technologies (like IP), without reinventing the wheel.

I have edited the question after reading about MAC-to-MAC protocol (RMII/MII) here: https://community.nxp.com/thread/316374.

My IOT device looks like a USB stick. I am building a datacenter for 1 million IOT devices. Each datacenter is a large room with 100 server racks. Each server rack will contain 10,000 devices. I have full design control over the IOT device - MCU/MPU, firmware, and so on.

I need to design a network technology that will allow me to connect 1M IOT devices to a local network, so that I can address (select) each device and talk to it from a central application server, and each device can reply to the application server. 

Here are some considerations:

1. Each device connects to a PCB board inside a 2U drawer inside the server rack. I can select any physical connector for this, like USB, or microSD (note that this is the connector - not the protocol). Each drawer can have maybe 400 devices (20x20). The full rack will contact perhaps 10 drawers.
2. The entire network will reside inside this datacenter. There is no need to connect to the outside world.
3. The IoT devices don't need to talk to each other. My requirement is only to be able to address (select) a device and talk it from an application server, and for each device to reply to the application server.
4. I do not have any specific bandwidth requirements. Bandwidth is not critical.
5. I need to keep the per-device cost low, so cannot put Ethernet on each device.

I can skip Layer 1 (PHY), and use Layer-2 (MAC) and Layer-3 (IP) to create a network. I would put a TCP/IP stack into the IoT device and then connect 100 of them in each drawer, via PCB connections, to one or more Layer-2 switches implemented in an integrated circuit (ASIC or FPGA), using MAC-to-MAC protocol (RMII/MII). Then connects these switches from the drawers together among them, in the same way, so each server rack ends up being a small LAN.

Then I would connect all rack servers to another switch (or router?), and end up with one big LAN. Would this work?

I am OK with designing and building my own PCB circuits/FPGA for the switches, routers, etc , whatever is necessary.