I'm taking a look at the windows IO completion ports today and writing about it. When an IO completion port is created by a process, there is a queue associated with this port that services the multiple asynchronous IO requests. This works well with a thread pool. The IO completion port  is associated with one or more file handles and when an asynchronous IO operation on the file completes, an IO completion packet is queued in the First in First out order to the completion port.
Note that the file handle can be in any arbitrary overlapped IO endpoint ranging from file, sockets, named pipes or mail slots etc.
The thread pool is maintained in the Last In First Out manner. This is done so that the running thread can continuously pick up the next queued completion packet and there is no time lost in context switches.This is hardly the case though since thread may switch ports or put in sleep or terminate and the other workers get to service the queue. When threads waiting on a GetQueuedCompletionStatus call can process a completion packet when another running thread enters a wait state. The system also prevents any new active threads until the number of active threads falls below the concurrency value.
In general, the concurrency value is chosen as the number of processors but this is subject to change and its best to use to profiling tool to see the benefits before thrashing.  I've  a case where I want to read from multiple mail slots and these are best serviced by a thread pool. The Threads from the pool can read the mail slots and place the data packets directly on the completion  port queue. The consumer for the completion port will then dequeue it for processing. In this example, The threads are all polling the mail slots directly for messages. and place them on the completion port.  This is fast and efficient and polling can delay for queues with same or no current message. However, this is not the same model as a completion port notification for that mail slot or a call back routine for that mail slot. In the latter model, there is a notification, subscription model and it is better at utilizing system resources. These resources can be quite some if the number of mail slots or their number of messages are high. we can make the polling model fast as well with a timeout value of zero for any calls to read the mail slots and skipping those that dont' have actionable messages. However, the notification model helps with little or no time spent on anything other than servicing the messages in the mail slots as and when they appear. The receive call seems to have a builtin wait that relieves high cpu usage.