This is a kernel module that attaches to a real interface on the host and filters and injects packets.
In the big picture we're one of the three trunk interface on the internal network, the one named "NIC Filter Driver":
The main challenge here is to make sure the netfilter and internal network instances won't be destroyed while someone is calling into them.
The main calls into or out of of the filter driver are:
- Async send completion (not implemented yet)
- Release by the internal network.
- Disappearance of the host networking interface.
- Reappearance of the host networking interface.
The latter two calls are can be caused by driver unloading/loading or the device being physical unplugged (e.g. a USB network device). Actually, the unload scenario must fervently be prevent as it will cause panics because the internal network will assume the trunk is around until it releases it.
- Need to figure which host allow unloading and block/fix it.
Currently the netfilter instance lives until the internal network releases it. So, it is the internal networks responsibility to make sure there are no active calls when it releases the trunk and destroys the network. The netfilter assists in this by providing INTNETTRUNKIFPORT::pfnSetState and INTNETTRUNKIFPORT::pfnWaitForIdle. The trunk state is used to enable/disable promiscuous mode on the hardware NIC (or similar activation) as well indicating that disconnect is imminent and no further calls shall be made into the internal network. After changing the state to disconnecting and prior to invoking INTNETTRUNKIFPORT::pfnDisconnectAndRelease, the internal network will use INTNETTRUNKIFPORT::pfnWaitForIdle to wait for any still active calls to complete.
The netfilter employs a busy counter and an internal state in addition to the public trunk state. All these variables are protected using a spinlock.
!OBSOLETE! - THIS WAS THE OLD APPROACH!
This secion contains a few sequence diagrams describing the problematic transitions of a host interface filter instance.
The thing that makes it all a bit problematic is that multiple events may happen at the same time, and that we have to be very careful to avoid deadlocks caused by mixing our locks with the ones in the host kernel. The main events are receive, send, async send completion, disappearance of the host networking interface and its reappearance. The latter two events are can be caused by driver unloading/loading or the device being physical unplugged (e.g. a USB network device).
The strategy for dealing with these issues are:
- Use a simple state machine.
- Require the user (IntNet) to serialize all its calls to us, while at the same time not owning any lock used by any of the the callbacks we might call on receive and async send completion.
- Make sure we're 100% idle before disconnecting, and have a disconnected status on both sides to fend off async calls.
- Protect the host specific interface handle and the state variables using a spinlock.
The ifnet_t (pIf) is a tricky customer as any reference to it can potentially race the filter detaching. The simple way of solving it on Darwin is to guard all access to the pIf member with a spinlock. The other host systems will probably have similar race conditions, so the spinlock is a generic thing.
The rediscovery is performed when we receive a send request and a certain period have elapsed since the last attempt, i.e. we're polling it. We synchronize the rediscovery with disconnection from the internal network by means of the pfnWaitForIdle call, so no special handling is required.