This paper investigates the optimal design of precoders or decoders
under a channel inversion criterion for multi-user (MU) MIMO filterbank
multicarrier (FBMC) modulations. The base station (BS)
is assumed to use a single tap precoding/decoding matrix at each
subcarrier in the downlink/uplink, resulting in a low complexity of
implementation. The expression of the asymptotic mean squared error
(MSE) for this precoding/decoding design in the case of strong
channel selectivity is recalled and simplified. Optimizing the MSE
under a channel inversion constraint, the expression of the optimal
precoding/decoding matrix is found. It is shown that as long as the
number of BS antennas is larger than the number of users, the optimized
precoder and decoder can compensate for the channel frequency
selectivity and even restore the system orthogonality for a
large enough number of BS antennas.