To solve the \textit{Small Sample Size} (SSS) problem, the recent linear discriminant analysis using the 2D matrix-based data representation model has demonstrated its superiority over that using the conventional vector-based data representation model in face recognition ~\cite{Author:2DFDA}. But the explicit reason why the matrix-based model is better than vectorized model has not been given until now. In this paper, a framework of Generalized 2D Fisher Discriminant Analysis (G2DFDA) is proposed. Three contributions are included in this framework: 1) the essence of these '2D' methods is analyzed and their relationships with conventional '1D' methods are given, 2) a bilateral and 3) a kernel-based 2D Fisher Discriminant Analysis methods are proposed. Extensive experiment results show its excellent performance.