Categorical variable selection in risk modelling

In financial institutions, categorical features appear quite often in credit datasets and in compliance models, for example, features related to clients’ risk profile.

Traditional feature selection methods (e.g. statistical significance, recursive feature elimination, LASSO) do not work well with categorical features since these methods would retain certain levels and remove other levels of the same feature. The Group Lasso approach has shown to be more stable in terms of variable selection but displays shortcoming in terms of predictability. Instead, for a given feature, would it be more appropriate to devise a method that aggregates neighbouring levels in bins in order to get a feature representation space that would better scale with the output?

Because of the numerous ways to represent categorical variables and to select which variables are of importance we ask what are the most appropriate methods for improving categorical feature selection?