I would like to understand how to refer to a given matrix in my English sentence.
The matrix, M, refers to model A.
M matrix refers to model A.
Which one is correct?
The preferred phrasing is
The matrix M refers to model A
without commas, but with appropriate typesetting if possible. (Boldface upper case is common, but see also Matrix (mathematics) - Notation on Wikipedia).