We know "am/is/are/was/were" are "to be" verb but some people say those verb are "be verb".which is correct? I think "to be" sounds like infinitive.Am I wrong?
Infinitives in English are sometimes described as 'to infinitives' (eg "I like to run in the park.") and 'infinitives without to' (eg "The teacher made us run in the park."). These are sometimes called 'full infinitives' and 'base infinitives' respectively.
As you can see from the comments to your question, some people regard the stem "to" as not being part of the infinitive, whereas others regard it as being an integral part of the infinitive. I am not qualified to enter into that discussion. However, this is the reason that some people say that the verb is "to be" and others say that the verb is "be." So, "No", you are not wrong, you are simply inexplicably involved in a grammatical debate that has not yet been fully resolved.