From a practical ESL perspective, I'd say it doesn't matter if 'accuse people to be racist' is grammatically correct. My reason is that it's hardly ever used. People do (probably incorrectly) say it sometimes, but it's rare, and because of that, even if it is correct, then many (if not most) people will think it's incorrect. You should use 'accuse of being racist' instead (since that is definitely correct and common).
As for a non-ESL perspective, here's my analysis:
It doesn't make grammatical sense, in my opinion to 'accuse someone to be racist' (unless as another person stated, you are meaning that your intention is to do it in order to become racist yourself) although it is terminology I've rarely heard before as a native speaker. Let me explain how it doesn't make sense: 'to be' is an infinitive, or in other words, an unconjugated verb (like 'estar' and 'ser' are infinitives in Spanish). 'Be' is not an infinitive without 'to' in English. 'Of being' is definitely not an infinitive (and you'd never say 'of be'). 'To' is not a preposition here (while 'of' is). People like to think it represents intention, but you can't use it like that if it's not an infinitive, and you can't use it with gerunds / verbal nouns / transitive verbs (i.e. You'd never use it with a conjugated verb; 'be' is the only form for it). Prepositions, like 'of', on the other hand, can be paired with other forms (like 'of being'), where appropriate. You can try to give meanings to 'to', but that doesn't really change that 'to' has no meaning without the verb (which is why, I suppose, people complain about split infinitives), even though full infinitives (e.g. 'to be') have meaning (without tense). Try putting some other infinitives in its place and see if it sounds like it could be grammatically correct for the other meanings:
- to have: accuse people to have racist (this makes no sense whatsoever)
- to like: accuse people to like racist (this also makes no sense whatsoever)
- to want: accuse people to want racist (ditto)
- to feel: accuse people to feel racist (sounds slightly more like something people would say, but no, it still doesn't make sense unless you're accusing them in order to feel racist yourself, which is grammatically correct; another answer already mentioned this context, which also applies to other verbs, such as the one at hand)
My conclusion is that unless 'to be' is an exceptional verb pairing for 'accuse' then it's probably incorrect. The hint that it might be uncommonly correct is that we have words like 'consider', which another answer already mentioned, which can be used with the infinitive 'to be' in this fashion: e.g. 'I consider him to be a racist' is not considering him in order to become racist yourself, but it really means that you think the person is racist (and it is grammatically correct and expected usage). It's possible that 'accuse' can be (or could once have been) used this way, as far as I know, but it would still be ambiguous and confusing to use it this way, since it has another, almost opposite meaning, and hardly anyone says it. However, I'm confident that almost every native English speaker would know your intention, even if they believe it's incorrect.
It is much more grammatically correct (or at least less controversial) to 'accuse someone of being racist'.
As to whether you should pluralize 'racist' as 'racists' when you pluralize 'people', that depends on what you mean by 'racist'. Are you using it as an adjective (like a state of being) or a noun (like a person)? If you're using it as an adjective, then of course you don't pluralize it. If you use it as a noun, then you would want to pluralize it. So,
- accuse people of being racists (this is correct if racists is a noun)
- accuse people of being racist (this is correct if racist is an adjective)
Wiktionary says in the legal sense, 'of' is the proper word to follow accuse. This seems to imply that people might be tempted to use another word, and that it would be incorrect. I don't know their source for this information, however.
Dictionary.com lists several contemporary usage examples, and none of them mention 'to be', but most mention 'of'. That's a good indicator that you shouldn't normally (if ever) say 'accuse to be'.
It should also be noted that 'to' in 'to be' should not be looked at as a preposition, but rather as part of an infinitive. Even if it were correct to use as you mentioned, then it's not really a drop-in replacement for 'of', but it changes the grammatical structure altogether.