**Defy** in this case is taking the slightly less common meaning of "challenge" or "dare" (third at [this link)][1] rather than "resist" or "disobey." This usage of defy is always structured "defy [someone] to [do something]."

The "large number of people" refers to the corps. 

The speaker is saying that if the entire corps (i.e., the whole army, or a large division of it) joins in the activities, the authorities will not dare to upset the whole group by declaring their activities illegal. 

  [1]: https://www.merriam-webster.com/dictionary/defy