Dan's answer clarifies the meaning of the sentence itself. If I understand correctly, your question is about the meaning in context, as part of this exchange from Fiddler on the Roof? I don't think this is really an English language learning question, but I will answer anyway since I cannot add a comment.
Tevye's beloved daughter is leaving for a faraway place to join Perchik, who was arrested as a radical, intending to marry him regardless of his situation. Tevye acknowledges that these matters are out of his hands (and rather are in God's), and the comment in your quote is a humorous one in the face of this serious but absurd situation.
Think of it like: "Well! It sounds bad, but this is all fine, because they probably arrested a rabbi or two along with Perchik. My daughter will be able to go to him and get married without any problems."
It is not necessarily true that any rabbi were arrested, and Tevye knows that everything will not necessarily be fine. But he's humorously acting as though that's the case, while acknowledging that it's in God's hands what really happens.