The two objects were arranged (orthogonal | orthogonally) to each other.
Does "orthogonal" relate to the verb, or to the two objects? In other words, do I need to use the adverb or the adjective?
If you restructure the sentence, you could use either form, but in its current form you should use the adverb because you are describing how the objects are arranged, not simply describing the objects themselves. "Orthogonally" answers the question, "How were the objects arranged?".
The two objects were orthogonal to each other.
The two objects were arranged orthogonally to each other.
The two objects were arranged such that they were orthogonal to each other.
It's the adverb you need. Orthogonally. Although orthoganality implies arrangement, so you could just say the objects were orthogonal (to each other).