There is no grammatical reason why you can't repeat words in a sentence, but there may be reasons why it's not good writing style.
Usually, if there is no need to repeat the same word, it's fine to omit it if the meaning is obvious. In such cases if you do repeat the same word then the reader may assume it's for deliberate emphasis.
Is it too much to ask you to clean up after you eat? It's simple: pick up your dishes, take your dishes to the sink, wash your dishes, then put your dishes in the drying rack.
In your example, the use of "shape" helps the reader understand what you are referring to, but you don't have to repeat it quite as often as you do. Once you specify you're interested in shapes, you don't have to specify it again:
This shape is neither a star nor a triangle.
Alternately you can make a list of the characteristics of the shape. When making lists, all of the elements should be of the same form:
This is neither a star shape nor a triangle shape. It is a square shape.
Alternately, you can list the characteristics as adjectives rather than nouns:
This (object) is neither star-shaped nor triangle-shaped. It is square-shaped.
A real-world example:
Use caution when inserting the card into the mainboard. The lowest-number pin should be toward the left side of the board, not the right side.