Fill in is used when you tell someone to enter something (one field) on a form.
Fill in your name here.
It could possibly be more than one field, but generally not the entire form.
Fill in your name here, and your phone number over here.
Fill in means to supply something that's missing. So you fill in the blanks on a test, for example, or you can fill in a triangle with a color (change it from an outline to a solid triangle by coloring inside it), or you can fill in (substitute) for an absent colleague at work. And you can fill in a form because you're supplying missing information.
Fill out is generally used when you tell someone to enter all the fields on a form of more than one field. Example:
Please fill out this form.
Fill out means to complete by supplying requested information.