This is a difference of past perfect simple and past perfect continuous tense.
I can't find the pictures I had looked at earlier.
The above sentence uses the past perfect simple tense. It means that the event or action (in this case, looking at the pictures) started in the past and has been completed. It puts the focus on the completion of the activity/event.
I can't find the pictures I had been looking at earlier.
The above sentence uses the past perfect continuous tense. It means that the even or action (in this case, looking at the pictures) started in the past and is ongoing to the present. It focuses on the activity/event itself (or its process), rather than its temporality.
In the context of these sentences—you've lost the pictures and thus cannot be looking at them anymore—the past perfect simple tense is more appropriate. The past perfect continuous tense doesn't make sense, because you cannot be continuing to look at pictures that you can no longer find.
It also doesn't make sense to use the past perfect continuous tense when you've appended the temporal referent "earlier" to the sentence, because that implies an action that completed at some point in the past.
You aren't focusing on the process of looking at the pictures. You only bring up that you looked at the pictures to clarify which pictures it is that you cannot find.