First, the sentence isn't entirely correct, not because it's ungrammatical, but because it isn't precisely the way that HTML captions work. I'll explain how captions work in HTML, so we can be sure that you are saying what you want to say.
<caption> element only works with tables, not images. It has to be enclosed in a
<table> block (if you use it outside of a table block it is ignored), and defaults to being above the table (you can use CSS to reposition it below the table). The HTML
<figcaption> element (HTML5 only) is used with images. Here are two code samples, borrowed from w3schools.com. This is a table:
Here's an image:
<img src="img_pulpit.jpg" alt="The Pulpit Rock" width="304" height="228">
<figcaption>Fig.1 - A view of the pulpit rock in Norway.</figcaption>
All right. Now, your statement is saying that the caption is a short string of text positioned below the image or table. If that is what you want to say, it is grammatically correct. However, as I have explained, it is not factually completely accurate.
As Ustanak has observed in the comments, you may also say immediately followed by a table or image, but that has the opposite meaning: it means that the caption is on top and the table or image is underneath it. Again, this is the default behavior with tables. With images, it's simply a matter of whether you put the
<figcaption> tag first, or the
<img> tag first. (In the code sample, the caption is below the image, as is typical.)
These two sentences should make the difference clear:
Joe immediately followed Pete in the parade. (Joe was right behind Pete.)
Joe was immediately followed by Pete in the parade. (Pete was right behind Joe.)