There are several terms used, depending on the size, number and where you live.
If there are lots of small ones (grapes, apples, oranges): if you are American, you would call them seeds and, if you are British, you would call small ones seeds and slightly bigger ones (especially citrus fruit) pips.
If there is just one large one (olives, cherries, peaches etc): if you are American, you would call it a pit and, if you are British, you would call it a stone.
You can use this NGRAM graph to experiment with British/American terms for different fruits. As the NGram graph shows, these are not hard and fast rules.
One notable exception is processed fruits like dates and olives. Olives are mainly grown, prepared and packed in non-English speaking countries. They supply to both US (141,000 tonnes per year) and UK (1,600 tonnes per year). It's easy to see from these figures why the suppliers choose to use the American term "Pitted Olives" on their packaging, even on products supplied to the UK. The import figures were obtained from here and here.