It depends on the context (and even the orthography you are talking about):
Intonation and stress
You can use the word accent when you want to talk about the actual pronunciation of a word, i.e., the phonetic accent. For instance, one of the many definitions of accent in the Wiktionary is:
A mark or character used in writing, in order to indicate the place of the spoken accent, or to indicate the nature or quality of the vowel marked.
Or, in Merriam–Webster (emphasis mine:
a mark (as ´, `, ˆ) used in writing or printing to indicate a specific sound value, stress, or pitch, to distinguish words otherwise identically spelled, or to indicate that an ordinarily mute vowel should be pronounced
For example, you could say:
Your intonation of τόνος was wrong; mind the accent!
In this sense, it obviously only makes sense to talk about a diacritical mark as an accent if it actually indicates a phonetic accent in the language you are talking about. For example, in the Finnish orthography the caron does not have the role of an accent – it is only placed on consonants, where it indicates a completely different pronunciation. However, the caron is used to indicate the third tone in Pinyin and thus is an accent in that orthography.
Typography
In my experience (being a hobby type designer), type designers and similar prefer to use diacritical marks instead of accent. Because what is a phonetic accent in one language has some other function in the next one, this makes sense: For type designers, such pronunciation details do not matter and they cannot possibly keep track of every possible language anyway.
However, the typical speaker of English (and other languages with analogous words) is more likely to know the word accent than diacritical mark and thus accent has become a synonym for diacritical mark. Since a language is formed by its speakers, I would not consider it wrong to write:
Měšťáčtější is a word with a lot of accents, many of them carons.
To repeat one of the above dictionary definitions (with different emphasis):
a mark (as ´, `, ˆ) used in writing or printing to indicate a specific sound value, stress, or pitch, […]
Another evidence of the above and slightly different case is the use of accent in mathematics, which is the predominant term for diacritical marks placed on mathematical symbols – usually to distinguish them. See for example this page by the American Mathematical Society. This is completely detached from phonetics, but I would never use diacritical mark to talk about math accents with mathematicians.
