The "p" in "application" is unaspirated and the "p" in "apply" is aspirated since it is the first letter in the stressed syllable, am I right? When an unvoiced stop is the first letter of a stressed syllable, it should always be aspirated, correct? But for example in the word "excuse" since the second "k" is the second letter but not the first letter of the stressed syllable, I should pronounce it as an unaspirated sound, shouldn't I? I mean in the word "excuse" the first "k" should be aspirated and the second "k" should be unaspirated, right?
In English, ⟨p, t, k⟩ are aspirated when they're at the start of a syllable (usually stressed), so the ⟨p⟩ in pie, pool, paper is aspirated. However, when the ⟨p⟩ is at the end of a syllable or preceded by an /s/, it's unaspirated so in words like map, spy, spin etc., it's unaspirated.
As the OP pointed out, the ⟨p⟩ in apply is aspirated because it's at the start of a stressed syllable: [əˈpʰlaɪ] (or [əˈpl̥aɪ]1). By contrast, the ⟨p⟩ in application is unaspirated because it's at the end of the syllable: [ˌæp.ləˈkeɪ.ʃn̩].
Now excuse: it can either be syllabified as /ɪk.skjuːz/ or /ɪks.kjuːz/, the first ⟨k⟩ in both the pronunciations is unaspirated. If you think of it as being derived from the prefix ex-, then it can be analysed as /ɪks.kjuːz/2, in which case the second [k] is aspirated: [ɪks.kʰjuːz]. However, if you don't have a morpheme boundary, then the second [k] is likely to be unaspirated because ⟨p, t, k⟩ are unaspirated when preceded by an [s]: [ɪk.skjuːz]. Some speakers do have a morpheme boundary in the word excuse so they aspirate the ⟨k⟩ while others think of it as single morpheme and don't aspirate the ⟨k⟩.
In a nutshell, aspiration of the voiceless stops in words like sister, whisper, whiskey, excuse etc., is controversial.