This is more of an informal way of speaking. To "burst" implies (in this context) :
to be full as if to the bursting point with pride.
This simply means that the speaker is super proud of his hardworking girlfriend and his feeling of pride towards her is very high, and for the same he has used "burst" to describe the level of pride he has towards her. This feeling fills him up and he feels like he may explode (metaphorically, of course).
"To marry the sh*t out of her..." simply means to marry her and keep her very happy. This is a common usage, especially in American English. Similar sentences would be:
I'm going to scare the sh*t out of them this Halloween.
This is just to create an emphatic impact. There is specific meaning and sh*t has got nothing to do with the actual concept.
Another usage is to replace 'sh*t' with 'hell' or 'bejesus' :
I'm going to scare the hell/ bejesus out of them this Halloween.
This simply means : 'to make one feel extremely frightened'.
These are highly informal and never to be used formally. It may come off as very rude or offensive to many people.
NOTE: 'scare the sh*t out of someone' is commonly used. 'Marry the sh*t out of her' is an analogy to the same, but isn't used widely (it is content-specific). There are many ways you can recreate it, based on the context.