There is a difference.
"Break up" means to separate or divide. While it is true that you can divide things into multiple parts, such a process normally involves splitting one thing into two, then two into three etc. Cutting a cake is a good example - you may divide a cake into three equal parts, but you would do so first by cutting out one piece and then dividing the remaining piece. Things that split into two can be said to "break up", such as a relationship break-up. "Breaking up a vehicle" at a breaker's yard would involve removing one component at a time, not smashing it into fragments.
"Break down" describes the entire process of something being separated into component parts. It can also be used to describe the process of breaking something into the most basic elements, such as a chemical compound which is "broken down" into elements. I would not expect to hear this expression in connection with something that contained only two parts - it is normally used to mean the breakdown into multiple - three or more - parts.
In your example it is difficult to know which should be used, if any. This is because it seems to be talking about reducing the size of something, not breaking it up or breaking it down:
And what you've done is break this matrix up into half the size.
If the matrix is half the size, what has happened to the other half? If the intended meaning is that you had a matrix which you split into two matrices half the size of the original, then "break up" seems the right idiom, even though the sentence isn't perhaps phrased correctly. If on the other hand you have just removed data from the matrix to reduce the size, you haven't really broken it up or down at all - you have reduced it, or filtered it.