Trivial has nothing (inherently) to do with size
Something is trivial if it is insignificant or unimportant within the relevant context. Often, this will mean that it isn’t big either, but not necessarily.
For example, a person might have a visual impairment, say, a slight short-sightedness. If this doesn’t require correction for everyday living, it’s trivial. But, if it prevents you from pursuing your life’s ambition to be a fighter pilot, it’s not trivial.
Alternatively, consider a software fault in a motor vehicle. Let’s say it only happens once every 50 hours of operation. If it’s in the entertainment system, it’s trivial. If it’s in the braking system, it’s non-trivial.
In the context you are using, “Non-Trivial amount of work”, the context is clear that you are talking specifically about the amount of work, rather than, say, the difficulty or importance of the work. It doesn’t mean (necessarily) a large amount of work, it means enough work that it will have a negative impact on other tasks.