Easy. You use a custom application context for your unit tests, or you don't use one at all and you manually create and inject your beans.
It sounds to me like your testing might be a bit too broad. Unit testing is about testing, well, units. A Spring bean is a pretty good example of a unit. You shouldn't need an entire application context for that. I find that if your unit testing is so high-level that you need hundreds of beans, database connections etc., you have a really fragile unit test that is going to break on the very next change, will be hard to maintain and really isn't adding a lot of value.