Just because you're rich doesn't mean that no one wants to kill you. A refugee is someone who's life is in danger. And if they were rich in their own country they were likely hard workers. Doesn't this make them valuable?
As a fellow decendant of early settlers, pre federation, how do you think the locals felt when we came? There's certainly no Aboriginals left where I grew up, maybe a few genes floating around, doesn't take much to figure out where they went.
Immigration and asylum for refugees are seperate issues as is population growth.
Bigger questions need to be answered, like, why are Australians (as are people in many other countires) Xenophobic (fear of difference)? Is a strong economy worth the population growth? (With our currnet systems they are directly linked). Do we have an intrinsic right to keep fellow humans off this land?
I agree Australia is facing infastructure problems, especially in Melbourne and Sydney but this is related to population growth and a lack of preemptive infastructure investment. If the answer is we want space and not to have people move here in search of a better life or to escape persecution then the answers lie else where. Really its irresponsible of politicians to use "boat people" as a scape goat or a scare campaign to distract us from the important things.
Most people who immigrate do so for a better future and work really hard. I know I wouldn't consider studying full time and driving a taxi at night to pay my fees as is a common story for new Australians.