Having allies is a good thing, especially trade allies. They will protect you to protect their business, so you can develop with ease. The alien race will not be angered with your decline, but you have lost a good trade partner.