I run into this situation on a regular basis. You might not like the ants but they are a secondary problem. They eat the fungus that is causing the decay. Finding the source of the decay is more important for the health of the tree. If the decay is so widespread it is causing the tree to become hazardous, the tree may need to be removed. You need to have a certified arborist who specializes in plant health care look at the tree.
As an aside, ants do more to aerate the soil than earthworms. Most arborists do not consider them necessary to control.