Babylon, the nation, was an enemy of Israel. While God used it as a tool of chastisement for the chosen people, that did not absolve it of its part in the calamity. And prior to the end of the 70 years, that did happen. By that time, Babylon was not its own nation. It was overtaken by the Persians who were the ones that allowed the return. So in that sense, Babylon is judged for its part in the ransacking of Jerusalem and Judea. The Babylon they returned from was not the same Babylon that took them captive.
And when looking back at the nation that reached out to captivate so much of the area — much more than just Jerusalem — that nation was corrupt and was punished. And if you want a metaphor concerning a system that takes over everything in its path and consumes it, Babylon is a good metaphor. They took the property of their conquests, killed many of the people, and enslaved much of the rest.
But is this reference to Babylon in Revelation being cast upon some aspect of the church? If so, the presumption generally is that it is on the RCC (at a minimum). But if that is true, then I would expect that something sounding like the charge against Babylon there in chapter 18 would be foreshadowed somehow in the 7 letters to churches. In that case, the nearest thing is the existence of a woman (Jezebel) within Thyatira that is ordered to be cast out. The woman is not the church. And the church is not referred to as committing adultery.
I think this is one of those cases of over-"simply"-fying words and phrases into a single thought. Like all references to leaven are bad. Surely the nation of Babylon was not a positive thing. And this "Babylon the Great" is not someone to bring home to Mama. But nothing in those statements casts the aspersion of "Babylon and her whore daughters" onto Christianity in general, or necessarily onto the RCC (said with reasonable trepidation).
Last, in one way of looking at the original captivity, they were already out of Babylon before they began to return. At that point, they were in Persia. No. They were still in the area known as Babylon, but Babylon was no longer an independent country. It was now ruled by Persia. So, in a sense, they came out of Persia to return to Jerusalem. And those that stayed did not stay in Babylon, but in Persia.