Exactly what I'm thinking. The high cost of trying to get those houses insured will be the final reckoning. Or, finding out that there's no home insurer for an area.
I remember something about some insurance companies no longer wanting to do business in Florida, for example. I imagine there are other areas frequently hit by catastrophic weather, beach front erosion, or wild fires, where the same thing is happening.