I think you probably already know your limit has been reached. It's not always easy to end relationships, even when they are becoming toxic. You really do have to look after your own interests before those of friends who just couldn't be bothered organising their lives or finances.
Remember the fable about the ant and the grasshopper? Stop letting the grasshoppers of this world take liberties!