You don't have it all wrong at all. Carl has no idea what it is to love someone or be a man enough to get over old shit that happened in the past and look forward to what could have been a wonderful life with Keeley and the boys.
Maybe it's time Keeley let the past with Carl go and look to her own future, where she might just one day meet a man who deserves someone as great as her.