It's always risky to second guess the meaning of the lyrics by any poet/songwriter. What seems most obvious is often not the actual meaning. Case in point, Springsteen's "I'm on Fire" which seems to be about passionate, unrequited love but, he explained, is actually about his anger at his mother's having to borrow money so her children could have school clothes and Christmas presents.
Since none of us can see into RDJ's memory and heart, we really don't know what he was writing about. The human heart can be broken by many people and events other than a lost love.
My take on the song is that RDJ's heart had been broken at some time, by some person or some event, but it's not for us listeners to know the, probably very painful, details.
I dont believe Im second guessing at all. His lyrics like his speech is communicated in fables to be interpeted as the listener see's it. Merely Im assessing the time line when it was written and what he said about it to have the feeling it was about ending a 7 yr affair w SJP