Is this rude---------------------?

I went to see a friend last Sunday. I returned home later that evening. I seldom screen my calls. One friend called but did not leave a message to have me return his calls. When he does call, It's like 930 or after 10p. I have told him this numerous times to not call me after 930. But he does not get a clue. I'm getting of tired of hearing about his wife's and son's health issues. We used to be close, he married in 2007. I do not call or visit him as I used to, He works weird hours.