Let's break down what he's saying:
AKA "I'd forgotten about you/". It's going to be your fault he's taken you for granted.
AKA "I'm really busy with things that are more important things than you".
(his back out plan - he'll use this to blame you later "but I was really busy - you should have nagged me (because I'm more important)" (the last bit is unspoken of course) )
Nonchalance.
Basically this guy is playing games - there's nothing more to it, no need to resent it - it's how he's always got results so of course he's going to keep using it until it doesn't work.
So how to make it not work so he changes?
The safest response would be. "
Hi, I've made other arrangements for Saturday" - and if he gets shitty "
Well I've made other arrangements for Saturday" and if he phones "
I've made other arrangements for Saturday" he doesn't need any other information it's "
not his concern what you're doing instead" and giving anything more to him will just prolong any dispute as he quibbles over reasons, or tries to undermine your other arrangements. Sympathise if he's going to have trouble replacing you by all means (being unsympathetic will increase his resentment) but "
You've made other arrangements for Saturday" - see how undramatic it can be - see if you like it being un-dramatic (I bet you might like the results and I bet he doesn't like the lack of drama or the results) - but you'll have been totally reasonable and that's a great feeling.