Fair enough... I do think that you should perhaps evaluate whether the activities you are planning are asking too much of him, or requiring him to sit for too long, and so on.
But I still say; if you have prepared something for them, and it's "craft time" for 15 min or so, then I think it's important YOU to be the one that is calling the shots. We don't just get to do what we want, when we want, all the time.... LoL. He may not want to do things exactly the way you are showing them, but he can still do SOMETHING.