I had a parent babywear their little one all the time as she couldn't deal with his crying. As a mom, I understand that it's tough but at the same time ~ these little ones need to learn how to self soothe as there isn't going to always be someone there to carry him around. This mom would carry him around to fall asleep too!
I agree with Alphagetti that you don't want to leave them for too long as your trying to get the little one to trust you..having said that I am a tough love type of person too. I always reassure the little one that I'm here for them by having them sit with me and rub their backs and I for sure give lots of cuddles but no babywear for this provider. Some providers swear by it and it's all about what your comfortable with really.
Some little ones take longer than others to adjust. You just have to know your own breaking point and what your able to take. It can get really tough listening to a little one cry. I'm not sure how long this little one has been with you. Casual care can take longer to adjust too
Good luck!