Clay pots in the oven should be totally fine as long as it is heated up gradually, just to be safe. If put in from cold state to hot then they might shatter. I used to do ceramics so before they become pots, clay is first a soft dough-like substance and then it gets fired (baked) at a high temperature to make it permanent/hard. At way higher temperatures than what our ovens can do. I would suggest using plain ones, the terra cotta ones, or ones without any colored glaze or paint. Certain glazes can leach into food. I would bake the muffins in liners for sure. PLayfelt's suggestion is also good, to bake first then pop in prepared flowerpots afterwards.