You can buy clay bakers but not sure if there is a special finish on them to prevent chemicals in the clay from passing to the food assuming there might be something in flowerpots that isn't in bakeware. I have seen recipes for making cupcakes in the pots too.

Another option would be to use a small pot that the cupcake liner will fit into putting crunched paper under to lift it up if necessary - then bake the cupcakes/muffins in regular tins as usual and pop in the prepared flowerpots after the fact.