There are two options for you here depending on your requirements.
If you want to edit the 'carousel' layout of the aforementioned module, you don't need to rename the file. Simply create the override (which, as it seems, is created correctly), without renaming the file, and make your edits in the override. For CSS changes, you will again have to create an override for the css file for this module from 'media/mod_sellacious_specialcatsproducts/css/style.css' to 'css/mod_sellacious_specialcatsproducts/style.css'.
The second case will be a little complicated. If you want to create a whole new layout, you will have to create a new '.php' file under 'html/mod_sellacious_specialcatsproducts', just like you have already done by renaming the original 'carousel.php'. Additionally, for this to show up in module layout option in joomla administrator, you will have to edit the 'modules/mod_sellacious_specialcatsproducts/mod_sellacious_specialcatsproducts.xml' file. Find a '<field name="layout"' in that file, and change the value of 'type' attribute from 'list' to 'modulelayout'. Now you will be able to select your new layout from module options in Joomla! administrator.
I hope I have cleared your doubts. Do ask if you have any other doubts.