BTX Specifies the Format of the Case and Motherboard, in the simpliest terms the Motherboard is basically turned upside down to allow the processor to Be lower in the case and thus being cooler.
That wouldn't work. Since the PCI slots are now at the top of the board and the periphial connectors are at the bottom the opening on the back of the case wouldn't fit. You can get cases that support both ATX/BTX so whichever one you want you can get.