A rotary engine is quite unique both in operation and block dimensions. Since Mazda was the only car company making rotary engines, the only other choice would be a reciprocating piston engine. You're not going to be able to fit any other engine within the RX8 engine bay easily or without a lot of modifications.
There are conversion kits available for everything from Mazda to big V-8s, but they all require lots of work and investment. If your engine is still good, just learn to drive it properly and it will last for a long time. (Search "Rotary Engine forum" and join one to find out how to make your car last for many years.)
Be advised that there is a recall at the moment for a problem with a fuel pump seal on all "8's" made between 2004 and 2008. It is because that seal leaking could cause a fire, although that is just media sensationalism. (Make it look dangerous and people read your articles!) None have caused any fires, but they do make your gas mileage go down! Talk to your local Mazda dealer and they will tell you when to bring it in to be replaced. (a recall means you do NOT pay for the repair.)
Years ago NSU RO 80's that had the dreaded 'Wankel' engine were converted to Ford piston engines, but considering the low bonnet line of the RX8 I would think it was doubtful any conventional engine could be substituted.
unless you add another Mazda engine into there, you will likely need to do lots of modifications to get sh*t to fit. you have the space for it though. the rotaries are notoriously unreliable when people like you own them and dont run/service them properly. make a build car! swap an inline 6 into there!
Hi None as the body work was designed around the engine.