The original RCX that came in Robotics Invention System 1.0 and later LEGO Education sets has an adapter socket into which you can plug an adapter to power it, instead of using batteries.

Here it is:

If you look closely, you’ll see above the port it says:  9-12 V ~

So that means you can use any adapter that produces between 9 and 12 volts and has a plug that fits? Unfortunately, No.
That ~ symbol at the end means AC, alternating current. Most adapters you’ll find will output DC instead of AC.  The circuitry behind this port is quite good in that it will even appear to work fine on a DC adapter for quite a while, but you run a serious risk of overheating it, and there have been reports of burnt out RCXs!  So make sure you get an adapter with AC output.

Here is a LEGO adapter* I had lying around. How to tell it is the correct one?

At OUTPUT it says 10V, so that is okay, but there’s a different symbol next to it:
This symbol means Direct Current: DC.  So not good, it’s not AC.
* Adapter merely used as an example, this particular LEGO adapter has the wrong plug anyway, it doesn’t fit the RCX port.