Remove the jockey wheels, one at a time so you don't mix them up (they are different). You'll need either a 3mm allen key or a 8mm spanner, depending on model. If there is a thick buildup of muck.se a flat-blade screwdriver to scrape it off. Pop the bushings out, give them a good clean & apply a little bit of grease before putting them back together. The top (guide) jockey wheel has an additional steel sleeve pressed into the wheel to extend its life - when it gets dry it can get very squeaky as you have steel-on-steel, so give that one a really good service. Check the condition of the bushings. They should be a reasonably close fit, allowing only a little bit of play. The inner (loose) bushing in the guide wheel will wear over time. Check for a flat spot on one side, with a sharp lip at each end in bad cases. If you can observe these features, the guide pulley will need replacing. You can buy these from your LBS.
While the jockey wheels are out you can give the inner faces of the derailleur cage a really good clean.
A rag wrapped around a small screwdriver is really good for getting into the tight spots to clean the goo out.
Just ride the bloody thing and enjoy yourself!