So far I've encountered a couple issues, one which truly boggles the mind.

The first one is the one you're most likely to encounter after upgrading

 Fatal error: Call to a member function toHtml() on a non-object in <somewhere>/app/code/core/Mage/Core/Model/Layout.php on line 52

The solution is explained here:

http://rackspeed.de/forum/magento-faq-installation-and-updates/fatal-error-call-to-a-member-function-tohtml-on-a-non-object-layout-php-on-line-529-a-547

Basically edit your theme's layout/page.xml and change <block type="core/profiler" output="toHtml"/> to <block type="core/profiler" output="toHtml" name="core_profiler"/>

The next two are frankly amazing in that they ever made it into the release.

See here:

http://www.magentocommerce.com/boards/viewthread/195761/

To quote the finder of these issues 'Yes, preparing a shipment prepares an invoice instead..'