The most common solutions for email problems is to:
- Set the Sender Mail Header setting to an email that exists on the domain
- Install and configure the WP Mail SMTP plugin
As far as the confirmation message not appearing, it very well could be your theme or another plugin causing the problem. Try switching to a default Twenty Twelve/Thirteen theme and deactivating all other plugins to see if there is a conflict.
Let me know how that goes.
