http://www.wordpress.org - The WordPress software is available here, along with links to find a web hosting service for your blog.
http://www.wordpress.com - The business side of WordPress. If you want to experiment with WordPress without investing any money, you can quickly set up a WordPress blog for free.
http://www.wampserver.com - WAMP Server is a free software package that provides the Apache, MySQL, and PHP software needed to run WordPress on a Windows computer. Installing this software will allow you to experiment with WordPress on a desktop or laptop computer.
http://codex.wordpress.org - WordPress Codex is the online manual for using WordPress. It contains reference material for everything from getting started with WordPress to creating themes and plug-ins.
Akismet - Spam in post comments is a very common problem for bloggers, but Akismet will analyze any comments you receive and filter out the spam. Akismet is installed with WordPress, but it does need to be activated.
Google Analytics for WordPress - Let Google track all your users' interactions with your website so you can see what content attracts them and what content loses them.
W2 Super Cache - Speed up your website by caching static HTML pages for your website so your web server doesn't need to rebuild each page every time it's requested.
HTML and CSS
http://www.w3schools.com - W3 Schools provides tutorials and reference material for learning HTML, CSS, and PHP.
Google Webmaster Tools - Great help in understanding how Google indexes your website.
The ThemeShaper WordPress Theme Tutorial: 2nd Edition - Excellent walk-through for building a theme from scratch using a simple HTML page structure as a starting point.