There are numerous tutorials for setting up web servers running LAMP across the internet. I'm going to provide the same instructions, but customized the way I have found in my experience to be the easiest to manage and maintain in the future.
Step One - Install and Configure Ubuntu
You can either use the dedicated server disk image, or you can start with whatever Ubuntu or Kubuntu installation you already have. I won't get into the specifics of the operating system install, as there are some pretty good instructions at the Ubuntu site.
When I refer to configuration, this means setting up any tools you will need. For me this would include ssh (for desktop installs), htop, lynx, and nmap.
Step Two - Install MySQL
MySQL will be your database server. This server will store and provide access to all of the data your applications are going to want to save and persist.
During the installation of the MySQL server, you will need to enter a root password for the database server. This password will be important for accessing and leveraging your database later in the configuration.
Step Three - Install Apache2
Apache is your web server, this is the piece of software that is responsible for accepting and routing requests, and then tasking other programs with creating pages.
Step Four - Install PHP5
This command will install PHP, integrate it with Apache, and install the module that will enable PHP to talk to the database.
As soon as this command has completed, you have an installed and running LAMP server. If your LAMP is behind a router, you may need to configure NAT or Port Forwarding on your router (this will only apply to users attempting to run a server from home). You will also probably want to associate a domain name with your new LAMP. Instructions for doing this are beyond the scope of this tutorial. In order to access your LAMP with a browser, you will need to know what IP the computer has. You can use the ifconfig command to see the IP of the machine, or if it is behind a firewall, I recommend typing "lynx whatismyip.com", which will show you your public-facing IP.
Step Five - Check Apache Configurations
The default configuration will place your hosted files in "/var/www/". The first steps I take with any server is to setup name based virtual hosting. This allows you to host multiple sites from the same IP. When a user attempts to access your server, the site they are shown will depend on the domain name they used to access your site.
Ubuntu puts all of the apache configuration files in /etc/apache2/. You will find that the standard configuration file called httpd.conf is empty in Ubuntu. This is intentional. The "Ubuntu" way of doing things is to put any site configurations into /etc/apache2/sites-available/, and then use the command "a2ensite
Step Six - Install phpMyAdmin
phpMyAdmin is used as a tool to manage your MySQL databases. This is the equivalent to MSSQL's Management Studio and pgSQL's pgAdmin. Installing it will use PHP to provide a simple interface for managing the structure and contents of your databases and tables.
Step Seven - Install AWStats
AWStats is a great tool that was written using Perl to process log files into a miniature website consisting of HTML, CSS, and images that graphically represents and organizes statistics about your users. AWStats, or any local log processor, is different from a tool like Google Analtyics because it will show you all of the traffic on your server, not just page traffic. This means that if people are hotlinking to your images, or downloading other documents you host on your site, you will be able to see these in AWStat's logs, but not in Google Analytics.