Databases
In general, I think it's easier to use SQLite as your database for development (and switch to MySQL or PostgreSQL when you deploy).
But, most Cake tutorials assume you're using MySQL, and that comes with the XAMPP package described below and is easy enough to install on Ubuntu. So, my best advice is to, just work with MySQL.
In CSIL
Since PHP requires a full-blown web server (as opposed to having a mini development server), it will be quite difficult to work with in CSIL.
On Ubuntu
If you are working with Ubuntu, it's usually preferable to install the Ubuntu packages for the software you need (not manual installations, getting Git repository code, etc.). That way, the package manager will get all of the required libraries for you and everything will mostly just work. For Cake, that probably means:
sudo apt-get install libapache2-mod-php5 php5-mysql cakephp mysql-server sudo a2enmod rewrite
On Windows or Mac
You will likely need Apache installed on the computer where you are developing. The XAMPP package is probably the easiest way to do this: it provides everything you need.
You can then install CakePHP as described in its tutorial.
Apache Config
You will need to configure Apache so it finds your app. For development, you can probably just put the app at your server root:
DocumentRoot /path/to/my/work/app/webroot <Directory /path/to/my/work/app/webroot> AllowOverride All </Directory>