Open Visual Studio Command Prompt (2008), (2010), etc…
TFS 2008 and below:
TFSDeleteProject /server:ServerName ProjectName
TFS 2010 and above:
TFSDeleteProject /collection:http[s]://Servername:port/tfs “Project Name”
Warning: Deleting a team project is an irrecoverable operation. All version control, work item tracking and Team Foundation build data will be destroyed from the system. The only way to recover this data is by restoring a stored backup of the databases.