If you need to delete a task from the taskboard, simply click on it to bring up the "Task Details" pane on the right hand side. Click on the cog icon and select "Delete", then confirm that you wish to delete the task for good.
This applies to the Feedback and Archive tabs too.
