Introduction
Well admittedly the title of this post is kind of misleading... anticipating the result here: You can't rename a resource group. Not yet!
The following is a statement from a Microsoft representative that gives some insight to why the situation is what it is (source):
Renaming resource groups is treated like a resource move behind the scenes. We will address this feature request once resource move is broadly supported across the various services. We would not want to release a rename feature where some resources ended up in the “renamed” resource group, while others were left behind.
This means Microsoft is already working on a solution to make this possible.
In the meantime we have to be satisfied with a workaround, which involves creating a new resource group and then move all resources to the new place.
This poor man's rename is only possible if all the resources you'd like to move do support the move operation. For a list of which resources support move, see Move operation support for resources
Create a new group
az group create --name rg-my-new-group
Move the resources
az resource move \
--destination-group rg-my-new-group \
--ids <your space seperated resource ids>
Or via PowerShell
Get-AzureRmResource -ResourceGroupName <sourceResourceGroupName> | `
Move-AzureRmResource -DestinationResourceGroupName <destResourceGroupName>
Conclusion
The workaround we have seen is not applicable in all situations - so think twice when naming your items.
Did you know that Microsoft has an article covering best practices about naming and tagging? Here is the link: Recommended naming and tagging conventions