- what is the best practice ?

This really depends on your organization, what works best for you.  I would put some effort into designing governance around this with your platform team.  Some folks will encode certain functions in the name etc.  However, if you are asking if 'how' to change it then you are generally safe to just change it in the 'name' field (with caveats see below)  GlideFather also gave a pretty great technique where you can save the historic data via auditing.

    - whether renaming automatically roll across configurations/references/hard coded settings etc  

Yes and No,  if the references are by 'sys_id' you will be just fine.  Any hard coded references to the name itself or any conditions built off a hard coded value that expect a specific string value will fail.  For example if you had a piece of code 

<sys_id> === 'my_group_name'

 Expect this to fail.  However, good news is you can use the 'Code Search' tool within your instance to find these occurrences and clean them up (View Pradeep Sharmas Article: https://www.servicenow.com/community/developer-blog/using-code-search-to-make-developing-on-the-plat... )

    -  impact on current workflows or other configurations

See the above mentioned sys_id, everything is tied to the sys_id

    -  AD integration impact for user/group import.

Anything that is being integrated into AD should be changed in AD as AD is the source of truth.  If you are using something like a transform map, you would be coalescing on the AD GUID which your groups sys_id would be related to the AD GUID so you would just be changing the label everything under the hood will still function.  I always encourage reviewing your design/process/governance to find a better way rather than just lifting and shifting.

    -  impact of existing /closed REQ/RITM/SCTASK for the group

Depending on your implementation and level of customizations this would be trivial as the sys_id is referenced for the group and then retrieves the 'display name'.   This wouldn't create orphans OOTB unless you had some customizations.

    -  Notifications set up

Notifications referencing the group via reference fields will continue to function. However, any encoded queries, scripts, or notification conditions using the group name should be reviewed.

    -  Best Practice/recommendation

I don't know your architecture/governance/design, however here are some fairly basic guidelines I would recommend based on some practical experience I have: 

1. Verify how the AD transform map coalesces groups.
2. Search for scripts or configurations referencing the group name.
3. Rename the group in AD (source system).
4. Allow the next group import to update ServiceNow.
5. Validate assignment, notifications, and integrations.

     - renaming vs creating a new group ? 

Renaming is preferred when only the name changes. Creating a new group should be done only if there is a structural or ownership change (different team, responsibilities, or membership)

OR

If you inherited an environment that has not had governance, had neglected systems administration or needs to be unwound from a high amount of customization

     - orphan records

Previously, answered

     - auditing etc 

I like GlideFather's recommendation