Let's take incident_task. I assume the update is always the 'comment' field? Metrics will work here if your ticket only has one assignment group. If not you only have the assigned_to to report on.

You then could create a DB view to the sys_user_grmember, but that will only work if you don't have users as member of 2 groups. Create a report on that db_view and you can see the average per assignment group. 

Another possibility is to create a scheduled report following this article (only not for cmdb_ci, but for your own tables). You can 'catch' every change of the comments field and include the assignment group in the report when the change occured. You will need some smart reporting to get the duration though. 

Assuming PA isn't an option (because that would take some time, but could be done if you have premium): if this is really important to know, you could consider to create a table for this, and copy assignment group, duration, comments, times, dates, etc to that table to report on. 

At one time I have seen the SLA table misused for this, but most of the time I'm just wondering what value this is going to have in running the team(s). It won't improve quality of responses to the end user, that's for sure.